Jul 28 17:08:30.792502 ip-10-0-133-215 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 17:08:30.792512 ip-10-0-133-215 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 17:08:30.792519 ip-10-0-133-215 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 17:08:30.792729 ip-10-0-133-215 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 17:08:40.988767 ip-10-0-133-215 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 28 17:08:40.988785 ip-10-0-133-215 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot e10503a87c33450594ca19674fcf9918 --
Jul 28 17:10:43.497617 ip-10-0-133-215 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 17:10:43.860150 ip-10-0-133-215 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 17:10:43.860150 ip-10-0-133-215 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 17:10:43.860150 ip-10-0-133-215 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 17:10:43.860150 ip-10-0-133-215 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 17:10:43.860150 ip-10-0-133-215 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 17:10:43.861799 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.861742    2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 17:10:43.867721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867708    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:10:43.867721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867721    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:10:43.867782 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867724    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:10:43.867782 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867728    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:10:43.867782 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867731    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:10:43.867782 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867734    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:10:43.867782 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867736    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:10:43.867782 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867739    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:10:43.867782 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867742    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:10:43.867782 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867744    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:10:43.867782 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867747    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:10:43.867782 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867751    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:10:43.867782 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867753    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:10:43.867782 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867757    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:10:43.867782 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867759    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:10:43.867782 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867762    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:10:43.867782 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867765    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:10:43.867782 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867767    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:10:43.867782 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867770    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:10:43.867782 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867772    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:10:43.867782 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867775    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:10:43.868212 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867778    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:10:43.868212 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867780    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:10:43.868212 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867782    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:10:43.868212 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867785    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:10:43.868212 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867788    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:10:43.868212 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867791    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:10:43.868212 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867794    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:10:43.868212 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867796    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:10:43.868212 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867801    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:10:43.868212 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867804    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:10:43.868212 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867807    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:10:43.868212 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867810    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:10:43.868212 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867813    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:10:43.868212 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867815    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:10:43.868212 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867818    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:10:43.868212 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867820    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:10:43.868212 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867822    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:10:43.868212 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867825    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:10:43.868212 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867827    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:10:43.868721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867830    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:10:43.868721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867832    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:10:43.868721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867835    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:10:43.868721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867838    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:10:43.868721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867840    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:10:43.868721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867843    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:10:43.868721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867845    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:10:43.868721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867848    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:10:43.868721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867850    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:10:43.868721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867853    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:10:43.868721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867855    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:10:43.868721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867857    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:10:43.868721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867860    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:10:43.868721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867862    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:10:43.868721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867865    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:10:43.868721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867868    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:10:43.868721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867872    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:10:43.868721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867875    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:10:43.868721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867877    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:10:43.868721 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867880    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:10:43.869205 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867882    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:10:43.869205 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867885    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:10:43.869205 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867887    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:10:43.869205 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867890    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:10:43.869205 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867892    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:10:43.869205 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867895    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:10:43.869205 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867897    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:10:43.869205 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867900    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:10:43.869205 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867903    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:10:43.869205 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867905    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:10:43.869205 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867907    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:10:43.869205 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867910    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:10:43.869205 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867913    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:10:43.869205 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867916    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:10:43.869205 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867918    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:10:43.869205 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867920    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:10:43.869205 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867923    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:10:43.869205 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867925    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:10:43.869205 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867929    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:10:43.869205 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867932    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:10:43.869747 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867934    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:10:43.869747 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867937    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:10:43.869747 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867939    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:10:43.869747 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867942    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:10:43.869747 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867944    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:10:43.869747 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867948    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:10:43.869747 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867952    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:10:43.869747 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867954    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:10:43.869747 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867957    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:10:43.869747 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867961    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:10:43.869747 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867965    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:10:43.869747 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867968    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:10:43.869747 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867971    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:10:43.869747 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867974    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:10:43.869747 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867977    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:10:43.869747 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867979    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:10:43.869747 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867982    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:10:43.869747 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867985    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:10:43.869747 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.867987    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:10:43.869747 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868684    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:10:43.870209 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868692    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:10:43.870209 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868695    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:10:43.870209 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868698    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:10:43.870209 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868701    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:10:43.870209 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868704    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:10:43.870209 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868707    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:10:43.870209 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868710    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:10:43.870209 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868713    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:10:43.870209 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868715    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:10:43.870209 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868718    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:10:43.870209 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868721    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:10:43.870209 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868724    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:10:43.870209 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868726    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:10:43.870209 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868729    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:10:43.870209 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868731    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:10:43.870209 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868734    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:10:43.870209 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868736    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:10:43.870209 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868739    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:10:43.870209 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868742    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:10:43.870685 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868745    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:10:43.870685 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868747    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:10:43.870685 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868750    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:10:43.870685 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868753    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:10:43.870685 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868755    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:10:43.870685 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868758    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:10:43.870685 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868760    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:10:43.870685 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868763    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:10:43.870685 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868766    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:10:43.870685 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868769    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:10:43.870685 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868771    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:10:43.870685 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868774    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:10:43.870685 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868776    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:10:43.870685 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868779    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:10:43.870685 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868781    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:10:43.870685 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868785    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:10:43.870685 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868788    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:10:43.870685 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868791    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:10:43.870685 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868794    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:10:43.871146 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868796    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:10:43.871146 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868799    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:10:43.871146 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868801    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:10:43.871146 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868804    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:10:43.871146 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868806    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:10:43.871146 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868808    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:10:43.871146 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868811    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:10:43.871146 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868813    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:10:43.871146 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868816    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:10:43.871146 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868818    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:10:43.871146 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868821    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:10:43.871146 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868823    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:10:43.871146 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868826    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:10:43.871146 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868829    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:10:43.871146 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868831    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:10:43.871146 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868834    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:10:43.871146 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868836    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:10:43.871146 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868839    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:10:43.871146 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868841    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:10:43.871146 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868844    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:10:43.871632 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868848    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:10:43.871632 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868851    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:10:43.871632 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868853    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:10:43.871632 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868855    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:10:43.871632 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868858    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:10:43.871632 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868861    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:10:43.871632 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868863    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:10:43.871632 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868866    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:10:43.871632 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868868    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:10:43.871632 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868871    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:10:43.871632 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868873    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:10:43.871632 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868876    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:10:43.871632 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868879    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:10:43.871632 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868882    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:10:43.871632 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868884    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:10:43.871632 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868887    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:10:43.871632 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868890    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:10:43.871632 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868892    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:10:43.871632 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868895    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:10:43.871632 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868897    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:10:43.872105 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868900    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:10:43.872105 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868902    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:10:43.872105 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868905    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:10:43.872105 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868908    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:10:43.872105 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868910    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:10:43.872105 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868913    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:10:43.872105 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868915    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:10:43.872105 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868918    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:10:43.872105 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868920    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:10:43.872105 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868923    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:10:43.872105 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868926    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:10:43.872105 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868928    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:10:43.872105 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868934    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:10:43.872105 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868937    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:10:43.872105 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868940    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:10:43.872105 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868943    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:10:43.872105 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868946    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:10:43.872105 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868950    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:10:43.872105 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868953    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:10:43.872604 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.868956    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:10:43.872604 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869286    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:10:43.872604 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869291    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:10:43.872604 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869293    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:10:43.872604 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869296    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:10:43.872604 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869299    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:10:43.872604 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869302    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:10:43.872604 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869304    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:10:43.872604 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869307    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:10:43.872604 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869310    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:10:43.872604 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869312    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:10:43.872604 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869315    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:10:43.872604 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869317    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:10:43.872604 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869320    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:10:43.872604 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869322    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:10:43.872604 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869325    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:10:43.872604 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869327    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:10:43.872604 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869330    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:10:43.872604 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869332    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:10:43.872604 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869335    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:10:43.873096 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869337    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:10:43.873096 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869340    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:10:43.873096 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869343    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:10:43.873096 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869345    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:10:43.873096 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869362    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:10:43.873096 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869367    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:10:43.873096 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869369    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:10:43.873096 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869372    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:10:43.873096 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869374    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:10:43.873096 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869376    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:10:43.873096 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869380    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:10:43.873096 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869383    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:10:43.873096 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869385    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:10:43.873096 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869388    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:10:43.873096 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869390    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:10:43.873096 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869393    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:10:43.873096 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869396    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:10:43.873096 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869398    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:10:43.873096 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869402    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:10:43.873096 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869404    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:10:43.873606 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869407    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:10:43.873606 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869410    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:10:43.873606 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869412    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:10:43.873606 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869417    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:10:43.873606 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869420    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:10:43.873606 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869423    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:10:43.873606 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869426    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:10:43.873606 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869429    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:10:43.873606 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869432    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:10:43.873606 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869434    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:10:43.873606 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869437    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:10:43.873606 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869439    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:10:43.873606 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869442    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:10:43.873606 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869445    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:10:43.873606 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869447    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:10:43.873606 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869450    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:10:43.873606 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869452    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:10:43.873606 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869456    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:10:43.873606 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869459    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:10:43.873606 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869461    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:10:43.874078 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869463    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:10:43.874078 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869466    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:10:43.874078 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869472    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:10:43.874078 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869474    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:10:43.874078 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869477    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:10:43.874078 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869493    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:10:43.874078 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869495    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:10:43.874078 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869497    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:10:43.874078 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869500    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:10:43.874078 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869502    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:10:43.874078 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869504    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:10:43.874078 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869507    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:10:43.874078 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869510    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:10:43.874078 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869512    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:10:43.874078 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869514    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:10:43.874078 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869517    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:10:43.874078 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869519    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:10:43.874078 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869522    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:10:43.874078 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869524    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:10:43.874078 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869526    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:10:43.874603 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869529    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:10:43.874603 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869531    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:10:43.874603 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869534    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:10:43.874603 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869536    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:10:43.874603 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869538    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:10:43.874603 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869541    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:10:43.874603 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869543    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:10:43.874603 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869546    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:10:43.874603 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869549    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:10:43.874603 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869553    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:10:43.874603 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869556    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:10:43.874603 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869559    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:10:43.874603 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869561    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:10:43.874603 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869564    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:10:43.874603 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869568    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:10:43.874603 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869570    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:10:43.874603 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869574    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:10:43.874603 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869577    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:10:43.874603 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869580    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.869583    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869647    2567 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869657    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869662    2567 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869666    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869670    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869673    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869677    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869681    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869683    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869686    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869690    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869693    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869696    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869698    2567 flags.go:64] FLAG: --cgroup-root=""
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869701    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869704    2567 flags.go:64] FLAG: --client-ca-file=""
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869707    2567 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869710    2567 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869714    2567 flags.go:64] FLAG: --cluster-domain=""
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869717    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869720    2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869724    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 17:10:43.875128 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869727    2567 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869730    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869733    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869736    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869740    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869743    2567 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869746    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869749    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869752    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869755    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869759    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869762    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869765    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869768    2567 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869771    2567 flags.go:64] FLAG: --enable-server="true"
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869774    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869778    2567 flags.go:64] FLAG: --event-burst="100"
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869781    2567 flags.go:64] FLAG: --event-qps="50"
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869784    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869786    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869789    2567 flags.go:64] FLAG: --eviction-hard=""
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869793    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869795    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869798    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869801    2567 flags.go:64] FLAG: --eviction-soft=""
Jul 28 17:10:43.875705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869803    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869806    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869809    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869811    2567 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869814    2567 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869817    2567 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869821    2567 flags.go:64] FLAG: --feature-gates=""
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869824    2567 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869827    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869830    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869833    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869837    2567 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869839    2567 flags.go:64] FLAG: --help="false"
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869842    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-133-215.ec2.internal"
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869845    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869848    2567 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869850    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869854    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869857    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869860    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869862    2567 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869865    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869868    2567 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 17:10:43.876306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869871    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869874    2567 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869877    2567 flags.go:64] FLAG: --kube-reserved=""
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869879    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869882    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869885    2567 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869888    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869890    2567 flags.go:64] FLAG: --lock-file=""
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869893    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869896    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869898    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869903    2567 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869906    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869908    2567 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869911    2567 flags.go:64] FLAG: --logging-format="text"
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869915    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869918    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869921    2567 flags.go:64] FLAG: --manifest-url=""
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869923    2567 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869928    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869932    2567 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869935    2567 flags.go:64] FLAG: --max-pods="110"
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869938    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869941    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869944    2567 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 17:10:43.876900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869947    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869950    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869953    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869955    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869962    2567 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869965    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869968    2567 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869971    2567 flags.go:64] FLAG: --pod-cidr=""
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869974    2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869978    2567 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869981    2567 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869984    2567 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869986    2567 flags.go:64] FLAG: --port="10250"
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869989    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869992    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0582e6fa2ee08e2c4"
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869995    2567 flags.go:64] FLAG: --qos-reserved=""
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.869997    2567 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870000    2567 flags.go:64] FLAG: --register-node="true"
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870003    2567 flags.go:64] FLAG: --register-with-taints=""
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870006    2567 flags.go:64] FLAG: --registry-burst="10"
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870009    2567 flags.go:64] FLAG: --registry-qps="5"
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870012    2567 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870015    2567 flags.go:64] FLAG: --reserved-memory=""
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870019    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870022    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 17:10:43.877549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870024    2567 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870027    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870030    2567 flags.go:64] FLAG: --runonce="false"
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870033    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870036    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870039    2567 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870042    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870044    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870047    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870050    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870053    2567 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870055    2567 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870058    2567 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870061    2567 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870063    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870066    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870069    2567 flags.go:64] FLAG: --system-cgroups=""
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870072    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870077    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870079    2567 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870082    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870086    2567 flags.go:64] FLAG: --tls-min-version=""
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870088    2567 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870091    2567 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870094    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 17:10:43.878157 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870096    2567 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 17:10:43.878757 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870099    2567 flags.go:64] FLAG: --v="2"
Jul 28 17:10:43.878757 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870103    2567 flags.go:64] FLAG: --version="false"
Jul 28 17:10:43.878757 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870106    2567 flags.go:64] FLAG: --vmodule=""
Jul 28 17:10:43.878757 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870110    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 17:10:43.878757 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.870113    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 17:10:43.878757 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870209    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:10:43.878757 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870212    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:10:43.878757 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870215    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:10:43.878757 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870218    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:10:43.878757 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870222    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:10:43.878757 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870225    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:10:43.878757 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870228    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:10:43.878757 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870231    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:10:43.878757 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870234    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:10:43.878757 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870237    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:10:43.878757 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870239    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:10:43.878757 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870242    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:10:43.878757 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870245    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:10:43.878757 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870247    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:10:43.878757 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870250    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:10:43.878757 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870252    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:10:43.879255 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870255    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:10:43.879255 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870257    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:10:43.879255 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870260    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:10:43.879255 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870262    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:10:43.879255 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870265    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:10:43.879255 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870267    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:10:43.879255 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870271    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:10:43.879255 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870274    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:10:43.879255 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870277    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:10:43.879255 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870280    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:10:43.879255 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870283    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:10:43.879255 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870285    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:10:43.879255 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870288    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:10:43.879255 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870290    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:10:43.879255 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870294    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:10:43.879255 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870296    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:10:43.879255 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870299    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:10:43.879255 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870301    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:10:43.879255 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870304    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:10:43.879737 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870306    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:10:43.879737 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870309    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:10:43.879737 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870312    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:10:43.879737 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870314    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:10:43.879737 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870317    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:10:43.879737 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870319    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:10:43.879737 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870321    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:10:43.879737 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870324    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:10:43.879737 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870326    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:10:43.879737 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870329    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:10:43.879737 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870331    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:10:43.879737 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870333    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:10:43.879737 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870336    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:10:43.879737 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870338    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:10:43.879737 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870340    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:10:43.879737 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870343    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:10:43.879737 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870346    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:10:43.879737 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870360    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:10:43.879737 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870363    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:10:43.879737 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870365    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:10:43.880196 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870368    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:10:43.880196 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870370    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:10:43.880196 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870372    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:10:43.880196 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870375    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:10:43.880196 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870377    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:10:43.880196 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870380    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:10:43.880196 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870382    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:10:43.880196 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870386    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:10:43.880196 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870388    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:10:43.880196 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870390    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:10:43.880196 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870393    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:10:43.880196 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870395    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:10:43.880196 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870398    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:10:43.880196 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870400    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:10:43.880196 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870403    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:10:43.880196 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870406    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:10:43.880196 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870408    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:10:43.880196 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870411    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:10:43.880196 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870414    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:10:43.880686 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870418    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:10:43.880686 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870420    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:10:43.880686 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870423    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:10:43.880686 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870426    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:10:43.880686 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870428    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:10:43.880686 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870431    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:10:43.880686 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870433    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:10:43.880686 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870436    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:10:43.880686 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870438    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:10:43.880686 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870441    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:10:43.880686 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870444    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:10:43.880686 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870448    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:10:43.880686 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870451    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:10:43.880686 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870454    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:10:43.880686 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870456    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:10:43.880686 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870459    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:10:43.880686 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870462    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:10:43.880686 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870465    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:10:43.880686 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870468    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:10:43.880686 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870470    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:10:43.881178 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870474    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:10:43.881178 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870477    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:10:43.881178 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870479    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:10:43.881178 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870481    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:10:43.881178 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.870484    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:10:43.881178 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.871365    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 17:10:43.881178 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.878569    2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 17:10:43.881178 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.878581    2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 17:10:43.881178 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.879278    2567 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 17:10:43.881178 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.879288    2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 17:10:43.881178 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879343    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:10:43.881178 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879362    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:10:43.881178 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879366    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:10:43.881178 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879369    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:10:43.881178 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879372    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:10:43.881586 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879374    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:10:43.881586 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879377    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:10:43.881586 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879379    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:10:43.881586 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879382    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:10:43.881586 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879384    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:10:43.881586 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879387    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:10:43.881586 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879389    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:10:43.881586 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879392    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:10:43.881586 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879394    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:10:43.881586 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879402    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:10:43.881586 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879405    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:10:43.881586 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879407    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:10:43.881586 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879410    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:10:43.881586 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879412    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:10:43.881586 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879415    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:10:43.881586 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879417    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:10:43.881586 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879419    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:10:43.881586 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879422    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:10:43.881586 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879425    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:10:43.881586 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879427    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:10:43.882057 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879429    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:10:43.882057 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879432    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:10:43.882057 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879435    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:10:43.882057 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879437    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:10:43.882057 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879441    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:10:43.882057 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879444    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:10:43.882057 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879446    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:10:43.882057 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879449    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:10:43.882057 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879451    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:10:43.882057 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879454    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:10:43.882057 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879456    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:10:43.882057 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879459    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:10:43.882057 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879461    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:10:43.882057 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879463    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:10:43.882057 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879466    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:10:43.882057 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879468    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:10:43.882057 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879471    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:10:43.882057 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879473    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:10:43.882057 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879476    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:10:43.882057 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879478    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:10:43.882550 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879481    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:10:43.882550 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879483    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:10:43.882550 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879493    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:10:43.882550 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879496    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:10:43.882550 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879498    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:10:43.882550 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879501    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:10:43.882550 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879504    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:10:43.882550 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879506    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:10:43.882550 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879508    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:10:43.882550 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879511    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:10:43.882550 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879513    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:10:43.882550 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879516    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:10:43.882550 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879518    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:10:43.882550 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879521    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:10:43.882550 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879523    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:10:43.882550 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879526    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:10:43.882550 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879528    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:10:43.882550 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879532    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:10:43.882550 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879536    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:10:43.882550 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879539    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:10:43.883038 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879542    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:10:43.883038 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879544    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:10:43.883038 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879547    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:10:43.883038 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879550    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:10:43.883038 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879553    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:10:43.883038 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879556    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:10:43.883038 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879559    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:10:43.883038 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879561    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:10:43.883038 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879565    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:10:43.883038 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879568    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:10:43.883038 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879571    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:10:43.883038 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879574    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:10:43.883038 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879576    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:10:43.883038 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879580    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:10:43.883038 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879584    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:10:43.883038 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879591    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:10:43.883038 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879593    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:10:43.883038 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879596    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:10:43.883038 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879598    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:10:43.883496 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879601    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:10:43.883496 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879603    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:10:43.883496 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879606    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:10:43.883496 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879609    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:10:43.883496 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879611    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:10:43.883496 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879614    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:10:43.883496 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879617    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:10:43.883496 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879619    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:10:43.883496 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879622    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:10:43.883496 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879625    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:10:43.883496 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879627    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:10:43.883496 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879630    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:10:43.883496 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879632    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:10:43.883496 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879635    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:10:43.883496 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879637    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:10:43.883860 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.879642    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 17:10:43.883860 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879786    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:10:43.883860 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879791    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:10:43.883860 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879794    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:10:43.883860 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879796    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:10:43.883860 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879799    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:10:43.883860 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879802    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:10:43.883860 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879804    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:10:43.883860 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879807    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:10:43.883860 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879809    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:10:43.883860 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879812    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:10:43.883860 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879814    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:10:43.883860 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879817    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:10:43.883860 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879819    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:10:43.883860 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879827    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:10:43.884218 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879830    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:10:43.884218 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879832    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:10:43.884218 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879835    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:10:43.884218 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879837    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:10:43.884218 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879840    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:10:43.884218 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879842    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:10:43.884218 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879844    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:10:43.884218 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879847    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:10:43.884218 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879849    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:10:43.884218 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879852    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:10:43.884218 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879854    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:10:43.884218 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879857    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:10:43.884218 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879860    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:10:43.884218 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879862    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:10:43.884218 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879865    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:10:43.884218 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879868    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:10:43.884218 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879870    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:10:43.884218 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879872    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:10:43.884218 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879875    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:10:43.884218 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879878    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:10:43.884856 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879880    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:10:43.884856 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879883    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:10:43.884856 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879885    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:10:43.884856 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879889    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:10:43.884856 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879893    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:10:43.884856 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879897    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:10:43.884856 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879900    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:10:43.884856 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879902    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:10:43.884856 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879905    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:10:43.884856 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879908    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:10:43.884856 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879910    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:10:43.884856 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879912    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:10:43.884856 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879920    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:10:43.884856 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879923    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:10:43.884856 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879925    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:10:43.884856 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879928    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:10:43.884856 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879930    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:10:43.884856 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879933    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:10:43.884856 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879935    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:10:43.885313 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879937    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:10:43.885313 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879940    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:10:43.885313 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879942    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:10:43.885313 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879944    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:10:43.885313 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879947    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:10:43.885313 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879950    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:10:43.885313 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879953    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:10:43.885313 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879955    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:10:43.885313 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879957    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:10:43.885313 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879960    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:10:43.885313 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879962    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:10:43.885313 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879964    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:10:43.885313 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879967    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:10:43.885313 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879969    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:10:43.885313 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879972    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:10:43.885313 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879974    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:10:43.885313 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879977    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:10:43.885313 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879979    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:10:43.885313 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879982    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:10:43.885313 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879984    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:10:43.885873 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879987    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:10:43.885873 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879989    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:10:43.885873 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879992    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:10:43.885873 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879994    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:10:43.885873 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879996    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:10:43.885873 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.879999    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:10:43.885873 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.880006    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:10:43.885873 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.880008    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:10:43.885873 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.880011    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:10:43.885873 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.880013    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:10:43.885873 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.880015    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:10:43.885873 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.880018    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:10:43.885873 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.880020    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:10:43.885873 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.880024    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:10:43.885873 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.880027    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:10:43.885873 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.880030    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:10:43.885873 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.880033    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:10:43.885873 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.880036    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:10:43.885873 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.880040    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:10:43.886346 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.880042    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:10:43.886346 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.880044    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:10:43.886346 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.880047    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:10:43.886346 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.880049    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:10:43.886346 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.880052    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:10:43.886346 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.880054    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:10:43.886346 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:43.880056    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:10:43.886346 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.880061    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 17:10:43.886346 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.880155    2567 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 17:10:43.886346 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.882380    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 17:10:43.886346 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.883910    2567 server.go:1019] "Starting client certificate rotation"
Jul 28 17:10:43.886346 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.884006    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 17:10:43.886346 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.884701    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 17:10:43.889309 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.889293    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 17:10:43.892183 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.892167    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 17:10:43.900063 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.900048    2567 log.go:25] "Validated CRI v1 runtime API"
Jul 28 17:10:43.902322 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.902308    2567 log.go:25] "Validated CRI v1 image API"
Jul 28 17:10:43.903261 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.903244    2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 17:10:43.905601 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.905582    2567 fs.go:135] Filesystem UUIDs: map[2a399c76-8a5a-4a3e-9f4d-7233cf91b15c:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 84a22b2a-38cf-42c5-b301-7fd743d60b67:/dev/nvme0n1p4]
Jul 28 17:10:43.905670 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.905599    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 17:10:43.909954 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.909857    2567 manager.go:216] Machine: {Timestamp:2026-07-28 17:10:43.909388377 +0000 UTC m=+0.300644665 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3118120 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2838dae64fb782f30b77dcec5617e5 SystemUUID:ec2838da-e64f-b782-f30b-77dcec5617e5 BootID:e10503a8-7c33-4505-94ca-19674fcf9918 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fe:94:a7:f2:09 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fe:94:a7:f2:09 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:56:eb:81:39:94:66 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 17:10:43.909954 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.909952    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 17:10:43.910044 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.910008    2567 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 17:10:43.913582 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.913557    2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 17:10:43.913696 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.913586    2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-215.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 17:10:43.914371 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.914346    2567 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 17:10:43.914402 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.914373    2567 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 17:10:43.914402 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.914381    2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 17:10:43.914402 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.914390    2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 17:10:43.915537 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.915528    2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 17:10:43.917385 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.917374    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 17:10:43.917516 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.917506    2567 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 17:10:43.917560 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.917537    2567 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 17:10:43.917560 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.917547    2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 17:10:43.917644 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.917568    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 17:10:43.917644 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.917579    2567 kubelet.go:398] "Adding apiserver pod source"
Jul 28 17:10:43.917644 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.917587    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 17:10:43.917878 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.917867    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 17:10:43.917909 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.917886    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 17:10:43.919689 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.919660    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 17:10:43.922028 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.922011    2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 17:10:43.924125 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.924111    2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 17:10:43.924183 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.924141    2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 17:10:43.925891 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.925881    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 17:10:43.925940 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.925895    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 17:10:43.925940 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.925901    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 17:10:43.925940 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.925906    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 17:10:43.925940 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.925915    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 17:10:43.925940 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.925920    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 17:10:43.925940 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.925927    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 17:10:43.925940 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.925932    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 17:10:43.925940 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.925938    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 17:10:43.925940 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.925943    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 17:10:43.926158 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.925951    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 17:10:43.926158 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.925956    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 17:10:43.927392 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.927375    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 17:10:43.927392 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.927392    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 17:10:43.930132 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.930119    2567 server.go:1268] "Started kubelet"
Jul 28 17:10:43.930215 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:43.930171    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 17:10:43.930272 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:43.930248    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-215.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 17:10:43.930272 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.930256    2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-215.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 17:10:43.930419 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.930257    2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 17:10:43.930419 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.930383    2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 17:10:43.930509 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.930422    2567 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 17:10:43.931439 ip-10-0-133-215 systemd[1]: Started Kubernetes Kubelet.
Jul 28 17:10:43.931662 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.931611    2567 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 17:10:43.932343 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.932209    2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 17:10:43.936385 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:43.936333    2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 17:10:43.938297 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:43.937464    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-215.ec2.internal.18c6834727a63724  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-215.ec2.internal,UID:ip-10-0-133-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-215.ec2.internal,},FirstTimestamp:2026-07-28 17:10:43.93010154 +0000 UTC m=+0.321357829,LastTimestamp:2026-07-28 17:10:43.93010154 +0000 UTC m=+0.321357829,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-215.ec2.internal,}"
Jul 28 17:10:43.941208 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.941195    2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 17:10:43.941336 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.941268    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 17:10:43.941859 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:43.941538    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 28 17:10:43.942089 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.942022    2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 17:10:43.942257 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.942183    2567 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 17:10:43.942257 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.942199    2567 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 17:10:43.942376 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.942314    2567 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 17:10:43.942376 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.942324    2567 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 17:10:43.943260 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.943181    2567 factory.go:55] Registering systemd factory
Jul 28 17:10:43.943260 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.943197    2567 factory.go:223] Registration of the systemd container factory successfully
Jul 28 17:10:43.943506 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.943491    2567 factory.go:153] Registering CRI-O factory
Jul 28 17:10:43.943563 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.943509    2567 factory.go:223] Registration of the crio container factory successfully
Jul 28 17:10:43.943563 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.943553    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 17:10:43.943658 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.943615    2567 factory.go:103] Registering Raw factory
Jul 28 17:10:43.943658 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.943628    2567 manager.go:1196] Started watching for new ooms in manager
Jul 28 17:10:43.944216 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.944201    2567 manager.go:349] Starting recovery of all containers
Jul 28 17:10:43.944728 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:43.944701    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 17:10:43.944801 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:43.944773    2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-215.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 17:10:43.956544 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.956417    2567 manager.go:354] Recovery completed
Jul 28 17:10:43.960873 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.960860    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:10:43.962617 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.962601    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:10:43.962670 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.962626    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:10:43.962670 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.962635    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:10:43.963167 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.963151    2567 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 17:10:43.963167 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.963164    2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 17:10:43.963274 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.963177    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 17:10:43.963274 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.963153    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:10:43.963859 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.963844    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:10:43.963939 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.963873    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:10:43.963939 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.963895    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:10:43.965129 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.965117    2567 policy_none.go:49] "None policy: Start"
Jul 28 17:10:43.965175 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.965132    2567 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 17:10:43.965175 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.965143    2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 17:10:43.966150 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.966141    2567 policy_none.go:47] "Start"
Jul 28 17:10:43.973001 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:43.972933    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-215.ec2.internal.18c6834729965533  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-215.ec2.internal,UID:ip-10-0-133-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-215.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-215.ec2.internal,},FirstTimestamp:2026-07-28 17:10:43.962615091 +0000 UTC m=+0.353871377,LastTimestamp:2026-07-28 17:10:43.962615091 +0000 UTC m=+0.353871377,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-215.ec2.internal,}"
Jul 28 17:10:43.983158 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:43.983096    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-215.ec2.internal.18c6834729969248  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-215.ec2.internal,UID:ip-10-0-133-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-215.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-215.ec2.internal,},FirstTimestamp:2026-07-28 17:10:43.962630728 +0000 UTC m=+0.353887016,LastTimestamp:2026-07-28 17:10:43.962630728 +0000 UTC m=+0.353887016,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-215.ec2.internal,}"
Jul 28 17:10:43.992741 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:43.992688    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-215.ec2.internal.18c683472996b4b8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-215.ec2.internal,UID:ip-10-0-133-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-215.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-215.ec2.internal,},FirstTimestamp:2026-07-28 17:10:43.962639544 +0000 UTC m=+0.353895831,LastTimestamp:2026-07-28 17:10:43.962639544 +0000 UTC m=+0.353895831,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-215.ec2.internal,}"
Jul 28 17:10:43.997869 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.997858    2567 manager.go:337] "Starting Device Plugin manager"
Jul 28 17:10:43.997932 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:43.997889    2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 17:10:43.997932 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.997898    2567 server.go:87] "Starting device plugin registration server"
Jul 28 17:10:43.998106 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.998094    2567 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 17:10:43.998143 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.998109    2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 17:10:43.998213 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.998199    2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 17:10:43.998292 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.998272    2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 17:10:43.998292 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:43.998282    2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 17:10:43.999130 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:43.999115    2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 17:10:43.999184 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:43.999153    2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 28 17:10:44.007527 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:44.007472    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-215.ec2.internal.18c6834729965533\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-215.ec2.internal.18c6834729965533  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-215.ec2.internal,UID:ip-10-0-133-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-215.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-215.ec2.internal,},FirstTimestamp:2026-07-28 17:10:43.962615091 +0000 UTC m=+0.353871377,LastTimestamp:2026-07-28 17:10:43.963858896 +0000 UTC m=+0.355115186,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-215.ec2.internal,}"
Jul 28 17:10:44.021009 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:44.020042    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-215.ec2.internal.18c6834729969248\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-215.ec2.internal.18c6834729969248  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-215.ec2.internal,UID:ip-10-0-133-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-215.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-215.ec2.internal,},FirstTimestamp:2026-07-28 17:10:43.962630728 +0000 UTC m=+0.353887016,LastTimestamp:2026-07-28 17:10:43.963879905 +0000 UTC m=+0.355136192,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-215.ec2.internal,}"
Jul 28 17:10:44.032602 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:44.032544    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-215.ec2.internal.18c683472996b4b8\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-215.ec2.internal.18c683472996b4b8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-215.ec2.internal,UID:ip-10-0-133-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-215.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-215.ec2.internal,},FirstTimestamp:2026-07-28 17:10:43.962639544 +0000 UTC m=+0.353895831,LastTimestamp:2026-07-28 17:10:43.963901992 +0000 UTC m=+0.355158284,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-215.ec2.internal,}"
Jul 28 17:10:44.043879 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:44.043823    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-215.ec2.internal.18c683472bd3395a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-215.ec2.internal,UID:ip-10-0-133-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-133-215.ec2.internal,},FirstTimestamp:2026-07-28 17:10:44.00016009 +0000 UTC m=+0.391416385,LastTimestamp:2026-07-28 17:10:44.00016009 +0000 UTC m=+0.391416385,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-215.ec2.internal,}"
Jul 28 17:10:44.073882 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:44.073862    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kxmk8"
Jul 28 17:10:44.079938 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:44.079921    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kxmk8"
Jul 28 17:10:44.098203 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:44.098189    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:10:44.098880 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:44.098866    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:10:44.098948 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:44.098890    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:10:44.098948 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:44.098904    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:10:44.098948 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:44.098939    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-215.ec2.internal"
Jul 28 17:10:44.116506 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:44.116463    2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-215.ec2.internal"
Jul 28 17:10:44.116506 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:44.116483    2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-215.ec2.internal\": node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 28 17:10:44.134847 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:44.134825    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 17:10:44.135858 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:44.135839    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 17:10:44.135926 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:44.135872    2567 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 17:10:44.135926 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:44.135896    2567 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 17:10:44.135926 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:44.135923    2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 17:10:44.138562 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:44.138542    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:10:44.143708 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:44.143695    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 28 17:10:45.381743 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.381707    2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal"]
Jul 28 17:10:45.381821 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.381780    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:10:45.382876 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.382863    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:10:45.382934 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.382888    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:10:45.382934 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.382900    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:10:45.384111 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.384100    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:10:45.384244 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.384231    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal"
Jul 28 17:10:45.384273 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.384258    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:10:45.384705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.384690    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:10:45.384769 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.384724    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:10:45.384769 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.384739    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:10:45.384831 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.384714    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:10:45.384831 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.384808    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:10:45.384831 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.384821    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:10:45.385829 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.385815    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal"
Jul 28 17:10:45.385881 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.385840    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:10:45.386546 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.386532    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:10:45.386635 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.386555    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:10:45.386635 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.386565    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:10:45.388757 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.388742    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3da3e528494d7d2552a2b35126ff8b14-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal\" (UID: \"3da3e528494d7d2552a2b35126ff8b14\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal"
Jul 28 17:10:45.388818 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.388764    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3da3e528494d7d2552a2b35126ff8b14-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal\" (UID: \"3da3e528494d7d2552a2b35126ff8b14\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal"
Jul 28 17:10:45.388818 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.388780    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f36b97c92bac0bfb582fa26faaad0625-config\") pod \"kube-apiserver-proxy-ip-10-0-133-215.ec2.internal\" (UID: \"f36b97c92bac0bfb582fa26faaad0625\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal"
Jul 28 17:10:45.389047 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:45.389022    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 28 17:10:45.414074 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:45.414058    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-215.ec2.internal\" not found" node="ip-10-0-133-215.ec2.internal"
Jul 28 17:10:45.418353 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:45.418339    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-215.ec2.internal\" not found" node="ip-10-0-133-215.ec2.internal"
Jul 28 17:10:45.489446 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:45.489406    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 28 17:10:45.489446 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.489410    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3da3e528494d7d2552a2b35126ff8b14-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal\" (UID: \"3da3e528494d7d2552a2b35126ff8b14\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal"
Jul 28 17:10:45.489587 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.489456    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3da3e528494d7d2552a2b35126ff8b14-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal\" (UID: \"3da3e528494d7d2552a2b35126ff8b14\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal"
Jul 28 17:10:45.489587 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.489481    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3da3e528494d7d2552a2b35126ff8b14-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal\" (UID: \"3da3e528494d7d2552a2b35126ff8b14\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal"
Jul 28 17:10:45.489587 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.489463    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3da3e528494d7d2552a2b35126ff8b14-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal\" (UID: \"3da3e528494d7d2552a2b35126ff8b14\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal"
Jul 28 17:10:45.489587 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.489520    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f36b97c92bac0bfb582fa26faaad0625-config\") pod \"kube-apiserver-proxy-ip-10-0-133-215.ec2.internal\" (UID: \"f36b97c92bac0bfb582fa26faaad0625\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal"
Jul 28 17:10:45.489587 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.489543    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f36b97c92bac0bfb582fa26faaad0625-config\") pod \"kube-apiserver-proxy-ip-10-0-133-215.ec2.internal\" (UID: \"f36b97c92bac0bfb582fa26faaad0625\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal"
Jul 28 17:10:45.590564 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:45.590515    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 28 17:10:45.690925 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:45.690899    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 28 17:10:45.718075 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.718058    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal"
Jul 28 17:10:45.722976 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:45.722962    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal"
Jul 28 17:10:45.791659 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:45.791634    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 28 17:10:45.892083 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:45.892039    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 28 17:10:45.992417 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:45.992380    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 28 17:10:46.028664 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:46.028649    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 17:10:46.029034 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:46.028741    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 17:10:46.093349 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:46.093330    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 28 17:10:46.131581 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:46.131566    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:10:46.187144 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:46.187100    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal"
Jul 28 17:10:46.202143 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:46.202123    2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:10:46.205109 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:46.205095    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 17:10:46.205848 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:46.205835    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal"
Jul 28 17:10:46.216530 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:46.216500    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 17:10:46.230652 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:46.230608    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 17:05:45 +0000 UTC" deadline="2028-03-16 23:50:30.626712832 +0000 UTC"
Jul 28 17:10:46.230652 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:46.230651    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14334h39m44.396064494s"
Jul 28 17:10:46.296811 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:46.296786    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3da3e528494d7d2552a2b35126ff8b14.slice/crio-edd0bcb05b2801b5e5413c150f0f8ce98a1b122dac40822c1faaf21c07232805 WatchSource:0}: Error finding container edd0bcb05b2801b5e5413c150f0f8ce98a1b122dac40822c1faaf21c07232805: Status 404 returned error can't find the container with id edd0bcb05b2801b5e5413c150f0f8ce98a1b122dac40822c1faaf21c07232805
Jul 28 17:10:46.300001 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:46.299987    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 17:10:46.604003 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:46.603957    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:10:46.633894 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:46.633874    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf36b97c92bac0bfb582fa26faaad0625.slice/crio-e0fe24a5560e93f7725df6b7ef597714c1daf8620192d0506b380ecb06f07e3d WatchSource:0}: Error finding container e0fe24a5560e93f7725df6b7ef597714c1daf8620192d0506b380ecb06f07e3d: Status 404 returned error can't find the container with id e0fe24a5560e93f7725df6b7ef597714c1daf8620192d0506b380ecb06f07e3d
Jul 28 17:10:47.063499 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.063436    2567 apiserver.go:52] "Watching apiserver"
Jul 28 17:10:47.074904 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.074870    2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 17:10:47.075259 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.075234    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-tncc8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal","openshift-network-operator/iptables-alerter-45tdk","openshift-ovn-kubernetes/ovnkube-node-v552f","kube-system/global-pull-secret-syncer-vcff5","kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal","openshift-dns/node-resolver-92rx8","openshift-image-registry/node-ca-cxrrh","openshift-multus/multus-additional-cni-plugins-zxsfx","openshift-multus/multus-q4vcf","openshift-multus/network-metrics-daemon-fr9xb","openshift-network-diagnostics/network-check-target-67x96","kube-system/konnectivity-agent-l82pq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"]
Jul 28 17:10:47.076767 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.076749    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-92rx8"
Jul 28 17:10:47.077942 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.077924    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l82pq"
Jul 28 17:10:47.079090 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.079075    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-45tdk"
Jul 28 17:10:47.079776 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.079762    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 17:10:47.080357 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.080322    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.080457 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.080341    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ng6cl\""
Jul 28 17:10:47.080457 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.080324    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 17:10:47.080715 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.080696    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 17:10:47.080764 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.080704    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9sw7f\""
Jul 28 17:10:47.081028 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.081012    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 17:10:47.081719 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.081668    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:10:47.081816 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:47.081732    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcff5" podUID="482929b8-7c61-4fee-bd3b-0f9ac86e61df"
Jul 28 17:10:47.082008 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.081987    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 17:10:47.082416 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.082255    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:10:47.082416 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.082376    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 17:10:47.083140 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.083123    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.084342 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.084323    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.085640 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.085620    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.086953 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.086937    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cxrrh"
Jul 28 17:10:47.087239 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.087130    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 17:10:47.087549 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.087527    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 17:10:47.087651 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.087617    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 17:10:47.087795 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.087729    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 17:10:47.087795 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.087741    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 17:10:47.087902 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.087800    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kxzfz\""
Jul 28 17:10:47.088286 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.088267    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 17:10:47.088772 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.088533    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.089448 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.089409    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 17:10:47.089547 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.089447    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 17:10:47.089547 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.089492    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 17:10:47.089886 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.089870    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:10:47.089952 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:47.089929    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fr9xb" podUID="97f130a6-e3b7-4882-b992-0d5eef10897b"
Jul 28 17:10:47.090395 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.090376    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-85wnf\""
Jul 28 17:10:47.090474 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.090431    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 17:10:47.090474 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.090440    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-skqtd\""
Jul 28 17:10:47.090474 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.090455    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9q7vj\""
Jul 28 17:10:47.090655 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.090597    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:10:47.090756 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.090739    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 17:10:47.090989 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.090832    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 17:10:47.090989 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.090944    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-982pd\""
Jul 28 17:10:47.091092 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.090993    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 17:10:47.091092 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.091071    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 17:10:47.091215 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.091199    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 17:10:47.091395 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.091375    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:10:47.091465 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.091395    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 17:10:47.091465 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.091409    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 17:10:47.091465 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:47.091432    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67x96" podUID="a16a1fb7-d21c-49cf-a545-faa8ae9d20b6"
Jul 28 17:10:47.091647 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.091632    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 17:10:47.092031 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.092014    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 17:10:47.092213 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.092194    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hzjbl\""
Jul 28 17:10:47.094028 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.094013    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 17:10:47.094471 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.094456    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nvqpg\""
Jul 28 17:10:47.097801 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.097786    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f363ee59-3092-46f1-a3e5-a01b034a1e14-cni-binary-copy\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.097892 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.097807    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-etc-kubernetes\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.097892 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.097824    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3f8e6a3b-524e-403d-bfdf-338bf7c5defc-agent-certs\") pod \"konnectivity-agent-l82pq\" (UID: \"3f8e6a3b-524e-403d-bfdf-338bf7c5defc\") " pod="kube-system/konnectivity-agent-l82pq"
Jul 28 17:10:47.097892 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.097838    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-etc-systemd\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.097892 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.097854    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-env-overrides\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.098091 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.097894    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2kxj\" (UniqueName: \"kubernetes.io/projected/7ad540e6-ff3a-4366-9176-0063eb45135c-kube-api-access-c2kxj\") pod \"node-resolver-92rx8\" (UID: \"7ad540e6-ff3a-4366-9176-0063eb45135c\") " pod="openshift-dns/node-resolver-92rx8"
Jul 28 17:10:47.098091 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.097948    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-host-kubelet\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.098091 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.097973    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ec7b38b2-8859-4743-875a-4148a0c342b8-device-dir\") pod \"aws-ebs-csi-driver-node-4cjsk\" (UID: \"ec7b38b2-8859-4743-875a-4148a0c342b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.098091 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098003    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-etc-modprobe-d\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.098091 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098024    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-host-var-lib-cni-bin\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.098091 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098038    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-host-slash\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.098091 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098051    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ec7b38b2-8859-4743-875a-4148a0c342b8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4cjsk\" (UID: \"ec7b38b2-8859-4743-875a-4148a0c342b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.098091 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098072    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpnx5\" (UniqueName: \"kubernetes.io/projected/f363ee59-3092-46f1-a3e5-a01b034a1e14-kube-api-access-lpnx5\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.098519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098109    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-host-var-lib-kubelet\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.098519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098145    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ad540e6-ff3a-4366-9176-0063eb45135c-tmp-dir\") pod \"node-resolver-92rx8\" (UID: \"7ad540e6-ff3a-4366-9176-0063eb45135c\") " pod="openshift-dns/node-resolver-92rx8"
Jul 28 17:10:47.098519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098170    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-systemd-units\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.098519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098193    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-var-lib-openvswitch\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.098519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098216    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-ovn-node-metrics-cert\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.098519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098238    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f363ee59-3092-46f1-a3e5-a01b034a1e14-cnibin\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.098519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098265    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-cnibin\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.098519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098286    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8337ae34-e335-4033-8ecd-49dd32a2ea88-multus-daemon-config\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.098519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098307    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgdk9\" (UniqueName: \"kubernetes.io/projected/961f03d7-1d8a-4728-aa72-9e35c4b4d931-kube-api-access-pgdk9\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.098519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098328    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-multus-socket-dir-parent\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.098519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098359    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f363ee59-3092-46f1-a3e5-a01b034a1e14-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.098519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098384    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-host\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.098519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098417    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-run-ovn\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.098519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098450    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ec7b38b2-8859-4743-875a-4148a0c342b8-socket-dir\") pod \"aws-ebs-csi-driver-node-4cjsk\" (UID: \"ec7b38b2-8859-4743-875a-4148a0c342b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.098519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098496    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ec7b38b2-8859-4743-875a-4148a0c342b8-sys-fs\") pod \"aws-ebs-csi-driver-node-4cjsk\" (UID: \"ec7b38b2-8859-4743-875a-4148a0c342b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.099194 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098550    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f363ee59-3092-46f1-a3e5-a01b034a1e14-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.099194 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098574    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-etc-sysctl-conf\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.099194 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098595    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-host-run-netns\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.099194 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098616    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szx6x\" (UniqueName: \"kubernetes.io/projected/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-kube-api-access-szx6x\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.099194 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098639    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/961f03d7-1d8a-4728-aa72-9e35c4b4d931-etc-tuned\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.099194 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098659    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-os-release\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.099194 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098696    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45znx\" (UniqueName: \"kubernetes.io/projected/8337ae34-e335-4033-8ecd-49dd32a2ea88-kube-api-access-45znx\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.099194 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098717    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-host-run-ovn-kubernetes\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.099194 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098737    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f363ee59-3092-46f1-a3e5-a01b034a1e14-system-cni-dir\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.099194 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098756    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-lib-modules\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.099194 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098780    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.099194 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098807    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ad540e6-ff3a-4366-9176-0063eb45135c-hosts-file\") pod \"node-resolver-92rx8\" (UID: \"7ad540e6-ff3a-4366-9176-0063eb45135c\") " pod="openshift-dns/node-resolver-92rx8"
Jul 28 17:10:47.099194 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098840    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-node-log\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.099194 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098885    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-log-socket\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.099194 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098909    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-host-cni-bin\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.099194 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098954    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtwth\" (UniqueName: \"kubernetes.io/projected/ec7b38b2-8859-4743-875a-4148a0c342b8-kube-api-access-qtwth\") pod \"aws-ebs-csi-driver-node-4cjsk\" (UID: \"ec7b38b2-8859-4743-875a-4148a0c342b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.100040 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098976    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-sys\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.100040 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.098995    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3f8e6a3b-524e-403d-bfdf-338bf7c5defc-konnectivity-ca\") pod \"konnectivity-agent-l82pq\" (UID: \"3f8e6a3b-524e-403d-bfdf-338bf7c5defc\") " pod="kube-system/konnectivity-agent-l82pq"
Jul 28 17:10:47.100040 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099029    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f85fcfde-92b5-45c4-9540-05d8dc614a9a-iptables-alerter-script\") pod \"iptables-alerter-45tdk\" (UID: \"f85fcfde-92b5-45c4-9540-05d8dc614a9a\") " pod="openshift-network-operator/iptables-alerter-45tdk"
Jul 28 17:10:47.100040 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099054    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-system-cni-dir\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.100040 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099079    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-host-var-lib-cni-multus\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.100040 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099115    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-hostroot\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.100040 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099138    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/961f03d7-1d8a-4728-aa72-9e35c4b4d931-tmp\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.100040 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099160    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-multus-cni-dir\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.100040 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099182    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-ovnkube-config\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.100040 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099204    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f363ee59-3092-46f1-a3e5-a01b034a1e14-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.100040 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099225    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-run\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.100040 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099246    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-var-lib-kubelet\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.100040 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099267    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f85fcfde-92b5-45c4-9540-05d8dc614a9a-host-slash\") pod \"iptables-alerter-45tdk\" (UID: \"f85fcfde-92b5-45c4-9540-05d8dc614a9a\") " pod="openshift-network-operator/iptables-alerter-45tdk"
Jul 28 17:10:47.100040 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099290    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/482929b8-7c61-4fee-bd3b-0f9ac86e61df-original-pull-secret\") pod \"global-pull-secret-syncer-vcff5\" (UID: \"482929b8-7c61-4fee-bd3b-0f9ac86e61df\") " pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:10:47.100040 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099329    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1b94f336-770a-4f5a-8ef9-cf867e720989-host\") pod \"node-ca-cxrrh\" (UID: \"1b94f336-770a-4f5a-8ef9-cf867e720989\") " pod="openshift-image-registry/node-ca-cxrrh"
Jul 28 17:10:47.100040 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099358    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-host-run-k8s-cni-cncf-io\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.100040 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099394    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-ovnkube-script-lib\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.100896 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099418    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-etc-openvswitch\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.100896 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099448    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ec7b38b2-8859-4743-875a-4148a0c342b8-registration-dir\") pod \"aws-ebs-csi-driver-node-4cjsk\" (UID: \"ec7b38b2-8859-4743-875a-4148a0c342b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.100896 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099473    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-etc-sysconfig\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.100896 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099515    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-etc-sysctl-d\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.100896 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099538    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/482929b8-7c61-4fee-bd3b-0f9ac86e61df-kubelet-config\") pod \"global-pull-secret-syncer-vcff5\" (UID: \"482929b8-7c61-4fee-bd3b-0f9ac86e61df\") " pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:10:47.100896 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099579    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/482929b8-7c61-4fee-bd3b-0f9ac86e61df-dbus\") pod \"global-pull-secret-syncer-vcff5\" (UID: \"482929b8-7c61-4fee-bd3b-0f9ac86e61df\") " pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:10:47.100896 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099875    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8337ae34-e335-4033-8ecd-49dd32a2ea88-cni-binary-copy\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.100896 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.099910    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-run-systemd\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.100896 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.100014    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-host-cni-netd\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.100896 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.100051    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1b94f336-770a-4f5a-8ef9-cf867e720989-serviceca\") pod \"node-ca-cxrrh\" (UID: \"1b94f336-770a-4f5a-8ef9-cf867e720989\") " pod="openshift-image-registry/node-ca-cxrrh"
Jul 28 17:10:47.100896 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.100075    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ec7b38b2-8859-4743-875a-4148a0c342b8-etc-selinux\") pod \"aws-ebs-csi-driver-node-4cjsk\" (UID: \"ec7b38b2-8859-4743-875a-4148a0c342b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.100896 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.100102    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkztd\" (UniqueName: \"kubernetes.io/projected/1b94f336-770a-4f5a-8ef9-cf867e720989-kube-api-access-nkztd\") pod \"node-ca-cxrrh\" (UID: \"1b94f336-770a-4f5a-8ef9-cf867e720989\") " pod="openshift-image-registry/node-ca-cxrrh"
Jul 28 17:10:47.100896 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.100218    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-multus-conf-dir\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.100896 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.100248    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-host-run-multus-certs\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.100896 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.100273    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-etc-kubernetes\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.100896 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.100303    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-host-run-netns\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.101751 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.100325    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-run-openvswitch\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.101751 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.100461    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f363ee59-3092-46f1-a3e5-a01b034a1e14-os-release\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.101751 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.100495    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h82bd\" (UniqueName: \"kubernetes.io/projected/f85fcfde-92b5-45c4-9540-05d8dc614a9a-kube-api-access-h82bd\") pod \"iptables-alerter-45tdk\" (UID: \"f85fcfde-92b5-45c4-9540-05d8dc614a9a\") " pod="openshift-network-operator/iptables-alerter-45tdk"
Jul 28 17:10:47.107613 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.107596    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 17:10:47.137352 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.137333    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7plmt"
Jul 28 17:10:47.146756 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.146738    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7plmt"
Jul 28 17:10:47.169675 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.169653    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:10:47.188590 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.188573    2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 17:10:47.201262 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201236    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/482929b8-7c61-4fee-bd3b-0f9ac86e61df-original-pull-secret\") pod \"global-pull-secret-syncer-vcff5\" (UID: \"482929b8-7c61-4fee-bd3b-0f9ac86e61df\") " pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:10:47.201359 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201270    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1b94f336-770a-4f5a-8ef9-cf867e720989-host\") pod \"node-ca-cxrrh\" (UID: \"1b94f336-770a-4f5a-8ef9-cf867e720989\") " pod="openshift-image-registry/node-ca-cxrrh"
Jul 28 17:10:47.201359 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201286    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-host-run-k8s-cni-cncf-io\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.201497 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201352    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1b94f336-770a-4f5a-8ef9-cf867e720989-host\") pod \"node-ca-cxrrh\" (UID: \"1b94f336-770a-4f5a-8ef9-cf867e720989\") " pod="openshift-image-registry/node-ca-cxrrh"
Jul 28 17:10:47.201497 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:47.201381    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:10:47.201497 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201432    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-host-run-k8s-cni-cncf-io\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.201497 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:47.201464    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/482929b8-7c61-4fee-bd3b-0f9ac86e61df-original-pull-secret podName:482929b8-7c61-4fee-bd3b-0f9ac86e61df nodeName:}" failed. No retries permitted until 2026-07-28 17:10:47.701442536 +0000 UTC m=+2.947544721 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/482929b8-7c61-4fee-bd3b-0f9ac86e61df-original-pull-secret") pod "global-pull-secret-syncer-vcff5" (UID: "482929b8-7c61-4fee-bd3b-0f9ac86e61df") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:10:47.201497 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201486    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-ovnkube-script-lib\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.201758 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201522    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-etc-openvswitch\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.201758 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201567    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-etc-openvswitch\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.201758 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201589    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ec7b38b2-8859-4743-875a-4148a0c342b8-registration-dir\") pod \"aws-ebs-csi-driver-node-4cjsk\" (UID: \"ec7b38b2-8859-4743-875a-4148a0c342b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.201758 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201610    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-etc-sysconfig\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.201758 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201633    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-etc-sysctl-d\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.201758 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201657    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/482929b8-7c61-4fee-bd3b-0f9ac86e61df-kubelet-config\") pod \"global-pull-secret-syncer-vcff5\" (UID: \"482929b8-7c61-4fee-bd3b-0f9ac86e61df\") " pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:10:47.201758 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201679    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/482929b8-7c61-4fee-bd3b-0f9ac86e61df-dbus\") pod \"global-pull-secret-syncer-vcff5\" (UID: \"482929b8-7c61-4fee-bd3b-0f9ac86e61df\") " pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:10:47.201758 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201679    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ec7b38b2-8859-4743-875a-4148a0c342b8-registration-dir\") pod \"aws-ebs-csi-driver-node-4cjsk\" (UID: \"ec7b38b2-8859-4743-875a-4148a0c342b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.201758 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201697    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8337ae34-e335-4033-8ecd-49dd32a2ea88-cni-binary-copy\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.201758 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201707    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/482929b8-7c61-4fee-bd3b-0f9ac86e61df-kubelet-config\") pod \"global-pull-secret-syncer-vcff5\" (UID: \"482929b8-7c61-4fee-bd3b-0f9ac86e61df\") " pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:10:47.201758 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201717    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-run-systemd\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.201758 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201746    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-host-cni-netd\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.202281 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201769    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1b94f336-770a-4f5a-8ef9-cf867e720989-serviceca\") pod \"node-ca-cxrrh\" (UID: \"1b94f336-770a-4f5a-8ef9-cf867e720989\") " pod="openshift-image-registry/node-ca-cxrrh"
Jul 28 17:10:47.202281 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201771    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-etc-sysctl-d\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.202281 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201747    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-run-systemd\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.202281 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201795    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-etc-sysconfig\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.202281 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201848    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-host-cni-netd\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.202281 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201856    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ec7b38b2-8859-4743-875a-4148a0c342b8-etc-selinux\") pod \"aws-ebs-csi-driver-node-4cjsk\" (UID: \"ec7b38b2-8859-4743-875a-4148a0c342b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.202281 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201856    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/482929b8-7c61-4fee-bd3b-0f9ac86e61df-dbus\") pod \"global-pull-secret-syncer-vcff5\" (UID: \"482929b8-7c61-4fee-bd3b-0f9ac86e61df\") " pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:10:47.202281 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201790    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ec7b38b2-8859-4743-875a-4148a0c342b8-etc-selinux\") pod \"aws-ebs-csi-driver-node-4cjsk\" (UID: \"ec7b38b2-8859-4743-875a-4148a0c342b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.202281 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201898    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkztd\" (UniqueName: \"kubernetes.io/projected/1b94f336-770a-4f5a-8ef9-cf867e720989-kube-api-access-nkztd\") pod \"node-ca-cxrrh\" (UID: \"1b94f336-770a-4f5a-8ef9-cf867e720989\") " pod="openshift-image-registry/node-ca-cxrrh"
Jul 28 17:10:47.202281 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201924    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-multus-conf-dir\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.202281 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.201961    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-multus-conf-dir\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.202281 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202055    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-host-run-multus-certs\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.202281 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202081    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-etc-kubernetes\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.202281 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202107    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-host-run-netns\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.202281 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202132    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-run-openvswitch\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.202281 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202154    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f363ee59-3092-46f1-a3e5-a01b034a1e14-os-release\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.202281 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202162    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-etc-kubernetes\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.202281 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202180    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h82bd\" (UniqueName: \"kubernetes.io/projected/f85fcfde-92b5-45c4-9540-05d8dc614a9a-kube-api-access-h82bd\") pod \"iptables-alerter-45tdk\" (UID: \"f85fcfde-92b5-45c4-9540-05d8dc614a9a\") " pod="openshift-network-operator/iptables-alerter-45tdk"
Jul 28 17:10:47.203058 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202205    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-host-run-multus-certs\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.203058 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202211    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xsck\" (UniqueName: \"kubernetes.io/projected/97f130a6-e3b7-4882-b992-0d5eef10897b-kube-api-access-9xsck\") pod \"network-metrics-daemon-fr9xb\" (UID: \"97f130a6-e3b7-4882-b992-0d5eef10897b\") " pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:10:47.203058 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202249    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-run-openvswitch\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.203058 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202260    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1b94f336-770a-4f5a-8ef9-cf867e720989-serviceca\") pod \"node-ca-cxrrh\" (UID: \"1b94f336-770a-4f5a-8ef9-cf867e720989\") " pod="openshift-image-registry/node-ca-cxrrh"
Jul 28 17:10:47.203058 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202276    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f363ee59-3092-46f1-a3e5-a01b034a1e14-cni-binary-copy\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.203058 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202278    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8337ae34-e335-4033-8ecd-49dd32a2ea88-cni-binary-copy\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.203058 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202299    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f363ee59-3092-46f1-a3e5-a01b034a1e14-os-release\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.203058 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202312    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-etc-kubernetes\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.203058 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202349    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-host-run-netns\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.203058 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202360    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-etc-kubernetes\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.203058 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202375    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3f8e6a3b-524e-403d-bfdf-338bf7c5defc-agent-certs\") pod \"konnectivity-agent-l82pq\" (UID: \"3f8e6a3b-524e-403d-bfdf-338bf7c5defc\") " pod="kube-system/konnectivity-agent-l82pq"
Jul 28 17:10:47.203058 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202376    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-ovnkube-script-lib\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.203058 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202399    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-etc-systemd\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.203058 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202443    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-etc-systemd\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.203058 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202536    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-env-overrides\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.203058 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202565    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2kxj\" (UniqueName: \"kubernetes.io/projected/7ad540e6-ff3a-4366-9176-0063eb45135c-kube-api-access-c2kxj\") pod \"node-resolver-92rx8\" (UID: \"7ad540e6-ff3a-4366-9176-0063eb45135c\") " pod="openshift-dns/node-resolver-92rx8"
Jul 28 17:10:47.203058 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202587    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-host-kubelet\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.203994 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202614    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ec7b38b2-8859-4743-875a-4148a0c342b8-device-dir\") pod \"aws-ebs-csi-driver-node-4cjsk\" (UID: \"ec7b38b2-8859-4743-875a-4148a0c342b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.203994 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202651    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-etc-modprobe-d\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.203994 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202674    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-host-var-lib-cni-bin\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.203994 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202674    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-host-kubelet\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.203994 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202695    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-host-slash\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.203994 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202712    2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 17:10:47.203994 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202747    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ec7b38b2-8859-4743-875a-4148a0c342b8-device-dir\") pod \"aws-ebs-csi-driver-node-4cjsk\" (UID: \"ec7b38b2-8859-4743-875a-4148a0c342b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.203994 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202766    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ec7b38b2-8859-4743-875a-4148a0c342b8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4cjsk\" (UID: \"ec7b38b2-8859-4743-875a-4148a0c342b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.203994 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202765    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f363ee59-3092-46f1-a3e5-a01b034a1e14-cni-binary-copy\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.203994 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202719    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ec7b38b2-8859-4743-875a-4148a0c342b8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4cjsk\" (UID: \"ec7b38b2-8859-4743-875a-4148a0c342b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.203994 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202814    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpnx5\" (UniqueName: \"kubernetes.io/projected/f363ee59-3092-46f1-a3e5-a01b034a1e14-kube-api-access-lpnx5\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.203994 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202818    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-host-slash\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.203994 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202823    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-host-var-lib-cni-bin\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.203994 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202838    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-etc-modprobe-d\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.203994 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202839    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-host-var-lib-kubelet\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.203994 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202873    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-host-var-lib-kubelet\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.203994 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202877    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ad540e6-ff3a-4366-9176-0063eb45135c-tmp-dir\") pod \"node-resolver-92rx8\" (UID: \"7ad540e6-ff3a-4366-9176-0063eb45135c\") " pod="openshift-dns/node-resolver-92rx8"
Jul 28 17:10:47.203994 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202903    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-systemd-units\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.204939 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202947    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-var-lib-openvswitch\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.204939 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202953    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-env-overrides\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.204939 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202971    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-ovn-node-metrics-cert\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.204939 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.202994    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f363ee59-3092-46f1-a3e5-a01b034a1e14-cnibin\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.204939 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203014    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-systemd-units\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.204939 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203017    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-cnibin\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.204939 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203062    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-cnibin\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.204939 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203052    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8337ae34-e335-4033-8ecd-49dd32a2ea88-multus-daemon-config\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.204939 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203061    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f363ee59-3092-46f1-a3e5-a01b034a1e14-cnibin\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.204939 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203017    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-var-lib-openvswitch\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.204939 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203093    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgdk9\" (UniqueName: \"kubernetes.io/projected/961f03d7-1d8a-4728-aa72-9e35c4b4d931-kube-api-access-pgdk9\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.204939 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203104    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ad540e6-ff3a-4366-9176-0063eb45135c-tmp-dir\") pod \"node-resolver-92rx8\" (UID: \"7ad540e6-ff3a-4366-9176-0063eb45135c\") " pod="openshift-dns/node-resolver-92rx8"
Jul 28 17:10:47.204939 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203123    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-multus-socket-dir-parent\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.204939 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203149    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs\") pod \"network-metrics-daemon-fr9xb\" (UID: \"97f130a6-e3b7-4882-b992-0d5eef10897b\") " pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:10:47.204939 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203189    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f363ee59-3092-46f1-a3e5-a01b034a1e14-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.204939 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203196    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-multus-socket-dir-parent\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.204939 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203213    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-host\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.204939 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203263    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-host\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.205897 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203277    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4d9x\" (UniqueName: \"kubernetes.io/projected/a16a1fb7-d21c-49cf-a545-faa8ae9d20b6-kube-api-access-l4d9x\") pod \"network-check-target-67x96\" (UID: \"a16a1fb7-d21c-49cf-a545-faa8ae9d20b6\") " pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:10:47.205897 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203319    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-run-ovn\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.205897 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203339    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ec7b38b2-8859-4743-875a-4148a0c342b8-socket-dir\") pod \"aws-ebs-csi-driver-node-4cjsk\" (UID: \"ec7b38b2-8859-4743-875a-4148a0c342b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.205897 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203360    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ec7b38b2-8859-4743-875a-4148a0c342b8-sys-fs\") pod \"aws-ebs-csi-driver-node-4cjsk\" (UID: \"ec7b38b2-8859-4743-875a-4148a0c342b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.205897 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203380    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f363ee59-3092-46f1-a3e5-a01b034a1e14-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.205897 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203401    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-etc-sysctl-conf\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.205897 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203420    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-host-run-netns\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.205897 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203422    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-run-ovn\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.205897 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203440    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szx6x\" (UniqueName: \"kubernetes.io/projected/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-kube-api-access-szx6x\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.205897 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203462    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/961f03d7-1d8a-4728-aa72-9e35c4b4d931-etc-tuned\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.205897 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203463    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8337ae34-e335-4033-8ecd-49dd32a2ea88-multus-daemon-config\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.205897 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203524    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-host-run-netns\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.205897 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203638    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f363ee59-3092-46f1-a3e5-a01b034a1e14-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.205897 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203686    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-os-release\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.205897 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203712    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45znx\" (UniqueName: \"kubernetes.io/projected/8337ae34-e335-4033-8ecd-49dd32a2ea88-kube-api-access-45znx\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.205897 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203733    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ec7b38b2-8859-4743-875a-4148a0c342b8-sys-fs\") pod \"aws-ebs-csi-driver-node-4cjsk\" (UID: \"ec7b38b2-8859-4743-875a-4148a0c342b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.205897 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203763    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-host-run-ovn-kubernetes\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.206813 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203790    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f363ee59-3092-46f1-a3e5-a01b034a1e14-system-cni-dir\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.206813 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203814    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-lib-modules\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.206813 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203838    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.206813 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203884    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ad540e6-ff3a-4366-9176-0063eb45135c-hosts-file\") pod \"node-resolver-92rx8\" (UID: \"7ad540e6-ff3a-4366-9176-0063eb45135c\") " pod="openshift-dns/node-resolver-92rx8"
Jul 28 17:10:47.206813 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203907    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f363ee59-3092-46f1-a3e5-a01b034a1e14-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.206813 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203909    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-node-log\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.206813 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203939    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-node-log\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.206813 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203949    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-log-socket\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.206813 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203971    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.206813 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203988    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-etc-sysctl-conf\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.206813 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204007    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-log-socket\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.206813 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204009    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-host-cni-bin\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.206813 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.203973    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-host-cni-bin\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.206813 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204068    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-host-run-ovn-kubernetes\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.206813 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204082    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ec7b38b2-8859-4743-875a-4148a0c342b8-socket-dir\") pod \"aws-ebs-csi-driver-node-4cjsk\" (UID: \"ec7b38b2-8859-4743-875a-4148a0c342b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.206813 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204080    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f363ee59-3092-46f1-a3e5-a01b034a1e14-system-cni-dir\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.206813 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204104    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtwth\" (UniqueName: \"kubernetes.io/projected/ec7b38b2-8859-4743-875a-4148a0c342b8-kube-api-access-qtwth\") pod \"aws-ebs-csi-driver-node-4cjsk\" (UID: \"ec7b38b2-8859-4743-875a-4148a0c342b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.207519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204093    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ad540e6-ff3a-4366-9176-0063eb45135c-hosts-file\") pod \"node-resolver-92rx8\" (UID: \"7ad540e6-ff3a-4366-9176-0063eb45135c\") " pod="openshift-dns/node-resolver-92rx8"
Jul 28 17:10:47.207519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204137    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-sys\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.207519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204160    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3f8e6a3b-524e-403d-bfdf-338bf7c5defc-konnectivity-ca\") pod \"konnectivity-agent-l82pq\" (UID: \"3f8e6a3b-524e-403d-bfdf-338bf7c5defc\") " pod="kube-system/konnectivity-agent-l82pq"
Jul 28 17:10:47.207519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204174    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-lib-modules\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.207519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204183    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f85fcfde-92b5-45c4-9540-05d8dc614a9a-iptables-alerter-script\") pod \"iptables-alerter-45tdk\" (UID: \"f85fcfde-92b5-45c4-9540-05d8dc614a9a\") " pod="openshift-network-operator/iptables-alerter-45tdk"
Jul 28 17:10:47.207519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204206    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-system-cni-dir\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.207519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204227    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-host-var-lib-cni-multus\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.207519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204223    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-sys\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.207519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204250    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-hostroot\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.207519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204270    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/961f03d7-1d8a-4728-aa72-9e35c4b4d931-tmp\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.207519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204282    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-system-cni-dir\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.207519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204293    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-multus-cni-dir\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.207519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204327    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-hostroot\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.207519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204346    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-ovnkube-config\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.207519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204367    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-host-var-lib-cni-multus\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.207519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204370    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f363ee59-3092-46f1-a3e5-a01b034a1e14-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.207519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204398    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-run\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.207519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204420    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-var-lib-kubelet\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.208161 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204441    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f85fcfde-92b5-45c4-9540-05d8dc614a9a-host-slash\") pod \"iptables-alerter-45tdk\" (UID: \"f85fcfde-92b5-45c4-9540-05d8dc614a9a\") " pod="openshift-network-operator/iptables-alerter-45tdk"
Jul 28 17:10:47.208161 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204482    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-multus-cni-dir\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.208161 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204498    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f85fcfde-92b5-45c4-9540-05d8dc614a9a-host-slash\") pod \"iptables-alerter-45tdk\" (UID: \"f85fcfde-92b5-45c4-9540-05d8dc614a9a\") " pod="openshift-network-operator/iptables-alerter-45tdk"
Jul 28 17:10:47.208161 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204517    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f363ee59-3092-46f1-a3e5-a01b034a1e14-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.208161 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204559    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-run\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.208161 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204600    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/961f03d7-1d8a-4728-aa72-9e35c4b4d931-var-lib-kubelet\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.208161 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204656    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8337ae34-e335-4033-8ecd-49dd32a2ea88-os-release\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.208161 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204727    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3f8e6a3b-524e-403d-bfdf-338bf7c5defc-konnectivity-ca\") pod \"konnectivity-agent-l82pq\" (UID: \"3f8e6a3b-524e-403d-bfdf-338bf7c5defc\") " pod="kube-system/konnectivity-agent-l82pq"
Jul 28 17:10:47.208161 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204799    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f85fcfde-92b5-45c4-9540-05d8dc614a9a-iptables-alerter-script\") pod \"iptables-alerter-45tdk\" (UID: \"f85fcfde-92b5-45c4-9540-05d8dc614a9a\") " pod="openshift-network-operator/iptables-alerter-45tdk"
Jul 28 17:10:47.208161 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.204985    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-ovnkube-config\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.208161 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.206739    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3f8e6a3b-524e-403d-bfdf-338bf7c5defc-agent-certs\") pod \"konnectivity-agent-l82pq\" (UID: \"3f8e6a3b-524e-403d-bfdf-338bf7c5defc\") " pod="kube-system/konnectivity-agent-l82pq"
Jul 28 17:10:47.208161 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.206740    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/961f03d7-1d8a-4728-aa72-9e35c4b4d931-tmp\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.208161 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.207172    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-ovn-node-metrics-cert\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.208161 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.207211    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/961f03d7-1d8a-4728-aa72-9e35c4b4d931-etc-tuned\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.210854 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.210774    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h82bd\" (UniqueName: \"kubernetes.io/projected/f85fcfde-92b5-45c4-9540-05d8dc614a9a-kube-api-access-h82bd\") pod \"iptables-alerter-45tdk\" (UID: \"f85fcfde-92b5-45c4-9540-05d8dc614a9a\") " pod="openshift-network-operator/iptables-alerter-45tdk"
Jul 28 17:10:47.211035 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.211012    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkztd\" (UniqueName: \"kubernetes.io/projected/1b94f336-770a-4f5a-8ef9-cf867e720989-kube-api-access-nkztd\") pod \"node-ca-cxrrh\" (UID: \"1b94f336-770a-4f5a-8ef9-cf867e720989\") " pod="openshift-image-registry/node-ca-cxrrh"
Jul 28 17:10:47.212307 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.212282    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2kxj\" (UniqueName: \"kubernetes.io/projected/7ad540e6-ff3a-4366-9176-0063eb45135c-kube-api-access-c2kxj\") pod \"node-resolver-92rx8\" (UID: \"7ad540e6-ff3a-4366-9176-0063eb45135c\") " pod="openshift-dns/node-resolver-92rx8"
Jul 28 17:10:47.213327 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.213308    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgdk9\" (UniqueName: \"kubernetes.io/projected/961f03d7-1d8a-4728-aa72-9e35c4b4d931-kube-api-access-pgdk9\") pod \"tuned-tncc8\" (UID: \"961f03d7-1d8a-4728-aa72-9e35c4b4d931\") " pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.213435 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.213350    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtwth\" (UniqueName: \"kubernetes.io/projected/ec7b38b2-8859-4743-875a-4148a0c342b8-kube-api-access-qtwth\") pod \"aws-ebs-csi-driver-node-4cjsk\" (UID: \"ec7b38b2-8859-4743-875a-4148a0c342b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.213566 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.213540    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpnx5\" (UniqueName: \"kubernetes.io/projected/f363ee59-3092-46f1-a3e5-a01b034a1e14-kube-api-access-lpnx5\") pod \"multus-additional-cni-plugins-zxsfx\" (UID: \"f363ee59-3092-46f1-a3e5-a01b034a1e14\") " pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.213791 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.213742    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szx6x\" (UniqueName: \"kubernetes.io/projected/cd62d7f3-7620-47de-a1dd-4e3bc6ea5652-kube-api-access-szx6x\") pod \"ovnkube-node-v552f\" (UID: \"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652\") " pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.213866 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.213852    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45znx\" (UniqueName: \"kubernetes.io/projected/8337ae34-e335-4033-8ecd-49dd32a2ea88-kube-api-access-45znx\") pod \"multus-q4vcf\" (UID: \"8337ae34-e335-4033-8ecd-49dd32a2ea88\") " pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.285081 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.285038    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal" event={"ID":"f36b97c92bac0bfb582fa26faaad0625","Type":"ContainerStarted","Data":"e0fe24a5560e93f7725df6b7ef597714c1daf8620192d0506b380ecb06f07e3d"}
Jul 28 17:10:47.286176 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.286152    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal" event={"ID":"3da3e528494d7d2552a2b35126ff8b14","Type":"ContainerStarted","Data":"edd0bcb05b2801b5e5413c150f0f8ce98a1b122dac40822c1faaf21c07232805"}
Jul 28 17:10:47.305231 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.305205    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs\") pod \"network-metrics-daemon-fr9xb\" (UID: \"97f130a6-e3b7-4882-b992-0d5eef10897b\") " pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:10:47.305231 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.305237    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4d9x\" (UniqueName: \"kubernetes.io/projected/a16a1fb7-d21c-49cf-a545-faa8ae9d20b6-kube-api-access-l4d9x\") pod \"network-check-target-67x96\" (UID: \"a16a1fb7-d21c-49cf-a545-faa8ae9d20b6\") " pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:10:47.305369 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.305290    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xsck\" (UniqueName: \"kubernetes.io/projected/97f130a6-e3b7-4882-b992-0d5eef10897b-kube-api-access-9xsck\") pod \"network-metrics-daemon-fr9xb\" (UID: \"97f130a6-e3b7-4882-b992-0d5eef10897b\") " pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:10:47.305369 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:47.305362    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:10:47.305487 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:47.305460    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs podName:97f130a6-e3b7-4882-b992-0d5eef10897b nodeName:}" failed. No retries permitted until 2026-07-28 17:10:47.805443132 +0000 UTC m=+3.051545334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs") pod "network-metrics-daemon-fr9xb" (UID: "97f130a6-e3b7-4882-b992-0d5eef10897b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:10:47.314356 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:47.314303    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:10:47.314356 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:47.314324    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:10:47.314356 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:47.314337    2567 projected.go:196] Error preparing data for projected volume kube-api-access-l4d9x for pod openshift-network-diagnostics/network-check-target-67x96: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:10:47.314547 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:47.314403    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a16a1fb7-d21c-49cf-a545-faa8ae9d20b6-kube-api-access-l4d9x podName:a16a1fb7-d21c-49cf-a545-faa8ae9d20b6 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:47.81438789 +0000 UTC m=+3.060490093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-l4d9x" (UniqueName: "kubernetes.io/projected/a16a1fb7-d21c-49cf-a545-faa8ae9d20b6-kube-api-access-l4d9x") pod "network-check-target-67x96" (UID: "a16a1fb7-d21c-49cf-a545-faa8ae9d20b6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:10:47.317035 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.317013    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xsck\" (UniqueName: \"kubernetes.io/projected/97f130a6-e3b7-4882-b992-0d5eef10897b-kube-api-access-9xsck\") pod \"network-metrics-daemon-fr9xb\" (UID: \"97f130a6-e3b7-4882-b992-0d5eef10897b\") " pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:10:47.389258 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.389221    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-92rx8"
Jul 28 17:10:47.394914 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.394895    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l82pq"
Jul 28 17:10:47.396702 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:47.396683    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ad540e6_ff3a_4366_9176_0063eb45135c.slice/crio-508ce8c397670c2ef5f7a1406f6809f3df8249bcedb4df5b344d75f89e7d0ef7 WatchSource:0}: Error finding container 508ce8c397670c2ef5f7a1406f6809f3df8249bcedb4df5b344d75f89e7d0ef7: Status 404 returned error can't find the container with id 508ce8c397670c2ef5f7a1406f6809f3df8249bcedb4df5b344d75f89e7d0ef7
Jul 28 17:10:47.406223 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.406197    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-45tdk"
Jul 28 17:10:47.411070 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.409710    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:10:47.414489 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:47.414464    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f8e6a3b_524e_403d_bfdf_338bf7c5defc.slice/crio-6ff56d1306699f76e22b321ccab072bc574f483526c5e53e59ee06883adc83a3 WatchSource:0}: Error finding container 6ff56d1306699f76e22b321ccab072bc574f483526c5e53e59ee06883adc83a3: Status 404 returned error can't find the container with id 6ff56d1306699f76e22b321ccab072bc574f483526c5e53e59ee06883adc83a3
Jul 28 17:10:47.416003 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.415990    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk"
Jul 28 17:10:47.417321 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:47.417298    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf85fcfde_92b5_45c4_9540_05d8dc614a9a.slice/crio-b68f56094fe103de3086938e0f12ebc76fef06b213ce6341183d435631eed7bb WatchSource:0}: Error finding container b68f56094fe103de3086938e0f12ebc76fef06b213ce6341183d435631eed7bb: Status 404 returned error can't find the container with id b68f56094fe103de3086938e0f12ebc76fef06b213ce6341183d435631eed7bb
Jul 28 17:10:47.420128 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:47.420109    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd62d7f3_7620_47de_a1dd_4e3bc6ea5652.slice/crio-e97596130d82b0b426f6645391a0326fa7558f0fb968f6f980cf7a641012bb6b WatchSource:0}: Error finding container e97596130d82b0b426f6645391a0326fa7558f0fb968f6f980cf7a641012bb6b: Status 404 returned error can't find the container with id e97596130d82b0b426f6645391a0326fa7558f0fb968f6f980cf7a641012bb6b
Jul 28 17:10:47.423133 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.423115    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zxsfx"
Jul 28 17:10:47.424810 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:47.424791    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec7b38b2_8859_4743_875a_4148a0c342b8.slice/crio-1c807c86523fe1bf04ac1483849cc2736657c4213f21a8a1f5a9b3b941dd3ce0 WatchSource:0}: Error finding container 1c807c86523fe1bf04ac1483849cc2736657c4213f21a8a1f5a9b3b941dd3ce0: Status 404 returned error can't find the container with id 1c807c86523fe1bf04ac1483849cc2736657c4213f21a8a1f5a9b3b941dd3ce0
Jul 28 17:10:47.429724 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.429704    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tncc8"
Jul 28 17:10:47.429811 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:47.429765    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf363ee59_3092_46f1_a3e5_a01b034a1e14.slice/crio-99cde829847219a23d6cbf50110f6537c13954ffaf0a56567b21a35f982cebbc WatchSource:0}: Error finding container 99cde829847219a23d6cbf50110f6537c13954ffaf0a56567b21a35f982cebbc: Status 404 returned error can't find the container with id 99cde829847219a23d6cbf50110f6537c13954ffaf0a56567b21a35f982cebbc
Jul 28 17:10:47.437775 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.436805    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cxrrh"
Jul 28 17:10:47.438906 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:47.438885    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod961f03d7_1d8a_4728_aa72_9e35c4b4d931.slice/crio-b93432ca4336cfe79a86c872a254d7fb6d38770847d4801cc6973e51c881036a WatchSource:0}: Error finding container b93432ca4336cfe79a86c872a254d7fb6d38770847d4801cc6973e51c881036a: Status 404 returned error can't find the container with id b93432ca4336cfe79a86c872a254d7fb6d38770847d4801cc6973e51c881036a
Jul 28 17:10:47.441832 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.441705    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q4vcf"
Jul 28 17:10:47.444358 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:47.444336    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b94f336_770a_4f5a_8ef9_cf867e720989.slice/crio-daa811c17513e5cd57c849acaeb3db1c8e1d948d25f00dc8868593f18c91306c WatchSource:0}: Error finding container daa811c17513e5cd57c849acaeb3db1c8e1d948d25f00dc8868593f18c91306c: Status 404 returned error can't find the container with id daa811c17513e5cd57c849acaeb3db1c8e1d948d25f00dc8868593f18c91306c
Jul 28 17:10:47.448340 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:10:47.448320    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8337ae34_e335_4033_8ecd_49dd32a2ea88.slice/crio-444336c38e3aa85d73a2cfffa03c3b30ff77a3a0b5023b096d9f3fa6926f656d WatchSource:0}: Error finding container 444336c38e3aa85d73a2cfffa03c3b30ff77a3a0b5023b096d9f3fa6926f656d: Status 404 returned error can't find the container with id 444336c38e3aa85d73a2cfffa03c3b30ff77a3a0b5023b096d9f3fa6926f656d
Jul 28 17:10:47.708905 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.708695    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/482929b8-7c61-4fee-bd3b-0f9ac86e61df-original-pull-secret\") pod \"global-pull-secret-syncer-vcff5\" (UID: \"482929b8-7c61-4fee-bd3b-0f9ac86e61df\") " pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:10:47.709011 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:47.708880    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:10:47.709011 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:47.708996    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/482929b8-7c61-4fee-bd3b-0f9ac86e61df-original-pull-secret podName:482929b8-7c61-4fee-bd3b-0f9ac86e61df nodeName:}" failed. No retries permitted until 2026-07-28 17:10:48.708976761 +0000 UTC m=+3.955078966 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/482929b8-7c61-4fee-bd3b-0f9ac86e61df-original-pull-secret") pod "global-pull-secret-syncer-vcff5" (UID: "482929b8-7c61-4fee-bd3b-0f9ac86e61df") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:10:47.810229 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.810196    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs\") pod \"network-metrics-daemon-fr9xb\" (UID: \"97f130a6-e3b7-4882-b992-0d5eef10897b\") " pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:10:47.810790 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:47.810370    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:10:47.810790 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:47.810444    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs podName:97f130a6-e3b7-4882-b992-0d5eef10897b nodeName:}" failed. No retries permitted until 2026-07-28 17:10:48.810424419 +0000 UTC m=+4.056526609 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs") pod "network-metrics-daemon-fr9xb" (UID: "97f130a6-e3b7-4882-b992-0d5eef10897b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:10:47.911055 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:47.910946    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4d9x\" (UniqueName: \"kubernetes.io/projected/a16a1fb7-d21c-49cf-a545-faa8ae9d20b6-kube-api-access-l4d9x\") pod \"network-check-target-67x96\" (UID: \"a16a1fb7-d21c-49cf-a545-faa8ae9d20b6\") " pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:10:47.911191 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:47.911106    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:10:47.911191 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:47.911129    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:10:47.911191 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:47.911141    2567 projected.go:196] Error preparing data for projected volume kube-api-access-l4d9x for pod openshift-network-diagnostics/network-check-target-67x96: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:10:47.911345 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:47.911199    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a16a1fb7-d21c-49cf-a545-faa8ae9d20b6-kube-api-access-l4d9x podName:a16a1fb7-d21c-49cf-a545-faa8ae9d20b6 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:48.911180596 +0000 UTC m=+4.157282795 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4d9x" (UniqueName: "kubernetes.io/projected/a16a1fb7-d21c-49cf-a545-faa8ae9d20b6-kube-api-access-l4d9x") pod "network-check-target-67x96" (UID: "a16a1fb7-d21c-49cf-a545-faa8ae9d20b6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:10:48.149647 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:48.149565    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:05:47 +0000 UTC" deadline="2028-03-05 20:57:48.820522589 +0000 UTC"
Jul 28 17:10:48.149647 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:48.149599    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14067h47m0.670927717s"
Jul 28 17:10:48.292272 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:48.292065    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-45tdk" event={"ID":"f85fcfde-92b5-45c4-9540-05d8dc614a9a","Type":"ContainerStarted","Data":"b68f56094fe103de3086938e0f12ebc76fef06b213ce6341183d435631eed7bb"}
Jul 28 17:10:48.294568 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:48.294536    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l82pq" event={"ID":"3f8e6a3b-524e-403d-bfdf-338bf7c5defc","Type":"ContainerStarted","Data":"6ff56d1306699f76e22b321ccab072bc574f483526c5e53e59ee06883adc83a3"}
Jul 28 17:10:48.297079 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:48.297014    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-92rx8" event={"ID":"7ad540e6-ff3a-4366-9176-0063eb45135c","Type":"ContainerStarted","Data":"508ce8c397670c2ef5f7a1406f6809f3df8249bcedb4df5b344d75f89e7d0ef7"}
Jul 28 17:10:48.301039 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:48.301014    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q4vcf" event={"ID":"8337ae34-e335-4033-8ecd-49dd32a2ea88","Type":"ContainerStarted","Data":"444336c38e3aa85d73a2cfffa03c3b30ff77a3a0b5023b096d9f3fa6926f656d"}
Jul 28 17:10:48.304203 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:48.304175    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cxrrh" event={"ID":"1b94f336-770a-4f5a-8ef9-cf867e720989","Type":"ContainerStarted","Data":"daa811c17513e5cd57c849acaeb3db1c8e1d948d25f00dc8868593f18c91306c"}
Jul 28 17:10:48.306533 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:48.306481    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zxsfx" event={"ID":"f363ee59-3092-46f1-a3e5-a01b034a1e14","Type":"ContainerStarted","Data":"99cde829847219a23d6cbf50110f6537c13954ffaf0a56567b21a35f982cebbc"}
Jul 28 17:10:48.310968 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:48.310444    2567 generic.go:356] "Generic (PLEG): container finished" podID="3da3e528494d7d2552a2b35126ff8b14" containerID="4c595cd203f8488282c0a01c1a09188864dd39f9f576f730038dc79b56a0b8f7" exitCode=0
Jul 28 17:10:48.310968 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:48.310539    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal" event={"ID":"3da3e528494d7d2552a2b35126ff8b14","Type":"ContainerDied","Data":"4c595cd203f8488282c0a01c1a09188864dd39f9f576f730038dc79b56a0b8f7"}
Jul 28 17:10:48.318680 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:48.318655    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tncc8" event={"ID":"961f03d7-1d8a-4728-aa72-9e35c4b4d931","Type":"ContainerStarted","Data":"b93432ca4336cfe79a86c872a254d7fb6d38770847d4801cc6973e51c881036a"}
Jul 28 17:10:48.322218 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:48.322194    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk" event={"ID":"ec7b38b2-8859-4743-875a-4148a0c342b8","Type":"ContainerStarted","Data":"1c807c86523fe1bf04ac1483849cc2736657c4213f21a8a1f5a9b3b941dd3ce0"}
Jul 28 17:10:48.327160 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:48.327064    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v552f" event={"ID":"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652","Type":"ContainerStarted","Data":"e97596130d82b0b426f6645391a0326fa7558f0fb968f6f980cf7a641012bb6b"}
Jul 28 17:10:48.718064 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:48.717975    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/482929b8-7c61-4fee-bd3b-0f9ac86e61df-original-pull-secret\") pod \"global-pull-secret-syncer-vcff5\" (UID: \"482929b8-7c61-4fee-bd3b-0f9ac86e61df\") " pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:10:48.718231 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:48.718136    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:10:48.718231 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:48.718203    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/482929b8-7c61-4fee-bd3b-0f9ac86e61df-original-pull-secret podName:482929b8-7c61-4fee-bd3b-0f9ac86e61df nodeName:}" failed. No retries permitted until 2026-07-28 17:10:50.718184807 +0000 UTC m=+5.964286995 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/482929b8-7c61-4fee-bd3b-0f9ac86e61df-original-pull-secret") pod "global-pull-secret-syncer-vcff5" (UID: "482929b8-7c61-4fee-bd3b-0f9ac86e61df") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:10:48.819123 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:48.818586    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs\") pod \"network-metrics-daemon-fr9xb\" (UID: \"97f130a6-e3b7-4882-b992-0d5eef10897b\") " pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:10:48.819123 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:48.818730    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:10:48.819123 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:48.818788    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs podName:97f130a6-e3b7-4882-b992-0d5eef10897b nodeName:}" failed. No retries permitted until 2026-07-28 17:10:50.818768978 +0000 UTC m=+6.064871178 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs") pod "network-metrics-daemon-fr9xb" (UID: "97f130a6-e3b7-4882-b992-0d5eef10897b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:10:48.920111 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:48.919888    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4d9x\" (UniqueName: \"kubernetes.io/projected/a16a1fb7-d21c-49cf-a545-faa8ae9d20b6-kube-api-access-l4d9x\") pod \"network-check-target-67x96\" (UID: \"a16a1fb7-d21c-49cf-a545-faa8ae9d20b6\") " pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:10:48.922856 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:48.920702    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:10:48.922856 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:48.920737    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:10:48.922856 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:48.920752    2567 projected.go:196] Error preparing data for projected volume kube-api-access-l4d9x for pod openshift-network-diagnostics/network-check-target-67x96: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:10:48.922856 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:48.920825    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a16a1fb7-d21c-49cf-a545-faa8ae9d20b6-kube-api-access-l4d9x podName:a16a1fb7-d21c-49cf-a545-faa8ae9d20b6 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:50.920803675 +0000 UTC m=+6.166905875 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4d9x" (UniqueName: "kubernetes.io/projected/a16a1fb7-d21c-49cf-a545-faa8ae9d20b6-kube-api-access-l4d9x") pod "network-check-target-67x96" (UID: "a16a1fb7-d21c-49cf-a545-faa8ae9d20b6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:10:49.149876 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:49.149688    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:05:47 +0000 UTC" deadline="2028-01-23 20:20:07.471982055 +0000 UTC"
Jul 28 17:10:49.149876 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:49.149845    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13059h9m18.322142674s"
Jul 28 17:10:49.283442 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:49.282647    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:10:49.283442 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:49.282763    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67x96" podUID="a16a1fb7-d21c-49cf-a545-faa8ae9d20b6"
Jul 28 17:10:49.283442 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:49.283158    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:10:49.283442 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:49.283246    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcff5" podUID="482929b8-7c61-4fee-bd3b-0f9ac86e61df"
Jul 28 17:10:49.283442 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:49.283325    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:10:49.283442 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:49.283402    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fr9xb" podUID="97f130a6-e3b7-4882-b992-0d5eef10897b"
Jul 28 17:10:50.734497 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:50.734462    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/482929b8-7c61-4fee-bd3b-0f9ac86e61df-original-pull-secret\") pod \"global-pull-secret-syncer-vcff5\" (UID: \"482929b8-7c61-4fee-bd3b-0f9ac86e61df\") " pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:10:50.734925 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:50.734679    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:10:50.734925 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:50.734746    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/482929b8-7c61-4fee-bd3b-0f9ac86e61df-original-pull-secret podName:482929b8-7c61-4fee-bd3b-0f9ac86e61df nodeName:}" failed. No retries permitted until 2026-07-28 17:10:54.734727079 +0000 UTC m=+9.980829267 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/482929b8-7c61-4fee-bd3b-0f9ac86e61df-original-pull-secret") pod "global-pull-secret-syncer-vcff5" (UID: "482929b8-7c61-4fee-bd3b-0f9ac86e61df") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:10:50.835721 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:50.835684    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs\") pod \"network-metrics-daemon-fr9xb\" (UID: \"97f130a6-e3b7-4882-b992-0d5eef10897b\") " pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:10:50.835892 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:50.835836    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:10:50.835971 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:50.835903    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs podName:97f130a6-e3b7-4882-b992-0d5eef10897b nodeName:}" failed. No retries permitted until 2026-07-28 17:10:54.835883433 +0000 UTC m=+10.081985622 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs") pod "network-metrics-daemon-fr9xb" (UID: "97f130a6-e3b7-4882-b992-0d5eef10897b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:10:50.936537 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:50.936333    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4d9x\" (UniqueName: \"kubernetes.io/projected/a16a1fb7-d21c-49cf-a545-faa8ae9d20b6-kube-api-access-l4d9x\") pod \"network-check-target-67x96\" (UID: \"a16a1fb7-d21c-49cf-a545-faa8ae9d20b6\") " pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:10:50.937216 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:50.936761    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:10:50.937216 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:50.936790    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:10:50.937216 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:50.936803    2567 projected.go:196] Error preparing data for projected volume kube-api-access-l4d9x for pod openshift-network-diagnostics/network-check-target-67x96: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:10:50.937216 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:50.936881    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a16a1fb7-d21c-49cf-a545-faa8ae9d20b6-kube-api-access-l4d9x podName:a16a1fb7-d21c-49cf-a545-faa8ae9d20b6 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:54.936842858 +0000 UTC m=+10.182945049 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4d9x" (UniqueName: "kubernetes.io/projected/a16a1fb7-d21c-49cf-a545-faa8ae9d20b6-kube-api-access-l4d9x") pod "network-check-target-67x96" (UID: "a16a1fb7-d21c-49cf-a545-faa8ae9d20b6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:10:51.281596 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:51.281492    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:10:51.281596 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:51.281550    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:10:51.281836 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:51.281499    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:10:51.281836 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:51.281687    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fr9xb" podUID="97f130a6-e3b7-4882-b992-0d5eef10897b"
Jul 28 17:10:51.281836 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:51.281763    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67x96" podUID="a16a1fb7-d21c-49cf-a545-faa8ae9d20b6"
Jul 28 17:10:51.281994 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:51.281845    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcff5" podUID="482929b8-7c61-4fee-bd3b-0f9ac86e61df"
Jul 28 17:10:53.281962 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:53.281646    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:10:53.281962 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:53.281680    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:10:53.281962 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:53.281646    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:10:53.281962 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:53.281787    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fr9xb" podUID="97f130a6-e3b7-4882-b992-0d5eef10897b"
Jul 28 17:10:53.281962 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:53.281855    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67x96" podUID="a16a1fb7-d21c-49cf-a545-faa8ae9d20b6"
Jul 28 17:10:53.281962 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:53.281932    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcff5" podUID="482929b8-7c61-4fee-bd3b-0f9ac86e61df"
Jul 28 17:10:54.766033 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:54.765997    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/482929b8-7c61-4fee-bd3b-0f9ac86e61df-original-pull-secret\") pod \"global-pull-secret-syncer-vcff5\" (UID: \"482929b8-7c61-4fee-bd3b-0f9ac86e61df\") " pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:10:54.766482 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:54.766159    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:10:54.766482 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:54.766222    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/482929b8-7c61-4fee-bd3b-0f9ac86e61df-original-pull-secret podName:482929b8-7c61-4fee-bd3b-0f9ac86e61df nodeName:}" failed. No retries permitted until 2026-07-28 17:11:02.766204824 +0000 UTC m=+18.012307030 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/482929b8-7c61-4fee-bd3b-0f9ac86e61df-original-pull-secret") pod "global-pull-secret-syncer-vcff5" (UID: "482929b8-7c61-4fee-bd3b-0f9ac86e61df") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:10:54.867451 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:54.867343    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs\") pod \"network-metrics-daemon-fr9xb\" (UID: \"97f130a6-e3b7-4882-b992-0d5eef10897b\") " pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:10:54.867618 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:54.867489    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:10:54.867618 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:54.867570    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs podName:97f130a6-e3b7-4882-b992-0d5eef10897b nodeName:}" failed. No retries permitted until 2026-07-28 17:11:02.8675517 +0000 UTC m=+18.113653904 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs") pod "network-metrics-daemon-fr9xb" (UID: "97f130a6-e3b7-4882-b992-0d5eef10897b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:10:54.968781 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:54.968197    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4d9x\" (UniqueName: \"kubernetes.io/projected/a16a1fb7-d21c-49cf-a545-faa8ae9d20b6-kube-api-access-l4d9x\") pod \"network-check-target-67x96\" (UID: \"a16a1fb7-d21c-49cf-a545-faa8ae9d20b6\") " pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:10:54.968781 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:54.968351    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:10:54.968781 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:54.968371    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:10:54.968781 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:54.968383    2567 projected.go:196] Error preparing data for projected volume kube-api-access-l4d9x for pod openshift-network-diagnostics/network-check-target-67x96: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:10:54.968781 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:54.968442    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a16a1fb7-d21c-49cf-a545-faa8ae9d20b6-kube-api-access-l4d9x podName:a16a1fb7-d21c-49cf-a545-faa8ae9d20b6 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:02.968422941 +0000 UTC m=+18.214525148 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4d9x" (UniqueName: "kubernetes.io/projected/a16a1fb7-d21c-49cf-a545-faa8ae9d20b6-kube-api-access-l4d9x") pod "network-check-target-67x96" (UID: "a16a1fb7-d21c-49cf-a545-faa8ae9d20b6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:10:55.282597 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:55.282423    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:10:55.282597 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:55.282562    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fr9xb" podUID="97f130a6-e3b7-4882-b992-0d5eef10897b"
Jul 28 17:10:55.283025 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:55.282946    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:10:55.283150 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:55.283040    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67x96" podUID="a16a1fb7-d21c-49cf-a545-faa8ae9d20b6"
Jul 28 17:10:55.283150 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:55.283092    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:10:55.283265 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:55.283157    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcff5" podUID="482929b8-7c61-4fee-bd3b-0f9ac86e61df"
Jul 28 17:10:57.281882 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:57.281851    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:10:57.281882 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:57.281865    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:10:57.282398 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:57.281971    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcff5" podUID="482929b8-7c61-4fee-bd3b-0f9ac86e61df"
Jul 28 17:10:57.282398 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:57.282019    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fr9xb" podUID="97f130a6-e3b7-4882-b992-0d5eef10897b"
Jul 28 17:10:57.282398 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:57.282076    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:10:57.282398 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:57.282194    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67x96" podUID="a16a1fb7-d21c-49cf-a545-faa8ae9d20b6"
Jul 28 17:10:59.281691 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:59.281658    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:10:59.282113 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:59.281668    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:10:59.282113 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:59.281771    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67x96" podUID="a16a1fb7-d21c-49cf-a545-faa8ae9d20b6"
Jul 28 17:10:59.282113 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:59.281907    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcff5" podUID="482929b8-7c61-4fee-bd3b-0f9ac86e61df"
Jul 28 17:10:59.282113 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:10:59.281968    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:10:59.282113 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:10:59.282048    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fr9xb" podUID="97f130a6-e3b7-4882-b992-0d5eef10897b"
Jul 28 17:11:01.281979 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:01.281949    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:11:01.282405 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:01.281987    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:11:01.282405 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:01.281956    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:11:01.282405 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:01.282066    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcff5" podUID="482929b8-7c61-4fee-bd3b-0f9ac86e61df"
Jul 28 17:11:01.282405 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:01.282176    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fr9xb" podUID="97f130a6-e3b7-4882-b992-0d5eef10897b"
Jul 28 17:11:01.282405 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:01.282216    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67x96" podUID="a16a1fb7-d21c-49cf-a545-faa8ae9d20b6"
Jul 28 17:11:02.831844 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:02.831812    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/482929b8-7c61-4fee-bd3b-0f9ac86e61df-original-pull-secret\") pod \"global-pull-secret-syncer-vcff5\" (UID: \"482929b8-7c61-4fee-bd3b-0f9ac86e61df\") " pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:11:02.832324 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:02.831955    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:11:02.832324 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:02.832016    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/482929b8-7c61-4fee-bd3b-0f9ac86e61df-original-pull-secret podName:482929b8-7c61-4fee-bd3b-0f9ac86e61df nodeName:}" failed. No retries permitted until 2026-07-28 17:11:18.831997711 +0000 UTC m=+34.078099898 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/482929b8-7c61-4fee-bd3b-0f9ac86e61df-original-pull-secret") pod "global-pull-secret-syncer-vcff5" (UID: "482929b8-7c61-4fee-bd3b-0f9ac86e61df") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:11:02.932411 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:02.932379    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs\") pod \"network-metrics-daemon-fr9xb\" (UID: \"97f130a6-e3b7-4882-b992-0d5eef10897b\") " pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:11:02.932561 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:02.932518    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:11:02.932614 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:02.932570    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs podName:97f130a6-e3b7-4882-b992-0d5eef10897b nodeName:}" failed. No retries permitted until 2026-07-28 17:11:18.93255283 +0000 UTC m=+34.178655028 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs") pod "network-metrics-daemon-fr9xb" (UID: "97f130a6-e3b7-4882-b992-0d5eef10897b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:11:03.033141 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:03.033115    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4d9x\" (UniqueName: \"kubernetes.io/projected/a16a1fb7-d21c-49cf-a545-faa8ae9d20b6-kube-api-access-l4d9x\") pod \"network-check-target-67x96\" (UID: \"a16a1fb7-d21c-49cf-a545-faa8ae9d20b6\") " pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:11:03.033299 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:03.033234    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:11:03.033299 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:03.033248    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:11:03.033299 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:03.033257    2567 projected.go:196] Error preparing data for projected volume kube-api-access-l4d9x for pod openshift-network-diagnostics/network-check-target-67x96: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:11:03.033456 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:03.033308    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a16a1fb7-d21c-49cf-a545-faa8ae9d20b6-kube-api-access-l4d9x podName:a16a1fb7-d21c-49cf-a545-faa8ae9d20b6 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:19.033293726 +0000 UTC m=+34.279395927 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4d9x" (UniqueName: "kubernetes.io/projected/a16a1fb7-d21c-49cf-a545-faa8ae9d20b6-kube-api-access-l4d9x") pod "network-check-target-67x96" (UID: "a16a1fb7-d21c-49cf-a545-faa8ae9d20b6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:11:03.282027 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:03.281991    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:11:03.282202 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:03.281996    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:11:03.282202 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:03.282131    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:11:03.282202 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:03.282126    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fr9xb" podUID="97f130a6-e3b7-4882-b992-0d5eef10897b"
Jul 28 17:11:03.282338 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:03.282247    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcff5" podUID="482929b8-7c61-4fee-bd3b-0f9ac86e61df"
Jul 28 17:11:03.282338 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:03.282329    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67x96" podUID="a16a1fb7-d21c-49cf-a545-faa8ae9d20b6"
Jul 28 17:11:05.282834 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:05.282800    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:11:05.283346 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:05.282909    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcff5" podUID="482929b8-7c61-4fee-bd3b-0f9ac86e61df"
Jul 28 17:11:05.283346 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:05.282940    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:11:05.283346 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:05.283042    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fr9xb" podUID="97f130a6-e3b7-4882-b992-0d5eef10897b"
Jul 28 17:11:05.283346 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:05.283082    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:11:05.283346 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:05.283129    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67x96" podUID="a16a1fb7-d21c-49cf-a545-faa8ae9d20b6"
Jul 28 17:11:06.365786 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:06.365600    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal" event={"ID":"f36b97c92bac0bfb582fa26faaad0625","Type":"ContainerStarted","Data":"ce8e74abbe8260a417e57feceb250c838e7384b5cd69c0cc34afbce7a8877390"}
Jul 28 17:11:06.367380 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:06.367360    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal" event={"ID":"3da3e528494d7d2552a2b35126ff8b14","Type":"ContainerStarted","Data":"c8d8907c0369bdb71f85eff48bc26878504163d742c91fa30e7ab768e325f097"}
Jul 28 17:11:06.368647 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:06.368627    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tncc8" event={"ID":"961f03d7-1d8a-4728-aa72-9e35c4b4d931","Type":"ContainerStarted","Data":"9d52ecd570f3eb8f596c4be775aa4a334dc2468c8df92acb79be902fb1b083f5"}
Jul 28 17:11:06.370141 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:06.370115    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk" event={"ID":"ec7b38b2-8859-4743-875a-4148a0c342b8","Type":"ContainerStarted","Data":"0d2adf0cba318374d243602c829a23fb96ad023e938dab00da93785606c12f35"}
Jul 28 17:11:06.371831 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:06.371810    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v552f" event={"ID":"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652","Type":"ContainerStarted","Data":"713acb27e843df283565cd76f18b3c526eaa84a779501ad22cec651b7fb387f5"}
Jul 28 17:11:06.371915 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:06.371880    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v552f" event={"ID":"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652","Type":"ContainerStarted","Data":"245d42ba0132d6083d885fa2eb6bd2b52bf581fd74b56b51d9a2930548d10132"}
Jul 28 17:11:06.373106 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:06.373083    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l82pq" event={"ID":"3f8e6a3b-524e-403d-bfdf-338bf7c5defc","Type":"ContainerStarted","Data":"3682ea3b79b41107d0aaf6ceba9d8a1af95fe3ecd657b2e6bcfab0e464c44362"}
Jul 28 17:11:06.374457 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:06.374435    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-92rx8" event={"ID":"7ad540e6-ff3a-4366-9176-0063eb45135c","Type":"ContainerStarted","Data":"61193f332a3dbdf1a3df5357385bd646603160fdd2d1ca1e4205b6aaa21d8f4f"}
Jul 28 17:11:06.375921 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:06.375901    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cxrrh" event={"ID":"1b94f336-770a-4f5a-8ef9-cf867e720989","Type":"ContainerStarted","Data":"8b86318667fff66e16ac435dc32e6f8fbd7adf262d0a637db41487c313aaf4ea"}
Jul 28 17:11:06.378137 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:06.377753    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zxsfx" event={"ID":"f363ee59-3092-46f1-a3e5-a01b034a1e14","Type":"ContainerStarted","Data":"2778ef07e356a543ef621cf044e99fda50822cadbddf68c3609ffaca7d3b6af0"}
Jul 28 17:11:06.398812 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:06.398777    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal" podStartSLOduration=20.398765292 podStartE2EDuration="20.398765292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:06.381799664 +0000 UTC m=+21.627901870" watchObservedRunningTime="2026-07-28 17:11:06.398765292 +0000 UTC m=+21.644867479"
Jul 28 17:11:06.399226 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:06.399204    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal" podStartSLOduration=20.39919891 podStartE2EDuration="20.39919891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:06.398887982 +0000 UTC m=+21.644990171" watchObservedRunningTime="2026-07-28 17:11:06.39919891 +0000 UTC m=+21.645301116"
Jul 28 17:11:06.415366 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:06.415335    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-92rx8" podStartSLOduration=3.29248957 podStartE2EDuration="21.41532534s" totalImagesPullingTime="18.12283577s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:06.414872688 +0000 UTC m=+21.660974894" watchObservedRunningTime="2026-07-28 17:11:06.41532534 +0000 UTC m=+21.661427546"
Jul 28 17:11:06.437313 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:06.437271    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tncc8" podStartSLOduration=2.939853304 podStartE2EDuration="21.437255956s" totalImagesPullingTime="18.497402652s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:06.43677862 +0000 UTC m=+21.682880830" watchObservedRunningTime="2026-07-28 17:11:06.437255956 +0000 UTC m=+21.683358164"
Jul 28 17:11:06.452155 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:06.452122    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-l82pq" podStartSLOduration=3.338720129 podStartE2EDuration="21.452110575s" totalImagesPullingTime="18.113390446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:10:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:06.452006148 +0000 UTC m=+21.698108355" watchObservedRunningTime="2026-07-28 17:11:06.452110575 +0000 UTC m=+21.698212780"
Jul 28 17:11:06.503222 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:06.503108    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cxrrh" podStartSLOduration=3.419567883 podStartE2EDuration="21.503094042s" totalImagesPullingTime="18.083526159s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:06.502895018 +0000 UTC m=+21.748997223" watchObservedRunningTime="2026-07-28 17:11:06.503094042 +0000 UTC m=+21.749196248"
Jul 28 17:11:07.281766 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:07.281747    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:11:07.281909 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:07.281745    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:11:07.281909 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:07.281848    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fr9xb" podUID="97f130a6-e3b7-4882-b992-0d5eef10897b"
Jul 28 17:11:07.281998 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:07.281927    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcff5" podUID="482929b8-7c61-4fee-bd3b-0f9ac86e61df"
Jul 28 17:11:07.281998 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:07.281981    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:11:07.282069 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:07.282037    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67x96" podUID="a16a1fb7-d21c-49cf-a545-faa8ae9d20b6"
Jul 28 17:11:07.382297 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:07.382275    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:11:07.382879 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:07.382639    2567 generic.go:356] "Generic (PLEG): container finished" podID="cd62d7f3-7620-47de-a1dd-4e3bc6ea5652" containerID="713acb27e843df283565cd76f18b3c526eaa84a779501ad22cec651b7fb387f5" exitCode=1
Jul 28 17:11:07.382879 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:07.382712    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v552f" event={"ID":"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652","Type":"ContainerDied","Data":"713acb27e843df283565cd76f18b3c526eaa84a779501ad22cec651b7fb387f5"}
Jul 28 17:11:07.382879 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:07.382788    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v552f" event={"ID":"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652","Type":"ContainerStarted","Data":"b94caa46b3c4d990399788c4e7bc57d8ee1aaa552f07438c176f71f3fc027d60"}
Jul 28 17:11:07.382879 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:07.382830    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v552f" event={"ID":"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652","Type":"ContainerStarted","Data":"edc21b44def5cea34c9bb297dac1b1172a3deb5024ca69ab5270b126854c820d"}
Jul 28 17:11:07.382879 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:07.382860    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v552f" event={"ID":"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652","Type":"ContainerStarted","Data":"ad176ea8fcc6c76cc1be041aa1a19ad1c5dee4cba8df952e1e3fbfcdf181f967"}
Jul 28 17:11:07.383050 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:07.382903    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v552f" event={"ID":"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652","Type":"ContainerStarted","Data":"c162ec11fd083f5df3895099abb1550fd595b3d51d0b7d55c6f7a81e0c5144a7"}
Jul 28 17:11:07.384042 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:07.384023    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-45tdk" event={"ID":"f85fcfde-92b5-45c4-9540-05d8dc614a9a","Type":"ContainerStarted","Data":"331217f4aab3ced317766e2793acf64fe5440737fbf3af7436675a9fa5df31c6"}
Jul 28 17:11:07.385409 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:07.385372    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q4vcf" event={"ID":"8337ae34-e335-4033-8ecd-49dd32a2ea88","Type":"ContainerStarted","Data":"cc765ca0b6666170547c60690496286ec311afecd2887ea086c468a64f0a662b"}
Jul 28 17:11:07.386777 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:07.386757    2567 generic.go:356] "Generic (PLEG): container finished" podID="f363ee59-3092-46f1-a3e5-a01b034a1e14" containerID="2778ef07e356a543ef621cf044e99fda50822cadbddf68c3609ffaca7d3b6af0" exitCode=0
Jul 28 17:11:07.386865 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:07.386841    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zxsfx" event={"ID":"f363ee59-3092-46f1-a3e5-a01b034a1e14","Type":"ContainerDied","Data":"2778ef07e356a543ef621cf044e99fda50822cadbddf68c3609ffaca7d3b6af0"}
Jul 28 17:11:07.400121 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:07.400088    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-45tdk" podStartSLOduration=3.915669043 podStartE2EDuration="22.400076928s" totalImagesPullingTime="18.484407885s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:07.400028018 +0000 UTC m=+22.646130223" watchObservedRunningTime="2026-07-28 17:11:07.400076928 +0000 UTC m=+22.646179134"
Jul 28 17:11:07.439753 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:07.439721    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-q4vcf" podStartSLOduration=3.359179195 podStartE2EDuration="22.439710821s" totalImagesPullingTime="19.080531626s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:07.439237671 +0000 UTC m=+22.685339877" watchObservedRunningTime="2026-07-28 17:11:07.439710821 +0000 UTC m=+22.685813027"
Jul 28 17:11:07.604097 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:07.603971    2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 17:11:08.154615 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:08.154425    2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T17:11:07.604093417Z","UUID":"de983308-dbfe-4ea0-9ad1-3bd922c1a8cb","Handler":null,"Name":"","Endpoint":""}
Jul 28 17:11:08.156240 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:08.156217    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 17:11:08.156359 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:08.156263    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 17:11:08.390838 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:08.390805    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk" event={"ID":"ec7b38b2-8859-4743-875a-4148a0c342b8","Type":"ContainerStarted","Data":"7804fff05b79484baa827668e95b2c390356e8fb5ea962b2a2b04356eddd36de"}
Jul 28 17:11:09.281530 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:09.281469    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:11:09.281530 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:09.281486    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:11:09.281761 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:09.281610    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:11:09.281761 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:09.281610    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcff5" podUID="482929b8-7c61-4fee-bd3b-0f9ac86e61df"
Jul 28 17:11:09.281761 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:09.281725    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fr9xb" podUID="97f130a6-e3b7-4882-b992-0d5eef10897b"
Jul 28 17:11:09.281916 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:09.281810    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67x96" podUID="a16a1fb7-d21c-49cf-a545-faa8ae9d20b6"
Jul 28 17:11:09.395626 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:09.395591    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk" event={"ID":"ec7b38b2-8859-4743-875a-4148a0c342b8","Type":"ContainerStarted","Data":"d30970590b5bd434d0071fcd58ab545ad214fec5d75c4052eeb082019c9bd0bd"}
Jul 28 17:11:09.398951 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:09.398932    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:11:09.399364 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:09.399328    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v552f" event={"ID":"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652","Type":"ContainerStarted","Data":"c897ac88d7a8c74ba4424ca7606a89b2cdd38c35a831732f841f0514bd9bbe57"}
Jul 28 17:11:09.416075 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:09.416033    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4cjsk" podStartSLOduration=3.685597022 podStartE2EDuration="24.416016928s" totalImagesPullingTime="20.730419906s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:09.415860231 +0000 UTC m=+24.661962442" watchObservedRunningTime="2026-07-28 17:11:09.416016928 +0000 UTC m=+24.662119140"
Jul 28 17:11:10.729532 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:10.729480    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-l82pq"
Jul 28 17:11:10.730253 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:10.730237    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-l82pq"
Jul 28 17:11:11.281566 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:11.281500    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:11:11.281744 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:11.281645    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcff5" podUID="482929b8-7c61-4fee-bd3b-0f9ac86e61df"
Jul 28 17:11:11.281744 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:11.281500    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:11:11.281744 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:11.281540    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:11:11.281905 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:11.281751    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fr9xb" podUID="97f130a6-e3b7-4882-b992-0d5eef10897b"
Jul 28 17:11:11.281905 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:11.281822    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67x96" podUID="a16a1fb7-d21c-49cf-a545-faa8ae9d20b6"
Jul 28 17:11:12.409735 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:12.409578    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:11:12.410126 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:12.410072    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v552f" event={"ID":"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652","Type":"ContainerStarted","Data":"3847fa3e8add4f589c4edcde77cc1a106a96efc283519ef3a73aced651152b0d"}
Jul 28 17:11:12.410349 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:12.410260    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:11:12.410529 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:12.410496    2567 scope.go:117] "RemoveContainer" containerID="713acb27e843df283565cd76f18b3c526eaa84a779501ad22cec651b7fb387f5"
Jul 28 17:11:12.412165 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:12.412100    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zxsfx" event={"ID":"f363ee59-3092-46f1-a3e5-a01b034a1e14","Type":"ContainerStarted","Data":"2c6aa90daf6e1f6ed9cd2fc10baca467e6431a3add4980aab78d41b11c4c825a"}
Jul 28 17:11:12.428930 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:12.428909    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:11:13.281702 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:13.281674    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:11:13.281827 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:13.281684    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:11:13.281827 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:13.281766    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fr9xb" podUID="97f130a6-e3b7-4882-b992-0d5eef10897b"
Jul 28 17:11:13.281920 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:13.281856    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcff5" podUID="482929b8-7c61-4fee-bd3b-0f9ac86e61df"
Jul 28 17:11:13.281920 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:13.281683    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:11:13.281983 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:13.281950    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67x96" podUID="a16a1fb7-d21c-49cf-a545-faa8ae9d20b6"
Jul 28 17:11:13.419194 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:13.416973    2567 generic.go:356] "Generic (PLEG): container finished" podID="f363ee59-3092-46f1-a3e5-a01b034a1e14" containerID="2c6aa90daf6e1f6ed9cd2fc10baca467e6431a3add4980aab78d41b11c4c825a" exitCode=0
Jul 28 17:11:13.419194 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:13.417084    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zxsfx" event={"ID":"f363ee59-3092-46f1-a3e5-a01b034a1e14","Type":"ContainerDied","Data":"2c6aa90daf6e1f6ed9cd2fc10baca467e6431a3add4980aab78d41b11c4c825a"}
Jul 28 17:11:13.422650 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:13.422629    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:11:13.423037 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:13.423015    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v552f" event={"ID":"cd62d7f3-7620-47de-a1dd-4e3bc6ea5652","Type":"ContainerStarted","Data":"fd031ed54df56a71aa5e4839ecbe5d024c69cc6bedc6bb670a86d4b51608cc42"}
Jul 28 17:11:13.423437 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:13.423334    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:11:13.423437 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:13.423413    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:11:13.437998 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:13.437977    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:11:13.470687 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:13.470648    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-v552f" podStartSLOduration=9.931140644 podStartE2EDuration="28.470634607s" totalImagesPullingTime="18.539493963s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:13.468750996 +0000 UTC m=+28.714853201" watchObservedRunningTime="2026-07-28 17:11:13.470634607 +0000 UTC m=+28.716736816"
Jul 28 17:11:14.091754 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:14.091693    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fr9xb"]
Jul 28 17:11:14.092079 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:14.092057    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:11:14.092415 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:14.092173    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fr9xb" podUID="97f130a6-e3b7-4882-b992-0d5eef10897b"
Jul 28 17:11:14.092478 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:14.092456    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vcff5"]
Jul 28 17:11:14.092559 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:14.092544    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:11:14.092659 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:14.092640    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcff5" podUID="482929b8-7c61-4fee-bd3b-0f9ac86e61df"
Jul 28 17:11:14.093060 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:14.093043    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-67x96"]
Jul 28 17:11:14.093126 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:14.093105    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:11:14.093189 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:14.093169    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67x96" podUID="a16a1fb7-d21c-49cf-a545-faa8ae9d20b6"
Jul 28 17:11:15.029665 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:15.029631    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-l82pq"
Jul 28 17:11:15.030437 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:15.029761    2567 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 17:11:15.030437 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:15.030363    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-l82pq"
Jul 28 17:11:15.429323 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:15.429287    2567 generic.go:356] "Generic (PLEG): container finished" podID="f363ee59-3092-46f1-a3e5-a01b034a1e14" containerID="d66d5c44fce25ee808cdd7417fc4df063fd01a8b8f4d8319b7d0f343ed42a5e5" exitCode=0
Jul 28 17:11:15.429477 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:15.429365    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zxsfx" event={"ID":"f363ee59-3092-46f1-a3e5-a01b034a1e14","Type":"ContainerDied","Data":"d66d5c44fce25ee808cdd7417fc4df063fd01a8b8f4d8319b7d0f343ed42a5e5"}
Jul 28 17:11:16.281583 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:16.281556    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:11:16.281968 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:16.281587    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:11:16.281968 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:16.281590    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:11:16.281968 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:16.281650    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fr9xb" podUID="97f130a6-e3b7-4882-b992-0d5eef10897b"
Jul 28 17:11:16.281968 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:16.281745    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcff5" podUID="482929b8-7c61-4fee-bd3b-0f9ac86e61df"
Jul 28 17:11:16.281968 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:16.281803    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67x96" podUID="a16a1fb7-d21c-49cf-a545-faa8ae9d20b6"
Jul 28 17:11:17.148565 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:17.148148    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-92rx8_7ad540e6-ff3a-4366-9176-0063eb45135c/dns-node-resolver/0.log"
Jul 28 17:11:17.435685 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:17.435589    2567 generic.go:356] "Generic (PLEG): container finished" podID="f363ee59-3092-46f1-a3e5-a01b034a1e14" containerID="0bfbbbb2c5d2db419da2053f0e39d71eee4bf23ce39944d2040dbecac352ab28" exitCode=0
Jul 28 17:11:17.435685 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:17.435628    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zxsfx" event={"ID":"f363ee59-3092-46f1-a3e5-a01b034a1e14","Type":"ContainerDied","Data":"0bfbbbb2c5d2db419da2053f0e39d71eee4bf23ce39944d2040dbecac352ab28"}
Jul 28 17:11:18.121251 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.121220    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cxrrh_1b94f336-770a-4f5a-8ef9-cf867e720989/node-ca/0.log"
Jul 28 17:11:18.138949 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.138922    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeReady"
Jul 28 17:11:18.139099 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.139086    2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 17:11:18.201370 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.201295    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rrkx5"]
Jul 28 17:11:18.217637 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.217613    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wfpdb"]
Jul 28 17:11:18.218052 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.218022    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rrkx5"
Jul 28 17:11:18.221493 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.221469    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 17:11:18.221835 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.221771    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 17:11:18.221835 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.221791    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-65jvx\""
Jul 28 17:11:18.222337 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.222316    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 17:11:18.238632 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.238610    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rrkx5"]
Jul 28 17:11:18.238728 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.238651    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wfpdb"]
Jul 28 17:11:18.238728 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.238679    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8dpn4"]
Jul 28 17:11:18.238972 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.238816    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wfpdb"
Jul 28 17:11:18.241991 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.241970    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 17:11:18.242088 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.242018    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 17:11:18.242159 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.242139    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kslsq\""
Jul 28 17:11:18.257648 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.257629    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8dpn4"]
Jul 28 17:11:18.257766 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.257741    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:18.261699 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.261679    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 17:11:18.261906 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.261890    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 17:11:18.262160 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.262144    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 17:11:18.262453 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.262433    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rd4np\""
Jul 28 17:11:18.262453 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.262450    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 17:11:18.282787 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.282464    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:11:18.282923 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.282909    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:11:18.282979 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.282937    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:11:18.286774 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.286755    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 17:11:18.286867 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.286829    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 17:11:18.287088 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.287073    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5l79z\""
Jul 28 17:11:18.287161 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.287087    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8c6kz\""
Jul 28 17:11:18.287161 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.287145    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 17:11:18.287408 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.287385    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 17:11:18.352718 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.352694    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6lvh\" (UniqueName: \"kubernetes.io/projected/8c7cadd8-f3b6-4499-96c4-e3a3c2c3f03e-kube-api-access-d6lvh\") pod \"ingress-canary-rrkx5\" (UID: \"8c7cadd8-f3b6-4499-96c4-e3a3c2c3f03e\") " pod="openshift-ingress-canary/ingress-canary-rrkx5"
Jul 28 17:11:18.352828 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.352730    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/63377241-ccfc-4a7a-844d-6bc00d6f7db5-crio-socket\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:18.352828 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.352759    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4vmm\" (UniqueName: \"kubernetes.io/projected/63377241-ccfc-4a7a-844d-6bc00d6f7db5-kube-api-access-l4vmm\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:18.352914 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.352836    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/63377241-ccfc-4a7a-844d-6bc00d6f7db5-data-volume\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:18.352914 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.352874    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14fdd663-eddf-46d8-ab05-2c125ec570fe-tmp-dir\") pod \"dns-default-wfpdb\" (UID: \"14fdd663-eddf-46d8-ab05-2c125ec570fe\") " pod="openshift-dns/dns-default-wfpdb"
Jul 28 17:11:18.352914 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.352903    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgz7x\" (UniqueName: \"kubernetes.io/projected/14fdd663-eddf-46d8-ab05-2c125ec570fe-kube-api-access-lgz7x\") pod \"dns-default-wfpdb\" (UID: \"14fdd663-eddf-46d8-ab05-2c125ec570fe\") " pod="openshift-dns/dns-default-wfpdb"
Jul 28 17:11:18.353019 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.352950    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63377241-ccfc-4a7a-844d-6bc00d6f7db5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:18.353019 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.352984    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/14fdd663-eddf-46d8-ab05-2c125ec570fe-config-volume\") pod \"dns-default-wfpdb\" (UID: \"14fdd663-eddf-46d8-ab05-2c125ec570fe\") " pod="openshift-dns/dns-default-wfpdb"
Jul 28 17:11:18.353019 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.353012    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8c7cadd8-f3b6-4499-96c4-e3a3c2c3f03e-cert\") pod \"ingress-canary-rrkx5\" (UID: \"8c7cadd8-f3b6-4499-96c4-e3a3c2c3f03e\") " pod="openshift-ingress-canary/ingress-canary-rrkx5"
Jul 28 17:11:18.353156 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.353032    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63377241-ccfc-4a7a-844d-6bc00d6f7db5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:18.353156 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.353077    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14fdd663-eddf-46d8-ab05-2c125ec570fe-metrics-tls\") pod \"dns-default-wfpdb\" (UID: \"14fdd663-eddf-46d8-ab05-2c125ec570fe\") " pod="openshift-dns/dns-default-wfpdb"
Jul 28 17:11:18.453542 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.453461    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6lvh\" (UniqueName: \"kubernetes.io/projected/8c7cadd8-f3b6-4499-96c4-e3a3c2c3f03e-kube-api-access-d6lvh\") pod \"ingress-canary-rrkx5\" (UID: \"8c7cadd8-f3b6-4499-96c4-e3a3c2c3f03e\") " pod="openshift-ingress-canary/ingress-canary-rrkx5"
Jul 28 17:11:18.453542 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.453496    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/63377241-ccfc-4a7a-844d-6bc00d6f7db5-crio-socket\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:18.453542 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.453537    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4vmm\" (UniqueName: \"kubernetes.io/projected/63377241-ccfc-4a7a-844d-6bc00d6f7db5-kube-api-access-l4vmm\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:18.454071 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.453581    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/63377241-ccfc-4a7a-844d-6bc00d6f7db5-data-volume\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:18.454071 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.453608    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14fdd663-eddf-46d8-ab05-2c125ec570fe-tmp-dir\") pod \"dns-default-wfpdb\" (UID: \"14fdd663-eddf-46d8-ab05-2c125ec570fe\") " pod="openshift-dns/dns-default-wfpdb"
Jul 28 17:11:18.454071 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.453636    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgz7x\" (UniqueName: \"kubernetes.io/projected/14fdd663-eddf-46d8-ab05-2c125ec570fe-kube-api-access-lgz7x\") pod \"dns-default-wfpdb\" (UID: \"14fdd663-eddf-46d8-ab05-2c125ec570fe\") " pod="openshift-dns/dns-default-wfpdb"
Jul 28 17:11:18.454071 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.453681    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63377241-ccfc-4a7a-844d-6bc00d6f7db5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:18.454071 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.453713    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/14fdd663-eddf-46d8-ab05-2c125ec570fe-config-volume\") pod \"dns-default-wfpdb\" (UID: \"14fdd663-eddf-46d8-ab05-2c125ec570fe\") " pod="openshift-dns/dns-default-wfpdb"
Jul 28 17:11:18.454071 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.453716    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/63377241-ccfc-4a7a-844d-6bc00d6f7db5-crio-socket\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:18.454071 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.453750    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8c7cadd8-f3b6-4499-96c4-e3a3c2c3f03e-cert\") pod \"ingress-canary-rrkx5\" (UID: \"8c7cadd8-f3b6-4499-96c4-e3a3c2c3f03e\") " pod="openshift-ingress-canary/ingress-canary-rrkx5"
Jul 28 17:11:18.454071 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.453774    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63377241-ccfc-4a7a-844d-6bc00d6f7db5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:18.454071 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.453943    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14fdd663-eddf-46d8-ab05-2c125ec570fe-tmp-dir\") pod \"dns-default-wfpdb\" (UID: \"14fdd663-eddf-46d8-ab05-2c125ec570fe\") " pod="openshift-dns/dns-default-wfpdb"
Jul 28 17:11:18.454071 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.453965    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14fdd663-eddf-46d8-ab05-2c125ec570fe-metrics-tls\") pod \"dns-default-wfpdb\" (UID: \"14fdd663-eddf-46d8-ab05-2c125ec570fe\") " pod="openshift-dns/dns-default-wfpdb"
Jul 28 17:11:18.454071 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.453968    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/63377241-ccfc-4a7a-844d-6bc00d6f7db5-data-volume\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:18.454071 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:18.453907    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:11:18.454445 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:18.454081    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:11:18.454445 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:18.454029    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:11:18.454445 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:18.454120    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/63377241-ccfc-4a7a-844d-6bc00d6f7db5-kube-rbac-proxy-cm podName:63377241-ccfc-4a7a-844d-6bc00d6f7db5 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:18.954100132 +0000 UTC m=+34.200202319 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/63377241-ccfc-4a7a-844d-6bc00d6f7db5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8dpn4" (UID: "63377241-ccfc-4a7a-844d-6bc00d6f7db5") : configmap "kube-rbac-proxy" not found
Jul 28 17:11:18.454445 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:18.454141    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14fdd663-eddf-46d8-ab05-2c125ec570fe-metrics-tls podName:14fdd663-eddf-46d8-ab05-2c125ec570fe nodeName:}" failed. No retries permitted until 2026-07-28 17:11:18.954130456 +0000 UTC m=+34.200232643 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/14fdd663-eddf-46d8-ab05-2c125ec570fe-metrics-tls") pod "dns-default-wfpdb" (UID: "14fdd663-eddf-46d8-ab05-2c125ec570fe") : secret "dns-default-metrics-tls" not found
Jul 28 17:11:18.454445 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:18.454157    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63377241-ccfc-4a7a-844d-6bc00d6f7db5-insights-runtime-extractor-tls podName:63377241-ccfc-4a7a-844d-6bc00d6f7db5 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:18.954147602 +0000 UTC m=+34.200249790 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/63377241-ccfc-4a7a-844d-6bc00d6f7db5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8dpn4" (UID: "63377241-ccfc-4a7a-844d-6bc00d6f7db5") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:11:18.454445 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.454333    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/14fdd663-eddf-46d8-ab05-2c125ec570fe-config-volume\") pod \"dns-default-wfpdb\" (UID: \"14fdd663-eddf-46d8-ab05-2c125ec570fe\") " pod="openshift-dns/dns-default-wfpdb"
Jul 28 17:11:18.458765 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.458744    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8c7cadd8-f3b6-4499-96c4-e3a3c2c3f03e-cert\") pod \"ingress-canary-rrkx5\" (UID: \"8c7cadd8-f3b6-4499-96c4-e3a3c2c3f03e\") " pod="openshift-ingress-canary/ingress-canary-rrkx5"
Jul 28 17:11:18.509365 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.509305    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6lvh\" (UniqueName: \"kubernetes.io/projected/8c7cadd8-f3b6-4499-96c4-e3a3c2c3f03e-kube-api-access-d6lvh\") pod \"ingress-canary-rrkx5\" (UID: \"8c7cadd8-f3b6-4499-96c4-e3a3c2c3f03e\") " pod="openshift-ingress-canary/ingress-canary-rrkx5"
Jul 28 17:11:18.509776 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.509744    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4vmm\" (UniqueName: \"kubernetes.io/projected/63377241-ccfc-4a7a-844d-6bc00d6f7db5-kube-api-access-l4vmm\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:18.509855 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.509788    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgz7x\" (UniqueName: \"kubernetes.io/projected/14fdd663-eddf-46d8-ab05-2c125ec570fe-kube-api-access-lgz7x\") pod \"dns-default-wfpdb\" (UID: \"14fdd663-eddf-46d8-ab05-2c125ec570fe\") " pod="openshift-dns/dns-default-wfpdb"
Jul 28 17:11:18.529614 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.529589    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rrkx5"
Jul 28 17:11:18.756138 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.756022    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rrkx5"]
Jul 28 17:11:18.761125 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:11:18.761092    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c7cadd8_f3b6_4499_96c4_e3a3c2c3f03e.slice/crio-05d92e1e926d517ce2c637100a872185c86451ac0c6bfa22fb36b2df0afc94f3 WatchSource:0}: Error finding container 05d92e1e926d517ce2c637100a872185c86451ac0c6bfa22fb36b2df0afc94f3: Status 404 returned error can't find the container with id 05d92e1e926d517ce2c637100a872185c86451ac0c6bfa22fb36b2df0afc94f3
Jul 28 17:11:18.858206 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.858172    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/482929b8-7c61-4fee-bd3b-0f9ac86e61df-original-pull-secret\") pod \"global-pull-secret-syncer-vcff5\" (UID: \"482929b8-7c61-4fee-bd3b-0f9ac86e61df\") " pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:11:18.861615 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.861589    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/482929b8-7c61-4fee-bd3b-0f9ac86e61df-original-pull-secret\") pod \"global-pull-secret-syncer-vcff5\" (UID: \"482929b8-7c61-4fee-bd3b-0f9ac86e61df\") " pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:11:18.894752 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.894724    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcff5"
Jul 28 17:11:18.960903 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.960869    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs\") pod \"network-metrics-daemon-fr9xb\" (UID: \"97f130a6-e3b7-4882-b992-0d5eef10897b\") " pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:11:18.961028 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.960932    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63377241-ccfc-4a7a-844d-6bc00d6f7db5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:18.961028 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.960986    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63377241-ccfc-4a7a-844d-6bc00d6f7db5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:18.961028 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:18.961017    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14fdd663-eddf-46d8-ab05-2c125ec570fe-metrics-tls\") pod \"dns-default-wfpdb\" (UID: \"14fdd663-eddf-46d8-ab05-2c125ec570fe\") " pod="openshift-dns/dns-default-wfpdb"
Jul 28 17:11:18.961179 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:18.961166    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:11:18.961652 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:18.961232    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14fdd663-eddf-46d8-ab05-2c125ec570fe-metrics-tls podName:14fdd663-eddf-46d8-ab05-2c125ec570fe nodeName:}" failed. No retries permitted until 2026-07-28 17:11:19.961212119 +0000 UTC m=+35.207314305 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/14fdd663-eddf-46d8-ab05-2c125ec570fe-metrics-tls") pod "dns-default-wfpdb" (UID: "14fdd663-eddf-46d8-ab05-2c125ec570fe") : secret "dns-default-metrics-tls" not found
Jul 28 17:11:18.961652 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:18.961314    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:11:18.961652 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:18.961384    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/63377241-ccfc-4a7a-844d-6bc00d6f7db5-kube-rbac-proxy-cm podName:63377241-ccfc-4a7a-844d-6bc00d6f7db5 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:19.961367037 +0000 UTC m=+35.207469235 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/63377241-ccfc-4a7a-844d-6bc00d6f7db5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8dpn4" (UID: "63377241-ccfc-4a7a-844d-6bc00d6f7db5") : configmap "kube-rbac-proxy" not found
Jul 28 17:11:18.961652 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:18.961460    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 17:11:18.961652 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:18.961496    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs podName:97f130a6-e3b7-4882-b992-0d5eef10897b nodeName:}" failed. No retries permitted until 2026-07-28 17:11:50.961484257 +0000 UTC m=+66.207586456 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs") pod "network-metrics-daemon-fr9xb" (UID: "97f130a6-e3b7-4882-b992-0d5eef10897b") : secret "metrics-daemon-secret" not found
Jul 28 17:11:18.961652 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:18.961580    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:11:18.961652 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:18.961624    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63377241-ccfc-4a7a-844d-6bc00d6f7db5-insights-runtime-extractor-tls podName:63377241-ccfc-4a7a-844d-6bc00d6f7db5 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:19.961605502 +0000 UTC m=+35.207707691 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/63377241-ccfc-4a7a-844d-6bc00d6f7db5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8dpn4" (UID: "63377241-ccfc-4a7a-844d-6bc00d6f7db5") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:11:19.040108 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:19.040074    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vcff5"]
Jul 28 17:11:19.043362 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:11:19.043336    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod482929b8_7c61_4fee_bd3b_0f9ac86e61df.slice/crio-10965bdb8edd39d675fa9cd7a90c8fca740097f9885e0f7b3c7c122450bb343a WatchSource:0}: Error finding container 10965bdb8edd39d675fa9cd7a90c8fca740097f9885e0f7b3c7c122450bb343a: Status 404 returned error can't find the container with id 10965bdb8edd39d675fa9cd7a90c8fca740097f9885e0f7b3c7c122450bb343a
Jul 28 17:11:19.061991 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:19.061968    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4d9x\" (UniqueName: \"kubernetes.io/projected/a16a1fb7-d21c-49cf-a545-faa8ae9d20b6-kube-api-access-l4d9x\") pod \"network-check-target-67x96\" (UID: \"a16a1fb7-d21c-49cf-a545-faa8ae9d20b6\") " pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:11:19.064861 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:19.064841    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4d9x\" (UniqueName: \"kubernetes.io/projected/a16a1fb7-d21c-49cf-a545-faa8ae9d20b6-kube-api-access-l4d9x\") pod \"network-check-target-67x96\" (UID: \"a16a1fb7-d21c-49cf-a545-faa8ae9d20b6\") " pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:11:19.206411 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:19.206375    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:11:19.365766 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:19.365401    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-67x96"]
Jul 28 17:11:19.368592 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:11:19.368548    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda16a1fb7_d21c_49cf_a545_faa8ae9d20b6.slice/crio-d7cf314033bffb550404400eb2c6557b07cef6fe92b28b0974473ceee49facd5 WatchSource:0}: Error finding container d7cf314033bffb550404400eb2c6557b07cef6fe92b28b0974473ceee49facd5: Status 404 returned error can't find the container with id d7cf314033bffb550404400eb2c6557b07cef6fe92b28b0974473ceee49facd5
Jul 28 17:11:19.441830 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:19.441792    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rrkx5" event={"ID":"8c7cadd8-f3b6-4499-96c4-e3a3c2c3f03e","Type":"ContainerStarted","Data":"05d92e1e926d517ce2c637100a872185c86451ac0c6bfa22fb36b2df0afc94f3"}
Jul 28 17:11:19.443277 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:19.443249    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-67x96" event={"ID":"a16a1fb7-d21c-49cf-a545-faa8ae9d20b6","Type":"ContainerStarted","Data":"d7cf314033bffb550404400eb2c6557b07cef6fe92b28b0974473ceee49facd5"}
Jul 28 17:11:19.444520 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:19.444482    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vcff5" event={"ID":"482929b8-7c61-4fee-bd3b-0f9ac86e61df","Type":"ContainerStarted","Data":"10965bdb8edd39d675fa9cd7a90c8fca740097f9885e0f7b3c7c122450bb343a"}
Jul 28 17:11:19.969736 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:19.969696    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63377241-ccfc-4a7a-844d-6bc00d6f7db5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:19.970290 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:19.969754    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63377241-ccfc-4a7a-844d-6bc00d6f7db5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:19.970290 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:19.969785    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14fdd663-eddf-46d8-ab05-2c125ec570fe-metrics-tls\") pod \"dns-default-wfpdb\" (UID: \"14fdd663-eddf-46d8-ab05-2c125ec570fe\") " pod="openshift-dns/dns-default-wfpdb"
Jul 28 17:11:19.970290 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:19.969855    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:11:19.970290 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:19.969924    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:11:19.970290 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:19.969944    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:11:19.970290 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:19.969930    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/63377241-ccfc-4a7a-844d-6bc00d6f7db5-kube-rbac-proxy-cm podName:63377241-ccfc-4a7a-844d-6bc00d6f7db5 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:21.969909384 +0000 UTC m=+37.216011570 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/63377241-ccfc-4a7a-844d-6bc00d6f7db5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8dpn4" (UID: "63377241-ccfc-4a7a-844d-6bc00d6f7db5") : configmap "kube-rbac-proxy" not found
Jul 28 17:11:19.970290 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:19.970009    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63377241-ccfc-4a7a-844d-6bc00d6f7db5-insights-runtime-extractor-tls podName:63377241-ccfc-4a7a-844d-6bc00d6f7db5 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:21.969994289 +0000 UTC m=+37.216096487 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/63377241-ccfc-4a7a-844d-6bc00d6f7db5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8dpn4" (UID: "63377241-ccfc-4a7a-844d-6bc00d6f7db5") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:11:19.970290 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:19.970024    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14fdd663-eddf-46d8-ab05-2c125ec570fe-metrics-tls podName:14fdd663-eddf-46d8-ab05-2c125ec570fe nodeName:}" failed. No retries permitted until 2026-07-28 17:11:21.970015203 +0000 UTC m=+37.216117400 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/14fdd663-eddf-46d8-ab05-2c125ec570fe-metrics-tls") pod "dns-default-wfpdb" (UID: "14fdd663-eddf-46d8-ab05-2c125ec570fe") : secret "dns-default-metrics-tls" not found
Jul 28 17:11:21.985291 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:21.985257    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63377241-ccfc-4a7a-844d-6bc00d6f7db5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:21.985784 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:21.985311    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63377241-ccfc-4a7a-844d-6bc00d6f7db5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:21.985784 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:21.985345    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14fdd663-eddf-46d8-ab05-2c125ec570fe-metrics-tls\") pod \"dns-default-wfpdb\" (UID: \"14fdd663-eddf-46d8-ab05-2c125ec570fe\") " pod="openshift-dns/dns-default-wfpdb"
Jul 28 17:11:21.985784 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:21.985405    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:11:21.985784 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:21.985497    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/63377241-ccfc-4a7a-844d-6bc00d6f7db5-kube-rbac-proxy-cm podName:63377241-ccfc-4a7a-844d-6bc00d6f7db5 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:25.98547547 +0000 UTC m=+41.231577673 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/63377241-ccfc-4a7a-844d-6bc00d6f7db5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8dpn4" (UID: "63377241-ccfc-4a7a-844d-6bc00d6f7db5") : configmap "kube-rbac-proxy" not found
Jul 28 17:11:21.985784 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:21.985537    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:11:21.985784 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:21.985550    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:11:21.985784 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:21.985601    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63377241-ccfc-4a7a-844d-6bc00d6f7db5-insights-runtime-extractor-tls podName:63377241-ccfc-4a7a-844d-6bc00d6f7db5 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:25.985581875 +0000 UTC m=+41.231684074 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/63377241-ccfc-4a7a-844d-6bc00d6f7db5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8dpn4" (UID: "63377241-ccfc-4a7a-844d-6bc00d6f7db5") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:11:21.985784 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:21.985621    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14fdd663-eddf-46d8-ab05-2c125ec570fe-metrics-tls podName:14fdd663-eddf-46d8-ab05-2c125ec570fe nodeName:}" failed. No retries permitted until 2026-07-28 17:11:25.985611275 +0000 UTC m=+41.231713477 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/14fdd663-eddf-46d8-ab05-2c125ec570fe-metrics-tls") pod "dns-default-wfpdb" (UID: "14fdd663-eddf-46d8-ab05-2c125ec570fe") : secret "dns-default-metrics-tls" not found
Jul 28 17:11:26.018536 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:26.018487    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63377241-ccfc-4a7a-844d-6bc00d6f7db5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:26.019098 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:26.018551    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63377241-ccfc-4a7a-844d-6bc00d6f7db5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:26.019098 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:26.018573    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14fdd663-eddf-46d8-ab05-2c125ec570fe-metrics-tls\") pod \"dns-default-wfpdb\" (UID: \"14fdd663-eddf-46d8-ab05-2c125ec570fe\") " pod="openshift-dns/dns-default-wfpdb"
Jul 28 17:11:26.019098 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:26.018590    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:11:26.019098 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:26.018664    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/63377241-ccfc-4a7a-844d-6bc00d6f7db5-kube-rbac-proxy-cm podName:63377241-ccfc-4a7a-844d-6bc00d6f7db5 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:34.01864435 +0000 UTC m=+49.264746542 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/63377241-ccfc-4a7a-844d-6bc00d6f7db5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8dpn4" (UID: "63377241-ccfc-4a7a-844d-6bc00d6f7db5") : configmap "kube-rbac-proxy" not found
Jul 28 17:11:26.019098 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:26.018704    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:11:26.019098 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:26.018764    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63377241-ccfc-4a7a-844d-6bc00d6f7db5-insights-runtime-extractor-tls podName:63377241-ccfc-4a7a-844d-6bc00d6f7db5 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:34.018750623 +0000 UTC m=+49.264852822 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/63377241-ccfc-4a7a-844d-6bc00d6f7db5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8dpn4" (UID: "63377241-ccfc-4a7a-844d-6bc00d6f7db5") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:11:26.021309 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:26.021210    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14fdd663-eddf-46d8-ab05-2c125ec570fe-metrics-tls\") pod \"dns-default-wfpdb\" (UID: \"14fdd663-eddf-46d8-ab05-2c125ec570fe\") " pod="openshift-dns/dns-default-wfpdb"
Jul 28 17:11:26.048326 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:26.048266    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wfpdb"
Jul 28 17:11:26.367698 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:26.367677    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wfpdb"]
Jul 28 17:11:26.370185 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:11:26.370159    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14fdd663_eddf_46d8_ab05_2c125ec570fe.slice/crio-d2410926d4355c10ee34c2f01a8e0805ba37a39bbbe2eb99b94276a1cf3ea2c0 WatchSource:0}: Error finding container d2410926d4355c10ee34c2f01a8e0805ba37a39bbbe2eb99b94276a1cf3ea2c0: Status 404 returned error can't find the container with id d2410926d4355c10ee34c2f01a8e0805ba37a39bbbe2eb99b94276a1cf3ea2c0
Jul 28 17:11:26.463732 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:26.463708    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-67x96" event={"ID":"a16a1fb7-d21c-49cf-a545-faa8ae9d20b6","Type":"ContainerStarted","Data":"e7ed4720815e8477378d50648e7ffba07f9947dc45e3734a6f00c2782669112b"}
Jul 28 17:11:26.463867 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:26.463843    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:11:26.464943 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:26.464913    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wfpdb" event={"ID":"14fdd663-eddf-46d8-ab05-2c125ec570fe","Type":"ContainerStarted","Data":"d2410926d4355c10ee34c2f01a8e0805ba37a39bbbe2eb99b94276a1cf3ea2c0"}
Jul 28 17:11:26.467443 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:26.467424    2567 generic.go:356] "Generic (PLEG): container finished" podID="f363ee59-3092-46f1-a3e5-a01b034a1e14" containerID="1182d96b012d5a5ace0039a685ce94b7eb5770fa0746a182780a87c760243e31" exitCode=0
Jul 28 17:11:26.467536 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:26.467481    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zxsfx" event={"ID":"f363ee59-3092-46f1-a3e5-a01b034a1e14","Type":"ContainerDied","Data":"1182d96b012d5a5ace0039a685ce94b7eb5770fa0746a182780a87c760243e31"}
Jul 28 17:11:26.468953 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:26.468929    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rrkx5" event={"ID":"8c7cadd8-f3b6-4499-96c4-e3a3c2c3f03e","Type":"ContainerStarted","Data":"680198ee9fca6595bcab08f24a8fec9ccdf7d0a5f9dd6ef7379f25f0316611bb"}
Jul 28 17:11:26.470338 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:26.470318    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vcff5" event={"ID":"482929b8-7c61-4fee-bd3b-0f9ac86e61df","Type":"ContainerStarted","Data":"4c76be433507b548408e19156a6b12f023e67699da1ce30cea4c7d075a35797e"}
Jul 28 17:11:26.498277 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:26.498235    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-67x96" podStartSLOduration=34.611153743 podStartE2EDuration="41.498223186s" totalImagesPullingTime="6.887069443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:10:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:26.480752255 +0000 UTC m=+41.726854462" watchObservedRunningTime="2026-07-28 17:11:26.498223186 +0000 UTC m=+41.744325444"
Jul 28 17:11:26.514773 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:26.514606    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vcff5" podStartSLOduration=34.678361845 podStartE2EDuration="41.514592s" totalImagesPullingTime="6.836230155s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:26.514501177 +0000 UTC m=+41.760603382" watchObservedRunningTime="2026-07-28 17:11:26.514592 +0000 UTC m=+41.760694201"
Jul 28 17:11:26.514890 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:26.514867    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rrkx5" podStartSLOduration=1.3965325960000001 podStartE2EDuration="8.514860948s" totalImagesPullingTime="7.118328352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:26.497946939 +0000 UTC m=+41.744049145" watchObservedRunningTime="2026-07-28 17:11:26.514860948 +0000 UTC m=+41.760963153"
Jul 28 17:11:27.476300 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:27.476260    2567 generic.go:356] "Generic (PLEG): container finished" podID="f363ee59-3092-46f1-a3e5-a01b034a1e14" containerID="4c1128b49e10d8a02ec214540710317087a127c58b59139eea104f8174c244d3" exitCode=0
Jul 28 17:11:27.476756 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:27.476374    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zxsfx" event={"ID":"f363ee59-3092-46f1-a3e5-a01b034a1e14","Type":"ContainerDied","Data":"4c1128b49e10d8a02ec214540710317087a127c58b59139eea104f8174c244d3"}
Jul 28 17:11:28.485124 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:28.485096    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zxsfx" event={"ID":"f363ee59-3092-46f1-a3e5-a01b034a1e14","Type":"ContainerStarted","Data":"c1218b9886c0f68465e54cc9d105c1cc574003b31ef581ebe864ff58c1060745"}
Jul 28 17:11:28.509876 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:28.509837    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zxsfx" podStartSLOduration=9.432348335 podStartE2EDuration="43.509822912s" totalImagesPullingTime="34.077474577s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:28.509615841 +0000 UTC m=+43.755718047" watchObservedRunningTime="2026-07-28 17:11:28.509822912 +0000 UTC m=+43.755925117"
Jul 28 17:11:29.490116 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:29.490078    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wfpdb" event={"ID":"14fdd663-eddf-46d8-ab05-2c125ec570fe","Type":"ContainerStarted","Data":"d010b633f80819ef56a3177b19c1552a0b177916ef69c9b082b98ccd973a28d2"}
Jul 28 17:11:29.490531 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:29.490136    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wfpdb" event={"ID":"14fdd663-eddf-46d8-ab05-2c125ec570fe","Type":"ContainerStarted","Data":"30b48b8d0bb830a5a6f6deca3d55a274fbfb1aeb68d1dc62ae31929460201329"}
Jul 28 17:11:29.490531 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:29.490212    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wfpdb"
Jul 28 17:11:34.070562 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:34.070494    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63377241-ccfc-4a7a-844d-6bc00d6f7db5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:34.070562 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:34.070566    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63377241-ccfc-4a7a-844d-6bc00d6f7db5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:34.071076 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:34.071013    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63377241-ccfc-4a7a-844d-6bc00d6f7db5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:34.072845 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:34.072823    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63377241-ccfc-4a7a-844d-6bc00d6f7db5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8dpn4\" (UID: \"63377241-ccfc-4a7a-844d-6bc00d6f7db5\") " pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:34.167043 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:34.167010    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8dpn4"
Jul 28 17:11:34.283347 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:34.283294    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wfpdb" podStartSLOduration=14.233019373 podStartE2EDuration="16.283278587s" totalImagesPullingTime="2.050259214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:29.509641712 +0000 UTC m=+44.755743930" watchObservedRunningTime="2026-07-28 17:11:34.283278587 +0000 UTC m=+49.529380793"
Jul 28 17:11:34.283713 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:34.283696    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8dpn4"]
Jul 28 17:11:34.286781 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:11:34.286755    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63377241_ccfc_4a7a_844d_6bc00d6f7db5.slice/crio-26c9352ae797fc0f0cfeaeeab9cd288746abb76217bca49cc54b0c6e0085c9de WatchSource:0}: Error finding container 26c9352ae797fc0f0cfeaeeab9cd288746abb76217bca49cc54b0c6e0085c9de: Status 404 returned error can't find the container with id 26c9352ae797fc0f0cfeaeeab9cd288746abb76217bca49cc54b0c6e0085c9de
Jul 28 17:11:34.506871 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:34.506780    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8dpn4" event={"ID":"63377241-ccfc-4a7a-844d-6bc00d6f7db5","Type":"ContainerStarted","Data":"26c9352ae797fc0f0cfeaeeab9cd288746abb76217bca49cc54b0c6e0085c9de"}
Jul 28 17:11:39.495375 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:39.495343    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wfpdb"
Jul 28 17:11:39.527323 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:39.527269    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8dpn4" event={"ID":"63377241-ccfc-4a7a-844d-6bc00d6f7db5","Type":"ContainerStarted","Data":"acbd261621827a808ab89d9200c23fe2fe5293bd0d6827783c2cae5b979615c9"}
Jul 28 17:11:41.898925 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:41.898738    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"]
Jul 28 17:11:41.906663 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:41.906643    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"
Jul 28 17:11:41.911414 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:41.911387    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 28 17:11:41.911606 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:41.911569    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 28 17:11:41.911914 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:41.911895    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 28 17:11:41.912020 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:41.911923    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 28 17:11:41.913116 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:41.912887    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 28 17:11:41.913116 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:41.912902    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 28 17:11:41.913116 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:41.912894    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 28 17:11:41.919398 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:41.919380    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-597ff4f69f-krcr5"]
Jul 28 17:11:41.921413 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:41.921390    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597ff4f69f-krcr5"
Jul 28 17:11:41.932240 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:41.932221    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 28 17:11:41.940203 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:41.940184    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"]
Jul 28 17:11:41.947311 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:41.947294    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-597ff4f69f-krcr5"]
Jul 28 17:11:42.025484 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.025407    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cb5cf6548-7wmk4"]
Jul 28 17:11:42.028984 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.028957    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:42.029986 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.029943    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e5c9a90-3a76-4406-bb8a-44c66c60dd7b-tmp\") pod \"klusterlet-addon-workmgr-597ff4f69f-krcr5\" (UID: \"1e5c9a90-3a76-4406-bb8a-44c66c60dd7b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597ff4f69f-krcr5"
Jul 28 17:11:42.029986 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.029979    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv4rv\" (UniqueName: \"kubernetes.io/projected/1e5c9a90-3a76-4406-bb8a-44c66c60dd7b-kube-api-access-fv4rv\") pod \"klusterlet-addon-workmgr-597ff4f69f-krcr5\" (UID: \"1e5c9a90-3a76-4406-bb8a-44c66c60dd7b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597ff4f69f-krcr5"
Jul 28 17:11:42.030159 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.030013    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e42d5171-57cf-4041-8d94-722a49c92c9b-ca\") pod \"cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb\" (UID: \"e42d5171-57cf-4041-8d94-722a49c92c9b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"
Jul 28 17:11:42.030159 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.030041    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26v2s\" (UniqueName: \"kubernetes.io/projected/e42d5171-57cf-4041-8d94-722a49c92c9b-kube-api-access-26v2s\") pod \"cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb\" (UID: \"e42d5171-57cf-4041-8d94-722a49c92c9b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"
Jul 28 17:11:42.030159 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.030074    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e42d5171-57cf-4041-8d94-722a49c92c9b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb\" (UID: \"e42d5171-57cf-4041-8d94-722a49c92c9b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"
Jul 28 17:11:42.030159 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.030126    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e42d5171-57cf-4041-8d94-722a49c92c9b-hub\") pod \"cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb\" (UID: \"e42d5171-57cf-4041-8d94-722a49c92c9b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"
Jul 28 17:11:42.030326 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.030162    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e42d5171-57cf-4041-8d94-722a49c92c9b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb\" (UID: \"e42d5171-57cf-4041-8d94-722a49c92c9b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"
Jul 28 17:11:42.030326 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.030189    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e42d5171-57cf-4041-8d94-722a49c92c9b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb\" (UID: \"e42d5171-57cf-4041-8d94-722a49c92c9b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"
Jul 28 17:11:42.030326 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.030227    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1e5c9a90-3a76-4406-bb8a-44c66c60dd7b-klusterlet-config\") pod \"klusterlet-addon-workmgr-597ff4f69f-krcr5\" (UID: \"1e5c9a90-3a76-4406-bb8a-44c66c60dd7b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597ff4f69f-krcr5"
Jul 28 17:11:42.056039 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.056013    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 17:11:42.056349 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.056325    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-l68jz\""
Jul 28 17:11:42.056349 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.056340    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 17:11:42.056493 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.056382    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 17:11:42.056493 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.056325    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 17:11:42.056493 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.056326    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 17:11:42.056493 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.056325    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 17:11:42.057065 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.057046    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 17:11:42.066359 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.066336    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cb5cf6548-7wmk4"]
Jul 28 17:11:42.131264 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.131242    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e5c9a90-3a76-4406-bb8a-44c66c60dd7b-tmp\") pod \"klusterlet-addon-workmgr-597ff4f69f-krcr5\" (UID: \"1e5c9a90-3a76-4406-bb8a-44c66c60dd7b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597ff4f69f-krcr5"
Jul 28 17:11:42.131373 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.131269    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv4rv\" (UniqueName: \"kubernetes.io/projected/1e5c9a90-3a76-4406-bb8a-44c66c60dd7b-kube-api-access-fv4rv\") pod \"klusterlet-addon-workmgr-597ff4f69f-krcr5\" (UID: \"1e5c9a90-3a76-4406-bb8a-44c66c60dd7b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597ff4f69f-krcr5"
Jul 28 17:11:42.131373 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.131287    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e42d5171-57cf-4041-8d94-722a49c92c9b-ca\") pod \"cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb\" (UID: \"e42d5171-57cf-4041-8d94-722a49c92c9b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"
Jul 28 17:11:42.131373 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.131311    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26v2s\" (UniqueName: \"kubernetes.io/projected/e42d5171-57cf-4041-8d94-722a49c92c9b-kube-api-access-26v2s\") pod \"cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb\" (UID: \"e42d5171-57cf-4041-8d94-722a49c92c9b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"
Jul 28 17:11:42.131373 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.131337    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e42d5171-57cf-4041-8d94-722a49c92c9b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb\" (UID: \"e42d5171-57cf-4041-8d94-722a49c92c9b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"
Jul 28 17:11:42.131606 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.131388    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e524bd8e-daf3-4b21-9c83-6f4e11a66164-console-serving-cert\") pod \"console-5cb5cf6548-7wmk4\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") " pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:42.131606 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.131412    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e524bd8e-daf3-4b21-9c83-6f4e11a66164-oauth-serving-cert\") pod \"console-5cb5cf6548-7wmk4\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") " pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:42.131606 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.131439    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e42d5171-57cf-4041-8d94-722a49c92c9b-hub\") pod \"cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb\" (UID: \"e42d5171-57cf-4041-8d94-722a49c92c9b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"
Jul 28 17:11:42.131606 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.131498    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e42d5171-57cf-4041-8d94-722a49c92c9b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb\" (UID: \"e42d5171-57cf-4041-8d94-722a49c92c9b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"
Jul 28 17:11:42.131606 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.131548    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e42d5171-57cf-4041-8d94-722a49c92c9b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb\" (UID: \"e42d5171-57cf-4041-8d94-722a49c92c9b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"
Jul 28 17:11:42.131606 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.131579    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e524bd8e-daf3-4b21-9c83-6f4e11a66164-service-ca\") pod \"console-5cb5cf6548-7wmk4\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") " pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:42.131900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.131623    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1e5c9a90-3a76-4406-bb8a-44c66c60dd7b-klusterlet-config\") pod \"klusterlet-addon-workmgr-597ff4f69f-krcr5\" (UID: \"1e5c9a90-3a76-4406-bb8a-44c66c60dd7b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597ff4f69f-krcr5"
Jul 28 17:11:42.131900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.131647    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e524bd8e-daf3-4b21-9c83-6f4e11a66164-console-oauth-config\") pod \"console-5cb5cf6548-7wmk4\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") " pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:42.131900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.131690    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e5c9a90-3a76-4406-bb8a-44c66c60dd7b-tmp\") pod \"klusterlet-addon-workmgr-597ff4f69f-krcr5\" (UID: \"1e5c9a90-3a76-4406-bb8a-44c66c60dd7b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597ff4f69f-krcr5"
Jul 28 17:11:42.131900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.131701    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e524bd8e-daf3-4b21-9c83-6f4e11a66164-console-config\") pod \"console-5cb5cf6548-7wmk4\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") " pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:42.131900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.131725    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-497pk\" (UniqueName: \"kubernetes.io/projected/e524bd8e-daf3-4b21-9c83-6f4e11a66164-kube-api-access-497pk\") pod \"console-5cb5cf6548-7wmk4\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") " pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:42.132192 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.132152    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e42d5171-57cf-4041-8d94-722a49c92c9b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb\" (UID: \"e42d5171-57cf-4041-8d94-722a49c92c9b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"
Jul 28 17:11:42.133844 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.133823    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e42d5171-57cf-4041-8d94-722a49c92c9b-ca\") pod \"cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb\" (UID: \"e42d5171-57cf-4041-8d94-722a49c92c9b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"
Jul 28 17:11:42.133948 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.133906    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e42d5171-57cf-4041-8d94-722a49c92c9b-hub\") pod \"cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb\" (UID: \"e42d5171-57cf-4041-8d94-722a49c92c9b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"
Jul 28 17:11:42.134251 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.134235    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e42d5171-57cf-4041-8d94-722a49c92c9b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb\" (UID: \"e42d5171-57cf-4041-8d94-722a49c92c9b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"
Jul 28 17:11:42.134292 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.134259    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1e5c9a90-3a76-4406-bb8a-44c66c60dd7b-klusterlet-config\") pod \"klusterlet-addon-workmgr-597ff4f69f-krcr5\" (UID: \"1e5c9a90-3a76-4406-bb8a-44c66c60dd7b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597ff4f69f-krcr5"
Jul 28 17:11:42.134332 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.134263    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e42d5171-57cf-4041-8d94-722a49c92c9b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb\" (UID: \"e42d5171-57cf-4041-8d94-722a49c92c9b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"
Jul 28 17:11:42.179262 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.179208    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26v2s\" (UniqueName: \"kubernetes.io/projected/e42d5171-57cf-4041-8d94-722a49c92c9b-kube-api-access-26v2s\") pod \"cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb\" (UID: \"e42d5171-57cf-4041-8d94-722a49c92c9b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"
Jul 28 17:11:42.180030 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.180003    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv4rv\" (UniqueName: \"kubernetes.io/projected/1e5c9a90-3a76-4406-bb8a-44c66c60dd7b-kube-api-access-fv4rv\") pod \"klusterlet-addon-workmgr-597ff4f69f-krcr5\" (UID: \"1e5c9a90-3a76-4406-bb8a-44c66c60dd7b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597ff4f69f-krcr5"
Jul 28 17:11:42.225373 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.225350    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"
Jul 28 17:11:42.232296 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.232275    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e524bd8e-daf3-4b21-9c83-6f4e11a66164-console-config\") pod \"console-5cb5cf6548-7wmk4\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") " pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:42.232388 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.232336    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-497pk\" (UniqueName: \"kubernetes.io/projected/e524bd8e-daf3-4b21-9c83-6f4e11a66164-kube-api-access-497pk\") pod \"console-5cb5cf6548-7wmk4\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") " pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:42.232388 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.232349    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597ff4f69f-krcr5"
Jul 28 17:11:42.232492 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.232395    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e524bd8e-daf3-4b21-9c83-6f4e11a66164-console-serving-cert\") pod \"console-5cb5cf6548-7wmk4\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") " pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:42.232492 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.232418    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e524bd8e-daf3-4b21-9c83-6f4e11a66164-oauth-serving-cert\") pod \"console-5cb5cf6548-7wmk4\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") " pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:42.232492 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.232449    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e524bd8e-daf3-4b21-9c83-6f4e11a66164-service-ca\") pod \"console-5cb5cf6548-7wmk4\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") " pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:42.232492 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.232480    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e524bd8e-daf3-4b21-9c83-6f4e11a66164-console-oauth-config\") pod \"console-5cb5cf6548-7wmk4\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") " pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:42.233088 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.233063    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e524bd8e-daf3-4b21-9c83-6f4e11a66164-console-config\") pod \"console-5cb5cf6548-7wmk4\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") " pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:42.233274 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.233250    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e524bd8e-daf3-4b21-9c83-6f4e11a66164-oauth-serving-cert\") pod \"console-5cb5cf6548-7wmk4\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") " pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:42.233334 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.233255    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e524bd8e-daf3-4b21-9c83-6f4e11a66164-service-ca\") pod \"console-5cb5cf6548-7wmk4\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") " pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:42.234707 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.234686    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e524bd8e-daf3-4b21-9c83-6f4e11a66164-console-oauth-config\") pod \"console-5cb5cf6548-7wmk4\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") " pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:42.234984 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.234965    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e524bd8e-daf3-4b21-9c83-6f4e11a66164-console-serving-cert\") pod \"console-5cb5cf6548-7wmk4\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") " pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:42.271695 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.271669    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-497pk\" (UniqueName: \"kubernetes.io/projected/e524bd8e-daf3-4b21-9c83-6f4e11a66164-kube-api-access-497pk\") pod \"console-5cb5cf6548-7wmk4\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") " pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:42.344074 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.344020    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:42.373997 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.373971    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-597ff4f69f-krcr5"]
Jul 28 17:11:42.376298 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:11:42.376266    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e5c9a90_3a76_4406_bb8a_44c66c60dd7b.slice/crio-e7dc507779eae05721fb0dc1a314950199a213cc79783b0156d10e5c9d618c50 WatchSource:0}: Error finding container e7dc507779eae05721fb0dc1a314950199a213cc79783b0156d10e5c9d618c50: Status 404 returned error can't find the container with id e7dc507779eae05721fb0dc1a314950199a213cc79783b0156d10e5c9d618c50
Jul 28 17:11:42.401266 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.401239    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb"]
Jul 28 17:11:42.404908 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:11:42.404885    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode42d5171_57cf_4041_8d94_722a49c92c9b.slice/crio-3e7e93e6d5f4f9bea477fb373e4b6e2cc0f75fe47806cb485a1abc7abff1fca1 WatchSource:0}: Error finding container 3e7e93e6d5f4f9bea477fb373e4b6e2cc0f75fe47806cb485a1abc7abff1fca1: Status 404 returned error can't find the container with id 3e7e93e6d5f4f9bea477fb373e4b6e2cc0f75fe47806cb485a1abc7abff1fca1
Jul 28 17:11:42.471917 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.471829    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cb5cf6548-7wmk4"]
Jul 28 17:11:42.474100 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:11:42.474072    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode524bd8e_daf3_4b21_9c83_6f4e11a66164.slice/crio-f1fb2eb84c09c652faabec10d9b002c96d75c28d214793bb29ea5afcaedd86e3 WatchSource:0}: Error finding container f1fb2eb84c09c652faabec10d9b002c96d75c28d214793bb29ea5afcaedd86e3: Status 404 returned error can't find the container with id f1fb2eb84c09c652faabec10d9b002c96d75c28d214793bb29ea5afcaedd86e3
Jul 28 17:11:42.537900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.537868    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597ff4f69f-krcr5" event={"ID":"1e5c9a90-3a76-4406-bb8a-44c66c60dd7b","Type":"ContainerStarted","Data":"e7dc507779eae05721fb0dc1a314950199a213cc79783b0156d10e5c9d618c50"}
Jul 28 17:11:42.538853 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.538831    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb" event={"ID":"e42d5171-57cf-4041-8d94-722a49c92c9b","Type":"ContainerStarted","Data":"3e7e93e6d5f4f9bea477fb373e4b6e2cc0f75fe47806cb485a1abc7abff1fca1"}
Jul 28 17:11:42.540457 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.540436    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8dpn4" event={"ID":"63377241-ccfc-4a7a-844d-6bc00d6f7db5","Type":"ContainerStarted","Data":"9e6423f4d75f581728dade5dc15dcb8592fb958851f134bd74ba7a6314cba8a7"}
Jul 28 17:11:42.541562 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:42.541539    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb5cf6548-7wmk4" event={"ID":"e524bd8e-daf3-4b21-9c83-6f4e11a66164","Type":"ContainerStarted","Data":"f1fb2eb84c09c652faabec10d9b002c96d75c28d214793bb29ea5afcaedd86e3"}
Jul 28 17:11:45.447434 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:45.447398    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v552f"
Jul 28 17:11:50.157758 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.157675    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-798d7f87ff-868gf"]
Jul 28 17:11:50.163248 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.163222    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.172894 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.172861    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 17:11:50.173717 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.173671    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-798d7f87ff-868gf"]
Jul 28 17:11:50.297098 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.297066    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42fc4f58-4740-4391-a887-bf3594fc8bc8-console-oauth-config\") pod \"console-798d7f87ff-868gf\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") " pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.297098 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.297100    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42fc4f58-4740-4391-a887-bf3594fc8bc8-console-serving-cert\") pod \"console-798d7f87ff-868gf\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") " pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.297305 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.297130    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-trusted-ca-bundle\") pod \"console-798d7f87ff-868gf\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") " pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.297305 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.297159    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-console-config\") pod \"console-798d7f87ff-868gf\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") " pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.297305 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.297253    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-oauth-serving-cert\") pod \"console-798d7f87ff-868gf\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") " pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.297305 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.297292    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44z88\" (UniqueName: \"kubernetes.io/projected/42fc4f58-4740-4391-a887-bf3594fc8bc8-kube-api-access-44z88\") pod \"console-798d7f87ff-868gf\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") " pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.297494 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.297327    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-service-ca\") pod \"console-798d7f87ff-868gf\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") " pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.398312 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.398269    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44z88\" (UniqueName: \"kubernetes.io/projected/42fc4f58-4740-4391-a887-bf3594fc8bc8-kube-api-access-44z88\") pod \"console-798d7f87ff-868gf\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") " pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.398480 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.398319    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-service-ca\") pod \"console-798d7f87ff-868gf\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") " pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.398480 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.398379    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42fc4f58-4740-4391-a887-bf3594fc8bc8-console-oauth-config\") pod \"console-798d7f87ff-868gf\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") " pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.398480 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.398408    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42fc4f58-4740-4391-a887-bf3594fc8bc8-console-serving-cert\") pod \"console-798d7f87ff-868gf\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") " pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.398480 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.398449    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-trusted-ca-bundle\") pod \"console-798d7f87ff-868gf\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") " pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.398480 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.398473    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-console-config\") pod \"console-798d7f87ff-868gf\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") " pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.398754 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.398496    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-oauth-serving-cert\") pod \"console-798d7f87ff-868gf\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") " pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.399186 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.399129    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-service-ca\") pod \"console-798d7f87ff-868gf\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") " pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.399283 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.399227    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-oauth-serving-cert\") pod \"console-798d7f87ff-868gf\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") " pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.399378 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.399286    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-console-config\") pod \"console-798d7f87ff-868gf\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") " pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.399378 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.399312    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-trusted-ca-bundle\") pod \"console-798d7f87ff-868gf\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") " pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.401094 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.401069    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42fc4f58-4740-4391-a887-bf3594fc8bc8-console-serving-cert\") pod \"console-798d7f87ff-868gf\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") " pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.401262 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.401246    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42fc4f58-4740-4391-a887-bf3594fc8bc8-console-oauth-config\") pod \"console-798d7f87ff-868gf\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") " pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.407228 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.407186    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44z88\" (UniqueName: \"kubernetes.io/projected/42fc4f58-4740-4391-a887-bf3594fc8bc8-kube-api-access-44z88\") pod \"console-798d7f87ff-868gf\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") " pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.474069 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.473795    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:11:50.577215 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.577182    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb5cf6548-7wmk4" event={"ID":"e524bd8e-daf3-4b21-9c83-6f4e11a66164","Type":"ContainerStarted","Data":"c919511ad7795ae870b9b116428f539e1a3be98ec4cafe7ae2683ce6e7939d92"}
Jul 28 17:11:50.578965 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.578934    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597ff4f69f-krcr5" event={"ID":"1e5c9a90-3a76-4406-bb8a-44c66c60dd7b","Type":"ContainerStarted","Data":"a74da73b700537c4560abed76b2848871bb1e0ff5c7adae8273578aa1b416c08"}
Jul 28 17:11:50.579166 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.579088    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597ff4f69f-krcr5"
Jul 28 17:11:50.580620 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.580596    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597ff4f69f-krcr5"
Jul 28 17:11:50.581114 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.581091    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8dpn4" event={"ID":"63377241-ccfc-4a7a-844d-6bc00d6f7db5","Type":"ContainerStarted","Data":"46a272e8c6db8ff0c3a2c91e76de86a0f1f461dfcb37c75afd5adb5e93b6697c"}
Jul 28 17:11:50.602405 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.602359    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5cb5cf6548-7wmk4" podStartSLOduration=2.326573362 podStartE2EDuration="9.602344202s" totalImagesPullingTime="7.27577084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:50.600451726 +0000 UTC m=+65.846553931" watchObservedRunningTime="2026-07-28 17:11:50.602344202 +0000 UTC m=+65.848446410"
Jul 28 17:11:50.622106 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:50.622068    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597ff4f69f-krcr5" podStartSLOduration=2.228342559 podStartE2EDuration="9.622056221s" totalImagesPullingTime="7.393713662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:50.621021235 +0000 UTC m=+65.867123441" watchObservedRunningTime="2026-07-28 17:11:50.622056221 +0000 UTC m=+65.868158426"
Jul 28 17:11:51.004272 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:51.004234    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs\") pod \"network-metrics-daemon-fr9xb\" (UID: \"97f130a6-e3b7-4882-b992-0d5eef10897b\") " pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:11:51.006653 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:51.006632    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f130a6-e3b7-4882-b992-0d5eef10897b-metrics-certs\") pod \"network-metrics-daemon-fr9xb\" (UID: \"97f130a6-e3b7-4882-b992-0d5eef10897b\") " pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:11:51.304310 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:51.304287    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8c6kz\""
Jul 28 17:11:51.311906 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:51.311885    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fr9xb"
Jul 28 17:11:51.327330 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:51.327216    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8dpn4" podStartSLOduration=17.990473935 podStartE2EDuration="33.327197327s" totalImagesPullingTime="15.336723392s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:11:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:50.642869272 +0000 UTC m=+65.888971478" watchObservedRunningTime="2026-07-28 17:11:51.327197327 +0000 UTC m=+66.573299533"
Jul 28 17:11:51.327739 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:51.327687    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-798d7f87ff-868gf"]
Jul 28 17:11:51.399928 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:11:51.399903    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42fc4f58_4740_4391_a887_bf3594fc8bc8.slice/crio-b5d8c92cd55239bff2a3df5834f740304d024cc252f4b44d9160fb7f94349635 WatchSource:0}: Error finding container b5d8c92cd55239bff2a3df5834f740304d024cc252f4b44d9160fb7f94349635: Status 404 returned error can't find the container with id b5d8c92cd55239bff2a3df5834f740304d024cc252f4b44d9160fb7f94349635
Jul 28 17:11:51.558765 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:51.558737    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fr9xb"]
Jul 28 17:11:51.585810 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:51.585780    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb" event={"ID":"e42d5171-57cf-4041-8d94-722a49c92c9b","Type":"ContainerStarted","Data":"8fdc8036b4032774ef38313133f14f663250c7dbdab4554fe201ab83f5eb00d2"}
Jul 28 17:11:51.587190 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:51.587166    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-798d7f87ff-868gf" event={"ID":"42fc4f58-4740-4391-a887-bf3594fc8bc8","Type":"ContainerStarted","Data":"b45a7acd83532adc8b01ce0399b59e459f70adda6f88557f16a316a74bee6a55"}
Jul 28 17:11:51.587272 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:51.587209    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-798d7f87ff-868gf" event={"ID":"42fc4f58-4740-4391-a887-bf3594fc8bc8","Type":"ContainerStarted","Data":"b5d8c92cd55239bff2a3df5834f740304d024cc252f4b44d9160fb7f94349635"}
Jul 28 17:11:51.588259 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:51.588236    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fr9xb" event={"ID":"97f130a6-e3b7-4882-b992-0d5eef10897b","Type":"ContainerStarted","Data":"cc319f5ba3d48065306386b04269f74e4c7fc7560febaadb40fe659f65a145b2"}
Jul 28 17:11:51.610750 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:51.610702    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-798d7f87ff-868gf" podStartSLOduration=1.6106844040000001 podStartE2EDuration="1.610684404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:51.608644675 +0000 UTC m=+66.854746874" watchObservedRunningTime="2026-07-28 17:11:51.610684404 +0000 UTC m=+66.856786614"
Jul 28 17:11:52.344950 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:52.344916    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:52.345306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:52.344995    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:52.349541 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:52.349521    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:52.593433 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:52.593389    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb" event={"ID":"e42d5171-57cf-4041-8d94-722a49c92c9b","Type":"ContainerStarted","Data":"8b865afe08ce08b6629fb73c8186493e2c0c3e83901bdd79df311ec5f1e7f8b8"}
Jul 28 17:11:52.593621 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:52.593454    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb" event={"ID":"e42d5171-57cf-4041-8d94-722a49c92c9b","Type":"ContainerStarted","Data":"431c1b5d187106ebf4e65acd8086ea1783cdc3de2c59d515ebfeb07898891dcb"}
Jul 28 17:11:52.597492 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:52.597422    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:11:52.633557 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:52.633496    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5fbbdb8f87-rkdgb" podStartSLOduration=2.603821332 podStartE2EDuration="11.633457863s" totalImagesPullingTime="9.029636531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:52.633035808 +0000 UTC m=+67.879138016" watchObservedRunningTime="2026-07-28 17:11:52.633457863 +0000 UTC m=+67.879560109"
Jul 28 17:11:53.598365 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:53.598331    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fr9xb" event={"ID":"97f130a6-e3b7-4882-b992-0d5eef10897b","Type":"ContainerStarted","Data":"f8b9bf994798a68787d6daa08486cbed7cff865081948a881a42c1ae7fd9daf6"}
Jul 28 17:11:53.598767 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:53.598389    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fr9xb" event={"ID":"97f130a6-e3b7-4882-b992-0d5eef10897b","Type":"ContainerStarted","Data":"f391c94b4f15c1c35b4c5b4199946c654e08ea43b1bc7091680b9027c29310e9"}
Jul 28 17:11:53.621953 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:53.621899    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fr9xb" podStartSLOduration=67.302982551 podStartE2EDuration="1m8.621885061s" totalImagesPullingTime="1.31890251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:10:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:53.621739082 +0000 UTC m=+68.867841289" watchObservedRunningTime="2026-07-28 17:11:53.621885061 +0000 UTC m=+68.867987265"
Jul 28 17:11:56.119784 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.119744    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mfm74"]
Jul 28 17:11:56.190807 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.190768    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.194289 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.194263    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 17:11:56.194435 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.194263    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 17:11:56.194435 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.194264    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 17:11:56.194435 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.194276    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 17:11:56.194705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.194690    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 17:11:56.194784 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.194745    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pzgw5\""
Jul 28 17:11:56.196846 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.196809    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 17:11:56.244353 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.244324    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-node-exporter-textfile\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.244527 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.244362    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-node-exporter-tls\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.244527 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.244394    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-node-exporter-accelerators-collector-config\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.244527 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.244444    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-metrics-client-ca\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.244527 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.244472    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-sys\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.244694 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.244532    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-root\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.244694 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.244548    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-node-exporter-wtmp\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.244694 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.244596    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.244694 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.244617    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n52mn\" (UniqueName: \"kubernetes.io/projected/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-kube-api-access-n52mn\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.345333 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.345304    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.345333 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.345332    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n52mn\" (UniqueName: \"kubernetes.io/projected/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-kube-api-access-n52mn\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.345568 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.345355    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-node-exporter-textfile\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.345568 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.345378    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-node-exporter-tls\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.345568 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.345410    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-node-exporter-accelerators-collector-config\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.345568 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.345442    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-metrics-client-ca\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.345568 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.345463    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-sys\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.345568 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:56.345495    2567 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 17:11:56.345568 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.345527    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-root\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.345568 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.345550    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-node-exporter-wtmp\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.345885 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.345583    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-sys\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.345885 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.345620    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-root\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.345885 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:11:56.345666    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-node-exporter-tls podName:b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:56.845558899 +0000 UTC m=+72.091661094 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-node-exporter-tls") pod "node-exporter-mfm74" (UID: "b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2") : secret "node-exporter-tls" not found
Jul 28 17:11:56.345885 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.345666    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-node-exporter-wtmp\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.345885 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.345736    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-node-exporter-textfile\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.346062 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.346043    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-node-exporter-accelerators-collector-config\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.346110 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.346093    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-metrics-client-ca\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.347540 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.347522    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.356971 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.356952    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n52mn\" (UniqueName: \"kubernetes.io/projected/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-kube-api-access-n52mn\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.848842 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.848808    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-node-exporter-tls\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:56.851069 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:56.851043    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2-node-exporter-tls\") pod \"node-exporter-mfm74\" (UID: \"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2\") " pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:57.100425 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:57.100350    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mfm74"
Jul 28 17:11:57.141149 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:11:57.141115    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0c92dfd_b79b_4cf0_ab2b_5c8823ace7e2.slice/crio-715f5cf0b293d1da59a44b5e4ff971a469f8b15d997efe53848c6a5082d0ed4b WatchSource:0}: Error finding container 715f5cf0b293d1da59a44b5e4ff971a469f8b15d997efe53848c6a5082d0ed4b: Status 404 returned error can't find the container with id 715f5cf0b293d1da59a44b5e4ff971a469f8b15d997efe53848c6a5082d0ed4b
Jul 28 17:11:57.479087 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:57.479010    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-67x96"
Jul 28 17:11:57.611562 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:57.611531    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mfm74" event={"ID":"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2","Type":"ContainerStarted","Data":"715f5cf0b293d1da59a44b5e4ff971a469f8b15d997efe53848c6a5082d0ed4b"}
Jul 28 17:11:59.619167 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:59.619135    2567 generic.go:356] "Generic (PLEG): container finished" podID="b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2" containerID="38c4d7de776b8559c2cefdca4a66e1de7ddf6f36bb679bd2647200142f4a5f51" exitCode=0
Jul 28 17:11:59.619498 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:11:59.619209    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mfm74" event={"ID":"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2","Type":"ContainerDied","Data":"38c4d7de776b8559c2cefdca4a66e1de7ddf6f36bb679bd2647200142f4a5f51"}
Jul 28 17:12:00.474470 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:00.474439    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:12:00.474736 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:00.474493    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:12:00.480191 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:00.480161    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:12:00.624103 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:00.624067    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mfm74" event={"ID":"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2","Type":"ContainerStarted","Data":"64187733ba3e7abba5ca8b82a2e219fbc1954b79926c508f6a34ed088e403ba9"}
Jul 28 17:12:00.624478 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:00.624150    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mfm74" event={"ID":"b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2","Type":"ContainerStarted","Data":"fbb0c33486a72b2b727dbed832d202c5dbbac362a9a45c778c557c8ff68aab94"}
Jul 28 17:12:00.627798 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:00.627777    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:12:00.647381 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:00.647342    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mfm74" podStartSLOduration=3.166364276 podStartE2EDuration="4.647328839s" totalImagesPullingTime="1.37869971s" totalInitContainerRuntime="102.264853ms" isStatefulPod=true podCreationTimestamp="2026-07-28 17:11:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:12:00.645911839 +0000 UTC m=+75.892014045" watchObservedRunningTime="2026-07-28 17:12:00.647328839 +0000 UTC m=+75.893431043"
Jul 28 17:12:00.697596 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:00.697570    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cb5cf6548-7wmk4"]
Jul 28 17:12:02.201719 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.201688    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f69cd7647-dd5p4"]
Jul 28 17:12:02.204146 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.204128    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.216715 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.216694    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f69cd7647-dd5p4"]
Jul 28 17:12:02.286630 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.286603    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-trusted-ca-bundle\") pod \"console-5f69cd7647-dd5p4\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") " pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.286734 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.286633    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-oauth-serving-cert\") pod \"console-5f69cd7647-dd5p4\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") " pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.286734 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.286653    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-console-config\") pod \"console-5f69cd7647-dd5p4\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") " pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.286734 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.286670    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb465\" (UniqueName: \"kubernetes.io/projected/978503de-9c2f-4116-a702-311ef0795077-kube-api-access-hb465\") pod \"console-5f69cd7647-dd5p4\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") " pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.286830 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.286714    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/978503de-9c2f-4116-a702-311ef0795077-console-oauth-config\") pod \"console-5f69cd7647-dd5p4\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") " pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.286830 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.286763    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-service-ca\") pod \"console-5f69cd7647-dd5p4\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") " pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.286830 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.286791    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/978503de-9c2f-4116-a702-311ef0795077-console-serving-cert\") pod \"console-5f69cd7647-dd5p4\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") " pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.387890 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.387867    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/978503de-9c2f-4116-a702-311ef0795077-console-oauth-config\") pod \"console-5f69cd7647-dd5p4\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") " pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.388008 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.387895    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-service-ca\") pod \"console-5f69cd7647-dd5p4\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") " pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.388008 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.387925    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/978503de-9c2f-4116-a702-311ef0795077-console-serving-cert\") pod \"console-5f69cd7647-dd5p4\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") " pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.388008 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.387945    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-trusted-ca-bundle\") pod \"console-5f69cd7647-dd5p4\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") " pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.388008 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.387972    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-oauth-serving-cert\") pod \"console-5f69cd7647-dd5p4\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") " pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.388196 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.388085    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-console-config\") pod \"console-5f69cd7647-dd5p4\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") " pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.388196 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.388124    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb465\" (UniqueName: \"kubernetes.io/projected/978503de-9c2f-4116-a702-311ef0795077-kube-api-access-hb465\") pod \"console-5f69cd7647-dd5p4\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") " pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.388805 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.388779    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-service-ca\") pod \"console-5f69cd7647-dd5p4\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") " pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.388892 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.388807    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-oauth-serving-cert\") pod \"console-5f69cd7647-dd5p4\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") " pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.388892 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.388842    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-console-config\") pod \"console-5f69cd7647-dd5p4\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") " pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.388892 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.388870    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-trusted-ca-bundle\") pod \"console-5f69cd7647-dd5p4\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") " pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.390775 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.390754    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/978503de-9c2f-4116-a702-311ef0795077-console-serving-cert\") pod \"console-5f69cd7647-dd5p4\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") " pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.390888 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.390868    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/978503de-9c2f-4116-a702-311ef0795077-console-oauth-config\") pod \"console-5f69cd7647-dd5p4\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") " pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.398602 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.398582    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb465\" (UniqueName: \"kubernetes.io/projected/978503de-9c2f-4116-a702-311ef0795077-kube-api-access-hb465\") pod \"console-5f69cd7647-dd5p4\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") " pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.512899 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.512883    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:02.634957 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:02.634930    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f69cd7647-dd5p4"]
Jul 28 17:12:02.637113 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:12:02.637088    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod978503de_9c2f_4116_a702_311ef0795077.slice/crio-f8d5ff057418a015986e3cf1a9b796091dd4a07e5239c07791596172aeff4a3b WatchSource:0}: Error finding container f8d5ff057418a015986e3cf1a9b796091dd4a07e5239c07791596172aeff4a3b: Status 404 returned error can't find the container with id f8d5ff057418a015986e3cf1a9b796091dd4a07e5239c07791596172aeff4a3b
Jul 28 17:12:03.636272 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:03.636236    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f69cd7647-dd5p4" event={"ID":"978503de-9c2f-4116-a702-311ef0795077","Type":"ContainerStarted","Data":"52d7cf6e43a053c1e4802c555b2adfd27aa26f5f8ca801177ea63896124351cd"}
Jul 28 17:12:03.636730 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:03.636283    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f69cd7647-dd5p4" event={"ID":"978503de-9c2f-4116-a702-311ef0795077","Type":"ContainerStarted","Data":"f8d5ff057418a015986e3cf1a9b796091dd4a07e5239c07791596172aeff4a3b"}
Jul 28 17:12:03.669729 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:03.669682    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f69cd7647-dd5p4" podStartSLOduration=1.669667808 podStartE2EDuration="1.669667808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:12:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:12:03.668459118 +0000 UTC m=+78.914561326" watchObservedRunningTime="2026-07-28 17:12:03.669667808 +0000 UTC m=+78.915770014"
Jul 28 17:12:12.513717 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:12.513596    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:12.513717 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:12.513652    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:12.518013 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:12.517991    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:12.669262 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:12.669237    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:12.762967 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:12.762935    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-798d7f87ff-868gf"]
Jul 28 17:12:24.883563 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:24.883528    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f69cd7647-dd5p4"]
Jul 28 17:12:25.715812 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:25.715775    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5cb5cf6548-7wmk4" podUID="e524bd8e-daf3-4b21-9c83-6f4e11a66164" containerName="console" containerID="cri-o://c919511ad7795ae870b9b116428f539e1a3be98ec4cafe7ae2683ce6e7939d92" gracePeriod=15
Jul 28 17:12:25.937882 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:25.937856    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:12:26.066866 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.063776    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e524bd8e-daf3-4b21-9c83-6f4e11a66164-service-ca\") pod \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") "
Jul 28 17:12:26.066866 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.063882    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-497pk\" (UniqueName: \"kubernetes.io/projected/e524bd8e-daf3-4b21-9c83-6f4e11a66164-kube-api-access-497pk\") pod \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") "
Jul 28 17:12:26.066866 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.063942    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e524bd8e-daf3-4b21-9c83-6f4e11a66164-console-config\") pod \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") "
Jul 28 17:12:26.066866 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.063997    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e524bd8e-daf3-4b21-9c83-6f4e11a66164-console-serving-cert\") pod \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") "
Jul 28 17:12:26.066866 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.064033    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e524bd8e-daf3-4b21-9c83-6f4e11a66164-console-oauth-config\") pod \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") "
Jul 28 17:12:26.066866 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.064095    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e524bd8e-daf3-4b21-9c83-6f4e11a66164-oauth-serving-cert\") pod \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\" (UID: \"e524bd8e-daf3-4b21-9c83-6f4e11a66164\") "
Jul 28 17:12:26.066866 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.064175    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e524bd8e-daf3-4b21-9c83-6f4e11a66164-service-ca" (OuterVolumeSpecName: "service-ca") pod "e524bd8e-daf3-4b21-9c83-6f4e11a66164" (UID: "e524bd8e-daf3-4b21-9c83-6f4e11a66164"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:12:26.066866 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.064371    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e524bd8e-daf3-4b21-9c83-6f4e11a66164-service-ca\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:12:26.066866 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.064499    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e524bd8e-daf3-4b21-9c83-6f4e11a66164-console-config" (OuterVolumeSpecName: "console-config") pod "e524bd8e-daf3-4b21-9c83-6f4e11a66164" (UID: "e524bd8e-daf3-4b21-9c83-6f4e11a66164"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:12:26.067961 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.067746    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e524bd8e-daf3-4b21-9c83-6f4e11a66164-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e524bd8e-daf3-4b21-9c83-6f4e11a66164" (UID: "e524bd8e-daf3-4b21-9c83-6f4e11a66164"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:12:26.068282 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.068251    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e524bd8e-daf3-4b21-9c83-6f4e11a66164-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e524bd8e-daf3-4b21-9c83-6f4e11a66164" (UID: "e524bd8e-daf3-4b21-9c83-6f4e11a66164"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:12:26.068686 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.068664    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e524bd8e-daf3-4b21-9c83-6f4e11a66164-kube-api-access-497pk" (OuterVolumeSpecName: "kube-api-access-497pk") pod "e524bd8e-daf3-4b21-9c83-6f4e11a66164" (UID: "e524bd8e-daf3-4b21-9c83-6f4e11a66164"). InnerVolumeSpecName "kube-api-access-497pk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:12:26.069867 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.069845    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e524bd8e-daf3-4b21-9c83-6f4e11a66164-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e524bd8e-daf3-4b21-9c83-6f4e11a66164" (UID: "e524bd8e-daf3-4b21-9c83-6f4e11a66164"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:12:26.164720 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.164683    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e524bd8e-daf3-4b21-9c83-6f4e11a66164-oauth-serving-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:12:26.164720 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.164718    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-497pk\" (UniqueName: \"kubernetes.io/projected/e524bd8e-daf3-4b21-9c83-6f4e11a66164-kube-api-access-497pk\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:12:26.164720 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.164728    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e524bd8e-daf3-4b21-9c83-6f4e11a66164-console-config\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:12:26.164891 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.164737    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e524bd8e-daf3-4b21-9c83-6f4e11a66164-console-serving-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:12:26.164891 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.164745    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e524bd8e-daf3-4b21-9c83-6f4e11a66164-console-oauth-config\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:12:26.710012 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.709976    2567 generic.go:356] "Generic (PLEG): container finished" podID="e524bd8e-daf3-4b21-9c83-6f4e11a66164" containerID="c919511ad7795ae870b9b116428f539e1a3be98ec4cafe7ae2683ce6e7939d92" exitCode=0
Jul 28 17:12:26.710176 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.710052    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb5cf6548-7wmk4"
Jul 28 17:12:26.710176 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.710051    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb5cf6548-7wmk4" event={"ID":"e524bd8e-daf3-4b21-9c83-6f4e11a66164","Type":"ContainerDied","Data":"c919511ad7795ae870b9b116428f539e1a3be98ec4cafe7ae2683ce6e7939d92"}
Jul 28 17:12:26.710176 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.710165    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb5cf6548-7wmk4" event={"ID":"e524bd8e-daf3-4b21-9c83-6f4e11a66164","Type":"ContainerDied","Data":"f1fb2eb84c09c652faabec10d9b002c96d75c28d214793bb29ea5afcaedd86e3"}
Jul 28 17:12:26.710292 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.710186    2567 scope.go:117] "RemoveContainer" containerID="c919511ad7795ae870b9b116428f539e1a3be98ec4cafe7ae2683ce6e7939d92"
Jul 28 17:12:26.718353 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.718239    2567 scope.go:117] "RemoveContainer" containerID="c919511ad7795ae870b9b116428f539e1a3be98ec4cafe7ae2683ce6e7939d92"
Jul 28 17:12:26.718551 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:12:26.718525    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c919511ad7795ae870b9b116428f539e1a3be98ec4cafe7ae2683ce6e7939d92\": container with ID starting with c919511ad7795ae870b9b116428f539e1a3be98ec4cafe7ae2683ce6e7939d92 not found: ID does not exist" containerID="c919511ad7795ae870b9b116428f539e1a3be98ec4cafe7ae2683ce6e7939d92"
Jul 28 17:12:26.718638 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.718559    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c919511ad7795ae870b9b116428f539e1a3be98ec4cafe7ae2683ce6e7939d92"} err="failed to get container status \"c919511ad7795ae870b9b116428f539e1a3be98ec4cafe7ae2683ce6e7939d92\": rpc error: code = NotFound desc = could not find container \"c919511ad7795ae870b9b116428f539e1a3be98ec4cafe7ae2683ce6e7939d92\": container with ID starting with c919511ad7795ae870b9b116428f539e1a3be98ec4cafe7ae2683ce6e7939d92 not found: ID does not exist"
Jul 28 17:12:26.732528 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.732490    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cb5cf6548-7wmk4"]
Jul 28 17:12:26.736445 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:26.736424    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cb5cf6548-7wmk4"]
Jul 28 17:12:27.285608 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:27.285575    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e524bd8e-daf3-4b21-9c83-6f4e11a66164" path="/var/lib/kubelet/pods/e524bd8e-daf3-4b21-9c83-6f4e11a66164/volumes"
Jul 28 17:12:37.780359 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:37.780320    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-798d7f87ff-868gf" podUID="42fc4f58-4740-4391-a887-bf3594fc8bc8" containerName="console" containerID="cri-o://b45a7acd83532adc8b01ce0399b59e459f70adda6f88557f16a316a74bee6a55" gracePeriod=15
Jul 28 17:12:38.003977 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.003957    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:12:38.048977 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.048887    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42fc4f58-4740-4391-a887-bf3594fc8bc8-console-oauth-config\") pod \"42fc4f58-4740-4391-a887-bf3594fc8bc8\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") "
Jul 28 17:12:38.048977 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.048935    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-console-config\") pod \"42fc4f58-4740-4391-a887-bf3594fc8bc8\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") "
Jul 28 17:12:38.048977 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.048955    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-trusted-ca-bundle\") pod \"42fc4f58-4740-4391-a887-bf3594fc8bc8\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") "
Jul 28 17:12:38.049297 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.048986    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44z88\" (UniqueName: \"kubernetes.io/projected/42fc4f58-4740-4391-a887-bf3594fc8bc8-kube-api-access-44z88\") pod \"42fc4f58-4740-4391-a887-bf3594fc8bc8\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") "
Jul 28 17:12:38.049297 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.049008    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-service-ca\") pod \"42fc4f58-4740-4391-a887-bf3594fc8bc8\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") "
Jul 28 17:12:38.049297 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.049043    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-oauth-serving-cert\") pod \"42fc4f58-4740-4391-a887-bf3594fc8bc8\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") "
Jul 28 17:12:38.049297 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.049079    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42fc4f58-4740-4391-a887-bf3594fc8bc8-console-serving-cert\") pod \"42fc4f58-4740-4391-a887-bf3594fc8bc8\" (UID: \"42fc4f58-4740-4391-a887-bf3594fc8bc8\") "
Jul 28 17:12:38.049488 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.049420    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-console-config" (OuterVolumeSpecName: "console-config") pod "42fc4f58-4740-4391-a887-bf3594fc8bc8" (UID: "42fc4f58-4740-4391-a887-bf3594fc8bc8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:12:38.049488 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.049448    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-service-ca" (OuterVolumeSpecName: "service-ca") pod "42fc4f58-4740-4391-a887-bf3594fc8bc8" (UID: "42fc4f58-4740-4391-a887-bf3594fc8bc8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:12:38.049488 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.049472    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "42fc4f58-4740-4391-a887-bf3594fc8bc8" (UID: "42fc4f58-4740-4391-a887-bf3594fc8bc8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:12:38.049677 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.049465    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "42fc4f58-4740-4391-a887-bf3594fc8bc8" (UID: "42fc4f58-4740-4391-a887-bf3594fc8bc8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:12:38.051032 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.051006    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42fc4f58-4740-4391-a887-bf3594fc8bc8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "42fc4f58-4740-4391-a887-bf3594fc8bc8" (UID: "42fc4f58-4740-4391-a887-bf3594fc8bc8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:12:38.051175 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.051152    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42fc4f58-4740-4391-a887-bf3594fc8bc8-kube-api-access-44z88" (OuterVolumeSpecName: "kube-api-access-44z88") pod "42fc4f58-4740-4391-a887-bf3594fc8bc8" (UID: "42fc4f58-4740-4391-a887-bf3594fc8bc8"). InnerVolumeSpecName "kube-api-access-44z88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:12:38.051175 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.051165    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42fc4f58-4740-4391-a887-bf3594fc8bc8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "42fc4f58-4740-4391-a887-bf3594fc8bc8" (UID: "42fc4f58-4740-4391-a887-bf3594fc8bc8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:12:38.149760 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.149722    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-oauth-serving-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:12:38.149760 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.149754    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42fc4f58-4740-4391-a887-bf3594fc8bc8-console-serving-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:12:38.149760 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.149762    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42fc4f58-4740-4391-a887-bf3594fc8bc8-console-oauth-config\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:12:38.149973 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.149772    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-console-config\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:12:38.149973 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.149780    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-trusted-ca-bundle\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:12:38.149973 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.149788    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44z88\" (UniqueName: \"kubernetes.io/projected/42fc4f58-4740-4391-a887-bf3594fc8bc8-kube-api-access-44z88\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:12:38.149973 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.149797    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42fc4f58-4740-4391-a887-bf3594fc8bc8-service-ca\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:12:38.749305 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.749269    2567 generic.go:356] "Generic (PLEG): container finished" podID="42fc4f58-4740-4391-a887-bf3594fc8bc8" containerID="b45a7acd83532adc8b01ce0399b59e459f70adda6f88557f16a316a74bee6a55" exitCode=0
Jul 28 17:12:38.749470 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.749317    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-798d7f87ff-868gf"
Jul 28 17:12:38.749470 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.749349    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-798d7f87ff-868gf" event={"ID":"42fc4f58-4740-4391-a887-bf3594fc8bc8","Type":"ContainerDied","Data":"b45a7acd83532adc8b01ce0399b59e459f70adda6f88557f16a316a74bee6a55"}
Jul 28 17:12:38.749470 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.749397    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-798d7f87ff-868gf" event={"ID":"42fc4f58-4740-4391-a887-bf3594fc8bc8","Type":"ContainerDied","Data":"b5d8c92cd55239bff2a3df5834f740304d024cc252f4b44d9160fb7f94349635"}
Jul 28 17:12:38.749470 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.749417    2567 scope.go:117] "RemoveContainer" containerID="b45a7acd83532adc8b01ce0399b59e459f70adda6f88557f16a316a74bee6a55"
Jul 28 17:12:38.760896 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.760876    2567 scope.go:117] "RemoveContainer" containerID="b45a7acd83532adc8b01ce0399b59e459f70adda6f88557f16a316a74bee6a55"
Jul 28 17:12:38.761227 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:12:38.761205    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b45a7acd83532adc8b01ce0399b59e459f70adda6f88557f16a316a74bee6a55\": container with ID starting with b45a7acd83532adc8b01ce0399b59e459f70adda6f88557f16a316a74bee6a55 not found: ID does not exist" containerID="b45a7acd83532adc8b01ce0399b59e459f70adda6f88557f16a316a74bee6a55"
Jul 28 17:12:38.761269 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.761236    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b45a7acd83532adc8b01ce0399b59e459f70adda6f88557f16a316a74bee6a55"} err="failed to get container status \"b45a7acd83532adc8b01ce0399b59e459f70adda6f88557f16a316a74bee6a55\": rpc error: code = NotFound desc = could not find container \"b45a7acd83532adc8b01ce0399b59e459f70adda6f88557f16a316a74bee6a55\": container with ID starting with b45a7acd83532adc8b01ce0399b59e459f70adda6f88557f16a316a74bee6a55 not found: ID does not exist"
Jul 28 17:12:38.774367 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.774337    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-798d7f87ff-868gf"]
Jul 28 17:12:38.778214 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:38.778188    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-798d7f87ff-868gf"]
Jul 28 17:12:39.286644 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:39.286609    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42fc4f58-4740-4391-a887-bf3594fc8bc8" path="/var/lib/kubelet/pods/42fc4f58-4740-4391-a887-bf3594fc8bc8/volumes"
Jul 28 17:12:49.901930 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:49.901883    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f69cd7647-dd5p4" podUID="978503de-9c2f-4116-a702-311ef0795077" containerName="console" containerID="cri-o://52d7cf6e43a053c1e4802c555b2adfd27aa26f5f8ca801177ea63896124351cd" gracePeriod=15
Jul 28 17:12:50.138147 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.138121    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:50.241178 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.241088    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-service-ca\") pod \"978503de-9c2f-4116-a702-311ef0795077\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") "
Jul 28 17:12:50.241178 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.241133    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-console-config\") pod \"978503de-9c2f-4116-a702-311ef0795077\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") "
Jul 28 17:12:50.241178 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.241167    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-oauth-serving-cert\") pod \"978503de-9c2f-4116-a702-311ef0795077\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") "
Jul 28 17:12:50.241410 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.241187    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hb465\" (UniqueName: \"kubernetes.io/projected/978503de-9c2f-4116-a702-311ef0795077-kube-api-access-hb465\") pod \"978503de-9c2f-4116-a702-311ef0795077\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") "
Jul 28 17:12:50.241410 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.241378    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/978503de-9c2f-4116-a702-311ef0795077-console-oauth-config\") pod \"978503de-9c2f-4116-a702-311ef0795077\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") "
Jul 28 17:12:50.241539 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.241427    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-trusted-ca-bundle\") pod \"978503de-9c2f-4116-a702-311ef0795077\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") "
Jul 28 17:12:50.241539 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.241454    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/978503de-9c2f-4116-a702-311ef0795077-console-serving-cert\") pod \"978503de-9c2f-4116-a702-311ef0795077\" (UID: \"978503de-9c2f-4116-a702-311ef0795077\") "
Jul 28 17:12:50.241647 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.241532    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-service-ca" (OuterVolumeSpecName: "service-ca") pod "978503de-9c2f-4116-a702-311ef0795077" (UID: "978503de-9c2f-4116-a702-311ef0795077"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:12:50.241647 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.241584    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "978503de-9c2f-4116-a702-311ef0795077" (UID: "978503de-9c2f-4116-a702-311ef0795077"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:12:50.241647 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.241613    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-console-config" (OuterVolumeSpecName: "console-config") pod "978503de-9c2f-4116-a702-311ef0795077" (UID: "978503de-9c2f-4116-a702-311ef0795077"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:12:50.241820 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.241803    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-console-config\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:12:50.241877 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.241820    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-oauth-serving-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:12:50.241877 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.241830    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-service-ca\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:12:50.241946 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.241876    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "978503de-9c2f-4116-a702-311ef0795077" (UID: "978503de-9c2f-4116-a702-311ef0795077"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:12:50.243317 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.243285    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/978503de-9c2f-4116-a702-311ef0795077-kube-api-access-hb465" (OuterVolumeSpecName: "kube-api-access-hb465") pod "978503de-9c2f-4116-a702-311ef0795077" (UID: "978503de-9c2f-4116-a702-311ef0795077"). InnerVolumeSpecName "kube-api-access-hb465". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:12:50.243498 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.243472    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/978503de-9c2f-4116-a702-311ef0795077-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "978503de-9c2f-4116-a702-311ef0795077" (UID: "978503de-9c2f-4116-a702-311ef0795077"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:12:50.243637 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.243498    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/978503de-9c2f-4116-a702-311ef0795077-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "978503de-9c2f-4116-a702-311ef0795077" (UID: "978503de-9c2f-4116-a702-311ef0795077"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:12:50.342872 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.342831    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hb465\" (UniqueName: \"kubernetes.io/projected/978503de-9c2f-4116-a702-311ef0795077-kube-api-access-hb465\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:12:50.342872 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.342863    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/978503de-9c2f-4116-a702-311ef0795077-console-oauth-config\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:12:50.342872 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.342872    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/978503de-9c2f-4116-a702-311ef0795077-trusted-ca-bundle\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:12:50.342872 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.342881    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/978503de-9c2f-4116-a702-311ef0795077-console-serving-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:12:50.792756 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.792717    2567 generic.go:356] "Generic (PLEG): container finished" podID="978503de-9c2f-4116-a702-311ef0795077" containerID="52d7cf6e43a053c1e4802c555b2adfd27aa26f5f8ca801177ea63896124351cd" exitCode=0
Jul 28 17:12:50.792967 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.792760    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f69cd7647-dd5p4"
Jul 28 17:12:50.792967 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.792772    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f69cd7647-dd5p4" event={"ID":"978503de-9c2f-4116-a702-311ef0795077","Type":"ContainerDied","Data":"52d7cf6e43a053c1e4802c555b2adfd27aa26f5f8ca801177ea63896124351cd"}
Jul 28 17:12:50.792967 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.792823    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f69cd7647-dd5p4" event={"ID":"978503de-9c2f-4116-a702-311ef0795077","Type":"ContainerDied","Data":"f8d5ff057418a015986e3cf1a9b796091dd4a07e5239c07791596172aeff4a3b"}
Jul 28 17:12:50.792967 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.792848    2567 scope.go:117] "RemoveContainer" containerID="52d7cf6e43a053c1e4802c555b2adfd27aa26f5f8ca801177ea63896124351cd"
Jul 28 17:12:50.800893 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.800875    2567 scope.go:117] "RemoveContainer" containerID="52d7cf6e43a053c1e4802c555b2adfd27aa26f5f8ca801177ea63896124351cd"
Jul 28 17:12:50.801139 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:12:50.801119    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52d7cf6e43a053c1e4802c555b2adfd27aa26f5f8ca801177ea63896124351cd\": container with ID starting with 52d7cf6e43a053c1e4802c555b2adfd27aa26f5f8ca801177ea63896124351cd not found: ID does not exist" containerID="52d7cf6e43a053c1e4802c555b2adfd27aa26f5f8ca801177ea63896124351cd"
Jul 28 17:12:50.801189 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.801148    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52d7cf6e43a053c1e4802c555b2adfd27aa26f5f8ca801177ea63896124351cd"} err="failed to get container status \"52d7cf6e43a053c1e4802c555b2adfd27aa26f5f8ca801177ea63896124351cd\": rpc error: code = NotFound desc = could not find container \"52d7cf6e43a053c1e4802c555b2adfd27aa26f5f8ca801177ea63896124351cd\": container with ID starting with 52d7cf6e43a053c1e4802c555b2adfd27aa26f5f8ca801177ea63896124351cd not found: ID does not exist"
Jul 28 17:12:50.813764 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.813738    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f69cd7647-dd5p4"]
Jul 28 17:12:50.823800 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:50.820930    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f69cd7647-dd5p4"]
Jul 28 17:12:51.286419 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:12:51.286381    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="978503de-9c2f-4116-a702-311ef0795077" path="/var/lib/kubelet/pods/978503de-9c2f-4116-a702-311ef0795077/volumes"
Jul 28 17:13:27.834074 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.834036    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-778689875f-px9hf"]
Jul 28 17:13:27.834619 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.834339    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e524bd8e-daf3-4b21-9c83-6f4e11a66164" containerName="console"
Jul 28 17:13:27.834619 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.834358    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e524bd8e-daf3-4b21-9c83-6f4e11a66164" containerName="console"
Jul 28 17:13:27.834619 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.834371    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="978503de-9c2f-4116-a702-311ef0795077" containerName="console"
Jul 28 17:13:27.834619 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.834379    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="978503de-9c2f-4116-a702-311ef0795077" containerName="console"
Jul 28 17:13:27.834619 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.834397    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42fc4f58-4740-4391-a887-bf3594fc8bc8" containerName="console"
Jul 28 17:13:27.834619 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.834406    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="42fc4f58-4740-4391-a887-bf3594fc8bc8" containerName="console"
Jul 28 17:13:27.834619 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.834454    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e524bd8e-daf3-4b21-9c83-6f4e11a66164" containerName="console"
Jul 28 17:13:27.834619 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.834465    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="978503de-9c2f-4116-a702-311ef0795077" containerName="console"
Jul 28 17:13:27.834619 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.834475    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="42fc4f58-4740-4391-a887-bf3594fc8bc8" containerName="console"
Jul 28 17:13:27.836420 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.836401    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:27.839716 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.839696    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 17:13:27.839819 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.839727    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 17:13:27.839892 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.839830    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 17:13:27.840699 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.840680    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 17:13:27.840780 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.840721    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 17:13:27.840828 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.840793    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 17:13:27.841267 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.841251    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-l68jz\""
Jul 28 17:13:27.841347 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.841256    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 17:13:27.846111 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.846093    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 17:13:27.850655 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.850631    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-778689875f-px9hf"]
Jul 28 17:13:27.993385 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.993344    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-console-config\") pod \"console-778689875f-px9hf\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") " pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:27.993385 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.993379    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-oauth-serving-cert\") pod \"console-778689875f-px9hf\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") " pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:27.993639 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.993399    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1c742d2-d065-434d-8f21-f72fbb37b6a8-console-serving-cert\") pod \"console-778689875f-px9hf\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") " pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:27.993639 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.993447    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-service-ca\") pod \"console-778689875f-px9hf\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") " pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:27.993639 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.993495    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-trusted-ca-bundle\") pod \"console-778689875f-px9hf\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") " pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:27.993639 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.993564    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdzkl\" (UniqueName: \"kubernetes.io/projected/e1c742d2-d065-434d-8f21-f72fbb37b6a8-kube-api-access-bdzkl\") pod \"console-778689875f-px9hf\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") " pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:27.993639 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:27.993592    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1c742d2-d065-434d-8f21-f72fbb37b6a8-console-oauth-config\") pod \"console-778689875f-px9hf\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") " pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:28.094904 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:28.094829    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdzkl\" (UniqueName: \"kubernetes.io/projected/e1c742d2-d065-434d-8f21-f72fbb37b6a8-kube-api-access-bdzkl\") pod \"console-778689875f-px9hf\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") " pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:28.094904 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:28.094872    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1c742d2-d065-434d-8f21-f72fbb37b6a8-console-oauth-config\") pod \"console-778689875f-px9hf\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") " pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:28.095045 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:28.094905    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-console-config\") pod \"console-778689875f-px9hf\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") " pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:28.095045 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:28.094922    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-oauth-serving-cert\") pod \"console-778689875f-px9hf\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") " pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:28.095045 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:28.094937    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1c742d2-d065-434d-8f21-f72fbb37b6a8-console-serving-cert\") pod \"console-778689875f-px9hf\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") " pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:28.095045 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:28.094952    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-service-ca\") pod \"console-778689875f-px9hf\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") " pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:28.095045 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:28.094974    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-trusted-ca-bundle\") pod \"console-778689875f-px9hf\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") " pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:28.096217 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:28.096185    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-console-config\") pod \"console-778689875f-px9hf\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") " pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:28.096330 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:28.096227    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-oauth-serving-cert\") pod \"console-778689875f-px9hf\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") " pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:28.096411 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:28.096392    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-service-ca\") pod \"console-778689875f-px9hf\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") " pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:28.096550 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:28.096500    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-trusted-ca-bundle\") pod \"console-778689875f-px9hf\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") " pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:28.100955 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:28.100931    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1c742d2-d065-434d-8f21-f72fbb37b6a8-console-oauth-config\") pod \"console-778689875f-px9hf\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") " pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:28.101943 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:28.101920    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1c742d2-d065-434d-8f21-f72fbb37b6a8-console-serving-cert\") pod \"console-778689875f-px9hf\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") " pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:28.109487 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:28.109466    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdzkl\" (UniqueName: \"kubernetes.io/projected/e1c742d2-d065-434d-8f21-f72fbb37b6a8-kube-api-access-bdzkl\") pod \"console-778689875f-px9hf\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") " pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:28.146189 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:28.146160    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:28.266035 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:28.266012    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-778689875f-px9hf"]
Jul 28 17:13:28.268299 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:13:28.268274    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1c742d2_d065_434d_8f21_f72fbb37b6a8.slice/crio-af912c579d14ca0a87dea240adabb31a918c92a6cf584e51d7276f92b3646c4c WatchSource:0}: Error finding container af912c579d14ca0a87dea240adabb31a918c92a6cf584e51d7276f92b3646c4c: Status 404 returned error can't find the container with id af912c579d14ca0a87dea240adabb31a918c92a6cf584e51d7276f92b3646c4c
Jul 28 17:13:28.916078 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:28.916045    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-778689875f-px9hf" event={"ID":"e1c742d2-d065-434d-8f21-f72fbb37b6a8","Type":"ContainerStarted","Data":"61be3886d6c697a605406bd5be8908f3c7252130a60e098ff49106ee1aa73080"}
Jul 28 17:13:28.916487 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:28.916090    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-778689875f-px9hf" event={"ID":"e1c742d2-d065-434d-8f21-f72fbb37b6a8","Type":"ContainerStarted","Data":"af912c579d14ca0a87dea240adabb31a918c92a6cf584e51d7276f92b3646c4c"}
Jul 28 17:13:28.936443 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:28.936397    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-778689875f-px9hf" podStartSLOduration=1.936381882 podStartE2EDuration="1.936381882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:13:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:28.935920503 +0000 UTC m=+164.182022710" watchObservedRunningTime="2026-07-28 17:13:28.936381882 +0000 UTC m=+164.182484097"
Jul 28 17:13:38.147225 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:38.147139    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:38.147225 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:38.147190    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:38.151761 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:38.151740    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:13:38.949593 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:13:38.949564    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:15:07.600745 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:07.600673    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9"]
Jul 28 17:15:07.602989 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:07.602973    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9"
Jul 28 17:15:07.605895 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:07.605872    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 17:15:07.607211 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:07.607180    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-22pvl\""
Jul 28 17:15:07.607211 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:07.607191    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 17:15:07.614005 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:07.613983    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9"]
Jul 28 17:15:07.652260 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:07.652234    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c21b2db7-a694-46ee-8306-c23b69a28558-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9\" (UID: \"c21b2db7-a694-46ee-8306-c23b69a28558\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9"
Jul 28 17:15:07.652345 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:07.652271    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c21b2db7-a694-46ee-8306-c23b69a28558-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9\" (UID: \"c21b2db7-a694-46ee-8306-c23b69a28558\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9"
Jul 28 17:15:07.652400 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:07.652368    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz7vb\" (UniqueName: \"kubernetes.io/projected/c21b2db7-a694-46ee-8306-c23b69a28558-kube-api-access-fz7vb\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9\" (UID: \"c21b2db7-a694-46ee-8306-c23b69a28558\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9"
Jul 28 17:15:07.753610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:07.753579    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz7vb\" (UniqueName: \"kubernetes.io/projected/c21b2db7-a694-46ee-8306-c23b69a28558-kube-api-access-fz7vb\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9\" (UID: \"c21b2db7-a694-46ee-8306-c23b69a28558\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9"
Jul 28 17:15:07.753768 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:07.753616    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c21b2db7-a694-46ee-8306-c23b69a28558-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9\" (UID: \"c21b2db7-a694-46ee-8306-c23b69a28558\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9"
Jul 28 17:15:07.753768 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:07.753638    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c21b2db7-a694-46ee-8306-c23b69a28558-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9\" (UID: \"c21b2db7-a694-46ee-8306-c23b69a28558\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9"
Jul 28 17:15:07.753973 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:07.753953    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c21b2db7-a694-46ee-8306-c23b69a28558-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9\" (UID: \"c21b2db7-a694-46ee-8306-c23b69a28558\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9"
Jul 28 17:15:07.754029 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:07.753980    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c21b2db7-a694-46ee-8306-c23b69a28558-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9\" (UID: \"c21b2db7-a694-46ee-8306-c23b69a28558\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9"
Jul 28 17:15:07.762090 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:07.762070    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz7vb\" (UniqueName: \"kubernetes.io/projected/c21b2db7-a694-46ee-8306-c23b69a28558-kube-api-access-fz7vb\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9\" (UID: \"c21b2db7-a694-46ee-8306-c23b69a28558\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9"
Jul 28 17:15:07.911839 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:07.911763    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9"
Jul 28 17:15:08.028369 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:08.028343    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9"]
Jul 28 17:15:08.030770 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:15:08.030741    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc21b2db7_a694_46ee_8306_c23b69a28558.slice/crio-7ab4d22905614ba0edb828c05d95e4724ff9153b63a0824d3bb32184c4ec6bc5 WatchSource:0}: Error finding container 7ab4d22905614ba0edb828c05d95e4724ff9153b63a0824d3bb32184c4ec6bc5: Status 404 returned error can't find the container with id 7ab4d22905614ba0edb828c05d95e4724ff9153b63a0824d3bb32184c4ec6bc5
Jul 28 17:15:08.219054 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:08.218976    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9" event={"ID":"c21b2db7-a694-46ee-8306-c23b69a28558","Type":"ContainerStarted","Data":"7ab4d22905614ba0edb828c05d95e4724ff9153b63a0824d3bb32184c4ec6bc5"}
Jul 28 17:15:14.245267 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:14.245240    2567 generic.go:356] "Generic (PLEG): container finished" podID="c21b2db7-a694-46ee-8306-c23b69a28558" containerID="da9476cdbb52856db634c63b9564546c129f22bf464611ce6f6634f8d2d38c40" exitCode=0
Jul 28 17:15:14.245600 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:14.245322    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9" event={"ID":"c21b2db7-a694-46ee-8306-c23b69a28558","Type":"ContainerDied","Data":"da9476cdbb52856db634c63b9564546c129f22bf464611ce6f6634f8d2d38c40"}
Jul 28 17:15:19.262541 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:19.262488    2567 generic.go:356] "Generic (PLEG): container finished" podID="c21b2db7-a694-46ee-8306-c23b69a28558" containerID="7e1be687f3ebd58305acbbf26cf5c3a128d92a3330c95e9d238043fd2385fa45" exitCode=0
Jul 28 17:15:19.262892 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:19.262557    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9" event={"ID":"c21b2db7-a694-46ee-8306-c23b69a28558","Type":"ContainerDied","Data":"7e1be687f3ebd58305acbbf26cf5c3a128d92a3330c95e9d238043fd2385fa45"}
Jul 28 17:15:26.288808 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:26.288778    2567 generic.go:356] "Generic (PLEG): container finished" podID="c21b2db7-a694-46ee-8306-c23b69a28558" containerID="27a9eeb2e3d75dcbd7b2edc8b95c15cba9662fb939fd5cd3a3af2690f7912b4a" exitCode=0
Jul 28 17:15:26.289158 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:26.288863    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9" event={"ID":"c21b2db7-a694-46ee-8306-c23b69a28558","Type":"ContainerDied","Data":"27a9eeb2e3d75dcbd7b2edc8b95c15cba9662fb939fd5cd3a3af2690f7912b4a"}
Jul 28 17:15:27.409697 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:27.409675    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9"
Jul 28 17:15:27.504484 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:27.504457    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c21b2db7-a694-46ee-8306-c23b69a28558-bundle\") pod \"c21b2db7-a694-46ee-8306-c23b69a28558\" (UID: \"c21b2db7-a694-46ee-8306-c23b69a28558\") "
Jul 28 17:15:27.504637 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:27.504535    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fz7vb\" (UniqueName: \"kubernetes.io/projected/c21b2db7-a694-46ee-8306-c23b69a28558-kube-api-access-fz7vb\") pod \"c21b2db7-a694-46ee-8306-c23b69a28558\" (UID: \"c21b2db7-a694-46ee-8306-c23b69a28558\") "
Jul 28 17:15:27.504637 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:27.504556    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c21b2db7-a694-46ee-8306-c23b69a28558-util\") pod \"c21b2db7-a694-46ee-8306-c23b69a28558\" (UID: \"c21b2db7-a694-46ee-8306-c23b69a28558\") "
Jul 28 17:15:27.505082 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:27.505057    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c21b2db7-a694-46ee-8306-c23b69a28558-bundle" (OuterVolumeSpecName: "bundle") pod "c21b2db7-a694-46ee-8306-c23b69a28558" (UID: "c21b2db7-a694-46ee-8306-c23b69a28558"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:15:27.506619 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:27.506598    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c21b2db7-a694-46ee-8306-c23b69a28558-kube-api-access-fz7vb" (OuterVolumeSpecName: "kube-api-access-fz7vb") pod "c21b2db7-a694-46ee-8306-c23b69a28558" (UID: "c21b2db7-a694-46ee-8306-c23b69a28558"). InnerVolumeSpecName "kube-api-access-fz7vb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:15:27.508772 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:27.508752    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c21b2db7-a694-46ee-8306-c23b69a28558-util" (OuterVolumeSpecName: "util") pod "c21b2db7-a694-46ee-8306-c23b69a28558" (UID: "c21b2db7-a694-46ee-8306-c23b69a28558"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:15:27.605426 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:27.605378    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fz7vb\" (UniqueName: \"kubernetes.io/projected/c21b2db7-a694-46ee-8306-c23b69a28558-kube-api-access-fz7vb\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:15:27.605426 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:27.605397    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c21b2db7-a694-46ee-8306-c23b69a28558-util\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:15:27.605426 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:27.605406    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c21b2db7-a694-46ee-8306-c23b69a28558-bundle\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:15:28.296771 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:28.296753    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9"
Jul 28 17:15:28.296860 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:28.296749    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nzlj9" event={"ID":"c21b2db7-a694-46ee-8306-c23b69a28558","Type":"ContainerDied","Data":"7ab4d22905614ba0edb828c05d95e4724ff9153b63a0824d3bb32184c4ec6bc5"}
Jul 28 17:15:28.296898 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:28.296859    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7ab4d22905614ba0edb828c05d95e4724ff9153b63a0824d3bb32184c4ec6bc5"
Jul 28 17:15:39.954564 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:39.954532    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-77f648d694-gjlbf"]
Jul 28 17:15:39.954920 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:39.954745    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c21b2db7-a694-46ee-8306-c23b69a28558" containerName="util"
Jul 28 17:15:39.954920 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:39.954755    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c21b2db7-a694-46ee-8306-c23b69a28558" containerName="util"
Jul 28 17:15:39.954920 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:39.954767    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c21b2db7-a694-46ee-8306-c23b69a28558" containerName="extract"
Jul 28 17:15:39.954920 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:39.954773    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c21b2db7-a694-46ee-8306-c23b69a28558" containerName="extract"
Jul 28 17:15:39.954920 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:39.954781    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c21b2db7-a694-46ee-8306-c23b69a28558" containerName="pull"
Jul 28 17:15:39.954920 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:39.954787    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c21b2db7-a694-46ee-8306-c23b69a28558" containerName="pull"
Jul 28 17:15:39.954920 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:39.954834    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c21b2db7-a694-46ee-8306-c23b69a28558" containerName="extract"
Jul 28 17:15:39.956892 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:39.956876    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-gjlbf"
Jul 28 17:15:39.960366 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:39.960334    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 28 17:15:39.960562 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:39.960480    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Jul 28 17:15:39.961953 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:39.961920    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 28 17:15:39.961953 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:39.961956    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 28 17:15:39.962184 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:39.961921    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 28 17:15:39.962254 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:39.962182    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-rjzkr\""
Jul 28 17:15:39.967765 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:39.967743    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-gjlbf"]
Jul 28 17:15:40.089292 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.089257    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7ps4\" (UniqueName: \"kubernetes.io/projected/630a192e-e9f6-478c-a0dc-7425b0e89436-kube-api-access-q7ps4\") pod \"keda-operator-77f648d694-gjlbf\" (UID: \"630a192e-e9f6-478c-a0dc-7425b0e89436\") " pod="openshift-keda/keda-operator-77f648d694-gjlbf"
Jul 28 17:15:40.089292 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.089298    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/630a192e-e9f6-478c-a0dc-7425b0e89436-certificates\") pod \"keda-operator-77f648d694-gjlbf\" (UID: \"630a192e-e9f6-478c-a0dc-7425b0e89436\") " pod="openshift-keda/keda-operator-77f648d694-gjlbf"
Jul 28 17:15:40.089538 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.089320    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/630a192e-e9f6-478c-a0dc-7425b0e89436-cabundle0\") pod \"keda-operator-77f648d694-gjlbf\" (UID: \"630a192e-e9f6-478c-a0dc-7425b0e89436\") " pod="openshift-keda/keda-operator-77f648d694-gjlbf"
Jul 28 17:15:40.190611 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.190582    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7ps4\" (UniqueName: \"kubernetes.io/projected/630a192e-e9f6-478c-a0dc-7425b0e89436-kube-api-access-q7ps4\") pod \"keda-operator-77f648d694-gjlbf\" (UID: \"630a192e-e9f6-478c-a0dc-7425b0e89436\") " pod="openshift-keda/keda-operator-77f648d694-gjlbf"
Jul 28 17:15:40.190777 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.190624    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/630a192e-e9f6-478c-a0dc-7425b0e89436-certificates\") pod \"keda-operator-77f648d694-gjlbf\" (UID: \"630a192e-e9f6-478c-a0dc-7425b0e89436\") " pod="openshift-keda/keda-operator-77f648d694-gjlbf"
Jul 28 17:15:40.190777 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.190648    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/630a192e-e9f6-478c-a0dc-7425b0e89436-cabundle0\") pod \"keda-operator-77f648d694-gjlbf\" (UID: \"630a192e-e9f6-478c-a0dc-7425b0e89436\") " pod="openshift-keda/keda-operator-77f648d694-gjlbf"
Jul 28 17:15:40.190777 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:15:40.190727    2567 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found
Jul 28 17:15:40.190777 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:15:40.190747    2567 secret.go:281] references non-existent secret key: ca.crt
Jul 28 17:15:40.190777 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:15:40.190755    2567 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 28 17:15:40.190777 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:15:40.190766    2567 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-77f648d694-gjlbf: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Jul 28 17:15:40.191072 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:15:40.190811    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/630a192e-e9f6-478c-a0dc-7425b0e89436-certificates podName:630a192e-e9f6-478c-a0dc-7425b0e89436 nodeName:}" failed. No retries permitted until 2026-07-28 17:15:40.690797679 +0000 UTC m=+295.936899864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/630a192e-e9f6-478c-a0dc-7425b0e89436-certificates") pod "keda-operator-77f648d694-gjlbf" (UID: "630a192e-e9f6-478c-a0dc-7425b0e89436") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Jul 28 17:15:40.191331 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.191308    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/630a192e-e9f6-478c-a0dc-7425b0e89436-cabundle0\") pod \"keda-operator-77f648d694-gjlbf\" (UID: \"630a192e-e9f6-478c-a0dc-7425b0e89436\") " pod="openshift-keda/keda-operator-77f648d694-gjlbf"
Jul 28 17:15:40.199942 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.199913    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7ps4\" (UniqueName: \"kubernetes.io/projected/630a192e-e9f6-478c-a0dc-7425b0e89436-kube-api-access-q7ps4\") pod \"keda-operator-77f648d694-gjlbf\" (UID: \"630a192e-e9f6-478c-a0dc-7425b0e89436\") " pod="openshift-keda/keda-operator-77f648d694-gjlbf"
Jul 28 17:15:40.447962 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.447931    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-rt8c7"]
Jul 28 17:15:40.449972 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.449955    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-rt8c7"
Jul 28 17:15:40.454478 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.454455    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Jul 28 17:15:40.466425 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.466403    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-rt8c7"]
Jul 28 17:15:40.594014 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.593988    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/458049b7-7c0c-4945-a89b-2e16b5384d08-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-rt8c7\" (UID: \"458049b7-7c0c-4945-a89b-2e16b5384d08\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-rt8c7"
Jul 28 17:15:40.594139 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.594016    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwhcc\" (UniqueName: \"kubernetes.io/projected/458049b7-7c0c-4945-a89b-2e16b5384d08-kube-api-access-nwhcc\") pod \"keda-metrics-apiserver-55557df8c-rt8c7\" (UID: \"458049b7-7c0c-4945-a89b-2e16b5384d08\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-rt8c7"
Jul 28 17:15:40.594139 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.594043    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/458049b7-7c0c-4945-a89b-2e16b5384d08-certificates\") pod \"keda-metrics-apiserver-55557df8c-rt8c7\" (UID: \"458049b7-7c0c-4945-a89b-2e16b5384d08\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-rt8c7"
Jul 28 17:15:40.694655 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.694629    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/458049b7-7c0c-4945-a89b-2e16b5384d08-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-rt8c7\" (UID: \"458049b7-7c0c-4945-a89b-2e16b5384d08\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-rt8c7"
Jul 28 17:15:40.694786 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.694659    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwhcc\" (UniqueName: \"kubernetes.io/projected/458049b7-7c0c-4945-a89b-2e16b5384d08-kube-api-access-nwhcc\") pod \"keda-metrics-apiserver-55557df8c-rt8c7\" (UID: \"458049b7-7c0c-4945-a89b-2e16b5384d08\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-rt8c7"
Jul 28 17:15:40.694786 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.694690    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/458049b7-7c0c-4945-a89b-2e16b5384d08-certificates\") pod \"keda-metrics-apiserver-55557df8c-rt8c7\" (UID: \"458049b7-7c0c-4945-a89b-2e16b5384d08\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-rt8c7"
Jul 28 17:15:40.694896 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:15:40.694797    2567 projected.go:266] Couldn't get secret openshift-keda/keda-metrics-apiserver-certs: secret "keda-metrics-apiserver-certs" not found
Jul 28 17:15:40.694896 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:15:40.694818    2567 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-55557df8c-rt8c7: secret "keda-metrics-apiserver-certs" not found
Jul 28 17:15:40.694896 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:15:40.694883    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/458049b7-7c0c-4945-a89b-2e16b5384d08-certificates podName:458049b7-7c0c-4945-a89b-2e16b5384d08 nodeName:}" failed. No retries permitted until 2026-07-28 17:15:41.194862625 +0000 UTC m=+296.440964826 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/458049b7-7c0c-4945-a89b-2e16b5384d08-certificates") pod "keda-metrics-apiserver-55557df8c-rt8c7" (UID: "458049b7-7c0c-4945-a89b-2e16b5384d08") : secret "keda-metrics-apiserver-certs" not found
Jul 28 17:15:40.695040 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.694815    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/630a192e-e9f6-478c-a0dc-7425b0e89436-certificates\") pod \"keda-operator-77f648d694-gjlbf\" (UID: \"630a192e-e9f6-478c-a0dc-7425b0e89436\") " pod="openshift-keda/keda-operator-77f648d694-gjlbf"
Jul 28 17:15:40.695085 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.695049    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/458049b7-7c0c-4945-a89b-2e16b5384d08-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-rt8c7\" (UID: \"458049b7-7c0c-4945-a89b-2e16b5384d08\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-rt8c7"
Jul 28 17:15:40.696991 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.696967    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/630a192e-e9f6-478c-a0dc-7425b0e89436-certificates\") pod \"keda-operator-77f648d694-gjlbf\" (UID: \"630a192e-e9f6-478c-a0dc-7425b0e89436\") " pod="openshift-keda/keda-operator-77f648d694-gjlbf"
Jul 28 17:15:40.704568 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.704499    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwhcc\" (UniqueName: \"kubernetes.io/projected/458049b7-7c0c-4945-a89b-2e16b5384d08-kube-api-access-nwhcc\") pod \"keda-metrics-apiserver-55557df8c-rt8c7\" (UID: \"458049b7-7c0c-4945-a89b-2e16b5384d08\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-rt8c7"
Jul 28 17:15:40.725877 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.725857    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-b5dz8"]
Jul 28 17:15:40.727850 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.727837    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-b5dz8"
Jul 28 17:15:40.731538 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.731522    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Jul 28 17:15:40.742519 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.742483    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-b5dz8"]
Jul 28 17:15:40.866408 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.866386    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-gjlbf"
Jul 28 17:15:40.897415 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.897388    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/9c98f809-8162-4034-bf2f-cad86700ea0e-certificates\") pod \"keda-admission-56f9798bb4-b5dz8\" (UID: \"9c98f809-8162-4034-bf2f-cad86700ea0e\") " pod="openshift-keda/keda-admission-56f9798bb4-b5dz8"
Jul 28 17:15:40.897528 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.897488    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqq8t\" (UniqueName: \"kubernetes.io/projected/9c98f809-8162-4034-bf2f-cad86700ea0e-kube-api-access-tqq8t\") pod \"keda-admission-56f9798bb4-b5dz8\" (UID: \"9c98f809-8162-4034-bf2f-cad86700ea0e\") " pod="openshift-keda/keda-admission-56f9798bb4-b5dz8"
Jul 28 17:15:40.985034 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.985010    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-gjlbf"]
Jul 28 17:15:40.998361 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.998341    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/9c98f809-8162-4034-bf2f-cad86700ea0e-certificates\") pod \"keda-admission-56f9798bb4-b5dz8\" (UID: \"9c98f809-8162-4034-bf2f-cad86700ea0e\") " pod="openshift-keda/keda-admission-56f9798bb4-b5dz8"
Jul 28 17:15:40.998458 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:40.998376    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqq8t\" (UniqueName: \"kubernetes.io/projected/9c98f809-8162-4034-bf2f-cad86700ea0e-kube-api-access-tqq8t\") pod \"keda-admission-56f9798bb4-b5dz8\" (UID: \"9c98f809-8162-4034-bf2f-cad86700ea0e\") " pod="openshift-keda/keda-admission-56f9798bb4-b5dz8"
Jul 28 17:15:40.998530 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:15:40.998498    2567 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found
Jul 28 17:15:40.998580 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:15:40.998541    2567 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-56f9798bb4-b5dz8: secret "keda-admission-webhooks-certs" not found
Jul 28 17:15:40.998618 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:15:40.998600    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c98f809-8162-4034-bf2f-cad86700ea0e-certificates podName:9c98f809-8162-4034-bf2f-cad86700ea0e nodeName:}" failed. No retries permitted until 2026-07-28 17:15:41.498580461 +0000 UTC m=+296.744682645 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/9c98f809-8162-4034-bf2f-cad86700ea0e-certificates") pod "keda-admission-56f9798bb4-b5dz8" (UID: "9c98f809-8162-4034-bf2f-cad86700ea0e") : secret "keda-admission-webhooks-certs" not found
Jul 28 17:15:41.007959 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:41.007938    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqq8t\" (UniqueName: \"kubernetes.io/projected/9c98f809-8162-4034-bf2f-cad86700ea0e-kube-api-access-tqq8t\") pod \"keda-admission-56f9798bb4-b5dz8\" (UID: \"9c98f809-8162-4034-bf2f-cad86700ea0e\") " pod="openshift-keda/keda-admission-56f9798bb4-b5dz8"
Jul 28 17:15:41.200258 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:41.200229    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/458049b7-7c0c-4945-a89b-2e16b5384d08-certificates\") pod \"keda-metrics-apiserver-55557df8c-rt8c7\" (UID: \"458049b7-7c0c-4945-a89b-2e16b5384d08\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-rt8c7"
Jul 28 17:15:41.202528 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:41.202495    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/458049b7-7c0c-4945-a89b-2e16b5384d08-certificates\") pod \"keda-metrics-apiserver-55557df8c-rt8c7\" (UID: \"458049b7-7c0c-4945-a89b-2e16b5384d08\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-rt8c7"
Jul 28 17:15:41.335576 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:41.335551    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-gjlbf" event={"ID":"630a192e-e9f6-478c-a0dc-7425b0e89436","Type":"ContainerStarted","Data":"b0eab2332acd057e467709062bb3359fd4ca2f6c8995c8ee07febbc3db68fb2f"}
Jul 28 17:15:41.359858 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:41.359839    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-rt8c7"
Jul 28 17:15:41.489365 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:41.489338    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-rt8c7"]
Jul 28 17:15:41.491261 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:15:41.491232    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod458049b7_7c0c_4945_a89b_2e16b5384d08.slice/crio-b48e58e2beecd9e603ccaad2a56191d19438f8a876d275d9e28c249ecc3a215d WatchSource:0}: Error finding container b48e58e2beecd9e603ccaad2a56191d19438f8a876d275d9e28c249ecc3a215d: Status 404 returned error can't find the container with id b48e58e2beecd9e603ccaad2a56191d19438f8a876d275d9e28c249ecc3a215d
Jul 28 17:15:41.502798 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:41.502776    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/9c98f809-8162-4034-bf2f-cad86700ea0e-certificates\") pod \"keda-admission-56f9798bb4-b5dz8\" (UID: \"9c98f809-8162-4034-bf2f-cad86700ea0e\") " pod="openshift-keda/keda-admission-56f9798bb4-b5dz8"
Jul 28 17:15:41.505104 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:41.505086    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/9c98f809-8162-4034-bf2f-cad86700ea0e-certificates\") pod \"keda-admission-56f9798bb4-b5dz8\" (UID: \"9c98f809-8162-4034-bf2f-cad86700ea0e\") " pod="openshift-keda/keda-admission-56f9798bb4-b5dz8"
Jul 28 17:15:41.637183 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:41.637128    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-b5dz8"
Jul 28 17:15:41.754360 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:41.754323    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-b5dz8"]
Jul 28 17:15:41.757669 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:15:41.757638    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c98f809_8162_4034_bf2f_cad86700ea0e.slice/crio-11ac2ae045733991061e81d0ace9e025389741decb26026f4b031b697a50725b WatchSource:0}: Error finding container 11ac2ae045733991061e81d0ace9e025389741decb26026f4b031b697a50725b: Status 404 returned error can't find the container with id 11ac2ae045733991061e81d0ace9e025389741decb26026f4b031b697a50725b
Jul 28 17:15:42.339629 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:42.339589    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-b5dz8" event={"ID":"9c98f809-8162-4034-bf2f-cad86700ea0e","Type":"ContainerStarted","Data":"11ac2ae045733991061e81d0ace9e025389741decb26026f4b031b697a50725b"}
Jul 28 17:15:42.340709 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:42.340687    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-rt8c7" event={"ID":"458049b7-7c0c-4945-a89b-2e16b5384d08","Type":"ContainerStarted","Data":"b48e58e2beecd9e603ccaad2a56191d19438f8a876d275d9e28c249ecc3a215d"}
Jul 28 17:15:45.919573 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:45.919546    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:15:45.920162 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:45.920131    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:15:45.923345 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:45.923326    2567 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 17:15:46.358717 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:46.358682    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-b5dz8" event={"ID":"9c98f809-8162-4034-bf2f-cad86700ea0e","Type":"ContainerStarted","Data":"d6cf9ae887d8cd935f276e6190c75bd2455ffd0454569cc2732abbce47789d32"}
Jul 28 17:15:46.358927 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:46.358771    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-56f9798bb4-b5dz8"
Jul 28 17:15:46.360114 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:46.360092    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-rt8c7" event={"ID":"458049b7-7c0c-4945-a89b-2e16b5384d08","Type":"ContainerStarted","Data":"2563e0aa8e891b3c9b09c1c1581962d070931ec8fd73c21ad2e185fac0b7151b"}
Jul 28 17:15:46.360225 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:46.360149    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-rt8c7"
Jul 28 17:15:46.361296 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:46.361275    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-gjlbf" event={"ID":"630a192e-e9f6-478c-a0dc-7425b0e89436","Type":"ContainerStarted","Data":"04bbe61717408557f385dcca538eeb9db7596754b2830d58e11c840fb2b32e8b"}
Jul 28 17:15:46.361400 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:46.361389    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-77f648d694-gjlbf"
Jul 28 17:15:46.378853 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:46.378815    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-56f9798bb4-b5dz8" podStartSLOduration=2.234223562 podStartE2EDuration="6.378803124s" totalImagesPullingTime="4.144579562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:15:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:15:46.378061551 +0000 UTC m=+301.624163757" watchObservedRunningTime="2026-07-28 17:15:46.378803124 +0000 UTC m=+301.624905330"
Jul 28 17:15:46.396459 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:46.396417    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-77f648d694-gjlbf" podStartSLOduration=2.471463284 podStartE2EDuration="7.396405138s" totalImagesPullingTime="4.924941854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:15:46.395222565 +0000 UTC m=+301.641324771" watchObservedRunningTime="2026-07-28 17:15:46.396405138 +0000 UTC m=+301.642507344"
Jul 28 17:15:46.412158 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:46.412117    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-55557df8c-rt8c7" podStartSLOduration=2.001617943 podStartE2EDuration="6.412101895s" totalImagesPullingTime="4.410483952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:15:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:15:46.411818037 +0000 UTC m=+301.657920256" watchObservedRunningTime="2026-07-28 17:15:46.412101895 +0000 UTC m=+301.658204103"
Jul 28 17:15:57.368731 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:15:57.368704    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-rt8c7"
Jul 28 17:16:07.366243 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:16:07.366214    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-77f648d694-gjlbf"
Jul 28 17:16:07.366688 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:16:07.366611    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-56f9798bb4-b5dz8"
Jul 28 17:16:57.903499 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:16:57.903466    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-545d98d4c7-4jhnx"]
Jul 28 17:16:57.906124 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:16:57.906104    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-545d98d4c7-4jhnx"
Jul 28 17:16:57.909371 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:16:57.909349    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 28 17:16:57.909484 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:16:57.909411    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 28 17:16:57.910931 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:16:57.910907    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Jul 28 17:16:57.911044 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:16:57.910928    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-7fvqm\""
Jul 28 17:16:57.918958 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:16:57.918939    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-545d98d4c7-4jhnx"]
Jul 28 17:16:58.007208 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:16:58.007186    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nk47\" (UniqueName: \"kubernetes.io/projected/286a752c-1b32-4e69-9d52-42daa1ba014d-kube-api-access-9nk47\") pod \"llmisvc-controller-manager-545d98d4c7-4jhnx\" (UID: \"286a752c-1b32-4e69-9d52-42daa1ba014d\") " pod="kserve/llmisvc-controller-manager-545d98d4c7-4jhnx"
Jul 28 17:16:58.007291 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:16:58.007214    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/286a752c-1b32-4e69-9d52-42daa1ba014d-cert\") pod \"llmisvc-controller-manager-545d98d4c7-4jhnx\" (UID: \"286a752c-1b32-4e69-9d52-42daa1ba014d\") " pod="kserve/llmisvc-controller-manager-545d98d4c7-4jhnx"
Jul 28 17:16:58.107741 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:16:58.107719    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nk47\" (UniqueName: \"kubernetes.io/projected/286a752c-1b32-4e69-9d52-42daa1ba014d-kube-api-access-9nk47\") pod \"llmisvc-controller-manager-545d98d4c7-4jhnx\" (UID: \"286a752c-1b32-4e69-9d52-42daa1ba014d\") " pod="kserve/llmisvc-controller-manager-545d98d4c7-4jhnx"
Jul 28 17:16:58.107819 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:16:58.107748    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/286a752c-1b32-4e69-9d52-42daa1ba014d-cert\") pod \"llmisvc-controller-manager-545d98d4c7-4jhnx\" (UID: \"286a752c-1b32-4e69-9d52-42daa1ba014d\") " pod="kserve/llmisvc-controller-manager-545d98d4c7-4jhnx"
Jul 28 17:16:58.109943 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:16:58.109916    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/286a752c-1b32-4e69-9d52-42daa1ba014d-cert\") pod \"llmisvc-controller-manager-545d98d4c7-4jhnx\" (UID: \"286a752c-1b32-4e69-9d52-42daa1ba014d\") " pod="kserve/llmisvc-controller-manager-545d98d4c7-4jhnx"
Jul 28 17:16:58.116030 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:16:58.116011    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nk47\" (UniqueName: \"kubernetes.io/projected/286a752c-1b32-4e69-9d52-42daa1ba014d-kube-api-access-9nk47\") pod \"llmisvc-controller-manager-545d98d4c7-4jhnx\" (UID: \"286a752c-1b32-4e69-9d52-42daa1ba014d\") " pod="kserve/llmisvc-controller-manager-545d98d4c7-4jhnx"
Jul 28 17:16:58.216345 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:16:58.216296    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-545d98d4c7-4jhnx"
Jul 28 17:16:58.336530 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:16:58.336493    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-545d98d4c7-4jhnx"]
Jul 28 17:16:58.338854 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:16:58.338825    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod286a752c_1b32_4e69_9d52_42daa1ba014d.slice/crio-a337982ede6f79c83cc76b3c0e6ad947ffb8af3c248347db2003168f4c3a1426 WatchSource:0}: Error finding container a337982ede6f79c83cc76b3c0e6ad947ffb8af3c248347db2003168f4c3a1426: Status 404 returned error can't find the container with id a337982ede6f79c83cc76b3c0e6ad947ffb8af3c248347db2003168f4c3a1426
Jul 28 17:16:58.340085 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:16:58.340070    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 17:16:58.604518 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:16:58.604479    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-545d98d4c7-4jhnx" event={"ID":"286a752c-1b32-4e69-9d52-42daa1ba014d","Type":"ContainerStarted","Data":"a337982ede6f79c83cc76b3c0e6ad947ffb8af3c248347db2003168f4c3a1426"}
Jul 28 17:17:02.619693 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:02.619662    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-545d98d4c7-4jhnx" event={"ID":"286a752c-1b32-4e69-9d52-42daa1ba014d","Type":"ContainerStarted","Data":"a3a39aea37518d76c6b5e01a3fcb6f4d9a141fe075a55dec959f758197bcbabb"}
Jul 28 17:17:02.620028 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:02.619789    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-545d98d4c7-4jhnx"
Jul 28 17:17:02.638544 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:02.638477    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-545d98d4c7-4jhnx" podStartSLOduration=2.392421333 podStartE2EDuration="5.638463393s" totalImagesPullingTime="3.24604206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:02.637622574 +0000 UTC m=+377.883724780" watchObservedRunningTime="2026-07-28 17:17:02.638463393 +0000 UTC m=+377.884565633"
Jul 28 17:17:33.056402 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:33.056365    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-778689875f-px9hf"]
Jul 28 17:17:33.625171 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:33.625136    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-545d98d4c7-4jhnx"
Jul 28 17:17:41.025333 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:41.025289    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl"]
Jul 28 17:17:41.028173 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:41.028152    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl"
Jul 28 17:17:41.031338 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:41.031304    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 17:17:41.031487 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:41.031318    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 17:17:41.032808 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:41.032787    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-22pvl\""
Jul 28 17:17:41.037563 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:41.037535    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl"]
Jul 28 17:17:41.122164 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:41.122116    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pshmp\" (UniqueName: \"kubernetes.io/projected/d2970347-0b69-4747-98dc-629d3198ecad-kube-api-access-pshmp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl\" (UID: \"d2970347-0b69-4747-98dc-629d3198ecad\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl"
Jul 28 17:17:41.122351 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:41.122181    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2970347-0b69-4747-98dc-629d3198ecad-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl\" (UID: \"d2970347-0b69-4747-98dc-629d3198ecad\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl"
Jul 28 17:17:41.122351 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:41.122215    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2970347-0b69-4747-98dc-629d3198ecad-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl\" (UID: \"d2970347-0b69-4747-98dc-629d3198ecad\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl"
Jul 28 17:17:41.223456 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:41.223422    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pshmp\" (UniqueName: \"kubernetes.io/projected/d2970347-0b69-4747-98dc-629d3198ecad-kube-api-access-pshmp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl\" (UID: \"d2970347-0b69-4747-98dc-629d3198ecad\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl"
Jul 28 17:17:41.223627 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:41.223472    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2970347-0b69-4747-98dc-629d3198ecad-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl\" (UID: \"d2970347-0b69-4747-98dc-629d3198ecad\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl"
Jul 28 17:17:41.223627 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:41.223490    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2970347-0b69-4747-98dc-629d3198ecad-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl\" (UID: \"d2970347-0b69-4747-98dc-629d3198ecad\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl"
Jul 28 17:17:41.223895 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:41.223871    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2970347-0b69-4747-98dc-629d3198ecad-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl\" (UID: \"d2970347-0b69-4747-98dc-629d3198ecad\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl"
Jul 28 17:17:41.223949 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:41.223879    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2970347-0b69-4747-98dc-629d3198ecad-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl\" (UID: \"d2970347-0b69-4747-98dc-629d3198ecad\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl"
Jul 28 17:17:41.232884 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:41.232853    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pshmp\" (UniqueName: \"kubernetes.io/projected/d2970347-0b69-4747-98dc-629d3198ecad-kube-api-access-pshmp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl\" (UID: \"d2970347-0b69-4747-98dc-629d3198ecad\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl"
Jul 28 17:17:41.338730 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:41.338637    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl"
Jul 28 17:17:41.476108 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:41.476066    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl"]
Jul 28 17:17:41.480360 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:17:41.480325    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2970347_0b69_4747_98dc_629d3198ecad.slice/crio-dc6fdc872568babbbdf037dffc1d40402b78c99c09d91e84be18c4b451d0b6af WatchSource:0}: Error finding container dc6fdc872568babbbdf037dffc1d40402b78c99c09d91e84be18c4b451d0b6af: Status 404 returned error can't find the container with id dc6fdc872568babbbdf037dffc1d40402b78c99c09d91e84be18c4b451d0b6af
Jul 28 17:17:41.749202 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:41.749102    2567 generic.go:356] "Generic (PLEG): container finished" podID="d2970347-0b69-4747-98dc-629d3198ecad" containerID="561065fa0eabf5996874b629aa3ce17ebabfca831977cc515ce2abac372c8247" exitCode=0
Jul 28 17:17:41.749202 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:41.749174    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl" event={"ID":"d2970347-0b69-4747-98dc-629d3198ecad","Type":"ContainerDied","Data":"561065fa0eabf5996874b629aa3ce17ebabfca831977cc515ce2abac372c8247"}
Jul 28 17:17:41.749430 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:41.749208    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl" event={"ID":"d2970347-0b69-4747-98dc-629d3198ecad","Type":"ContainerStarted","Data":"dc6fdc872568babbbdf037dffc1d40402b78c99c09d91e84be18c4b451d0b6af"}
Jul 28 17:17:44.765187 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:44.765146    2567 generic.go:356] "Generic (PLEG): container finished" podID="d2970347-0b69-4747-98dc-629d3198ecad" containerID="08958686a716e57a087f7007ed379ac67b4931050c0e4b78a9bd8313d777d156" exitCode=0
Jul 28 17:17:44.765187 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:44.765190    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl" event={"ID":"d2970347-0b69-4747-98dc-629d3198ecad","Type":"ContainerDied","Data":"08958686a716e57a087f7007ed379ac67b4931050c0e4b78a9bd8313d777d156"}
Jul 28 17:17:45.770962 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:45.770921    2567 generic.go:356] "Generic (PLEG): container finished" podID="d2970347-0b69-4747-98dc-629d3198ecad" containerID="8644b601fb1cbff2371d348588816491c544f6436df80df7d9329d844cabe515" exitCode=0
Jul 28 17:17:45.771319 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:45.771015    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl" event={"ID":"d2970347-0b69-4747-98dc-629d3198ecad","Type":"ContainerDied","Data":"8644b601fb1cbff2371d348588816491c544f6436df80df7d9329d844cabe515"}
Jul 28 17:17:46.909368 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:46.909330    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl"
Jul 28 17:17:46.973948 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:46.973903    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2970347-0b69-4747-98dc-629d3198ecad-util\") pod \"d2970347-0b69-4747-98dc-629d3198ecad\" (UID: \"d2970347-0b69-4747-98dc-629d3198ecad\") "
Jul 28 17:17:46.974140 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:46.973969    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pshmp\" (UniqueName: \"kubernetes.io/projected/d2970347-0b69-4747-98dc-629d3198ecad-kube-api-access-pshmp\") pod \"d2970347-0b69-4747-98dc-629d3198ecad\" (UID: \"d2970347-0b69-4747-98dc-629d3198ecad\") "
Jul 28 17:17:46.974140 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:46.974006    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2970347-0b69-4747-98dc-629d3198ecad-bundle\") pod \"d2970347-0b69-4747-98dc-629d3198ecad\" (UID: \"d2970347-0b69-4747-98dc-629d3198ecad\") "
Jul 28 17:17:46.974768 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:46.974739    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2970347-0b69-4747-98dc-629d3198ecad-bundle" (OuterVolumeSpecName: "bundle") pod "d2970347-0b69-4747-98dc-629d3198ecad" (UID: "d2970347-0b69-4747-98dc-629d3198ecad"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:17:46.976152 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:46.976110    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2970347-0b69-4747-98dc-629d3198ecad-kube-api-access-pshmp" (OuterVolumeSpecName: "kube-api-access-pshmp") pod "d2970347-0b69-4747-98dc-629d3198ecad" (UID: "d2970347-0b69-4747-98dc-629d3198ecad"). InnerVolumeSpecName "kube-api-access-pshmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:17:47.000129 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:47.000065    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2970347-0b69-4747-98dc-629d3198ecad-util" (OuterVolumeSpecName: "util") pod "d2970347-0b69-4747-98dc-629d3198ecad" (UID: "d2970347-0b69-4747-98dc-629d3198ecad"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:17:47.074745 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:47.074662    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2970347-0b69-4747-98dc-629d3198ecad-util\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:17:47.074745 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:47.074692    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pshmp\" (UniqueName: \"kubernetes.io/projected/d2970347-0b69-4747-98dc-629d3198ecad-kube-api-access-pshmp\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:17:47.074745 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:47.074703    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2970347-0b69-4747-98dc-629d3198ecad-bundle\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:17:47.783868 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:47.783829    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl" event={"ID":"d2970347-0b69-4747-98dc-629d3198ecad","Type":"ContainerDied","Data":"dc6fdc872568babbbdf037dffc1d40402b78c99c09d91e84be18c4b451d0b6af"}
Jul 28 17:17:47.784047 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:47.783875    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jjvsl"
Jul 28 17:17:47.784047 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:47.783904    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc6fdc872568babbbdf037dffc1d40402b78c99c09d91e84be18c4b451d0b6af"
Jul 28 17:17:58.075372 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.075324    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-778689875f-px9hf" podUID="e1c742d2-d065-434d-8f21-f72fbb37b6a8" containerName="console" containerID="cri-o://61be3886d6c697a605406bd5be8908f3c7252130a60e098ff49106ee1aa73080" gracePeriod=15
Jul 28 17:17:58.324881 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.324856    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:17:58.461676 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.461564    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-service-ca\") pod \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") "
Jul 28 17:17:58.461676 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.461615    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-trusted-ca-bundle\") pod \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") "
Jul 28 17:17:58.461676 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.461648    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-console-config\") pod \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") "
Jul 28 17:17:58.461676 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.461664    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1c742d2-d065-434d-8f21-f72fbb37b6a8-console-serving-cert\") pod \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") "
Jul 28 17:17:58.461996 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.461693    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-oauth-serving-cert\") pod \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") "
Jul 28 17:17:58.461996 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.461720    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1c742d2-d065-434d-8f21-f72fbb37b6a8-console-oauth-config\") pod \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") "
Jul 28 17:17:58.461996 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.461749    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdzkl\" (UniqueName: \"kubernetes.io/projected/e1c742d2-d065-434d-8f21-f72fbb37b6a8-kube-api-access-bdzkl\") pod \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\" (UID: \"e1c742d2-d065-434d-8f21-f72fbb37b6a8\") "
Jul 28 17:17:58.462152 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.462128    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e1c742d2-d065-434d-8f21-f72fbb37b6a8" (UID: "e1c742d2-d065-434d-8f21-f72fbb37b6a8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:17:58.462205 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.462161    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-console-config" (OuterVolumeSpecName: "console-config") pod "e1c742d2-d065-434d-8f21-f72fbb37b6a8" (UID: "e1c742d2-d065-434d-8f21-f72fbb37b6a8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:17:58.462205 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.462176    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e1c742d2-d065-434d-8f21-f72fbb37b6a8" (UID: "e1c742d2-d065-434d-8f21-f72fbb37b6a8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:17:58.462205 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.462186    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-service-ca" (OuterVolumeSpecName: "service-ca") pod "e1c742d2-d065-434d-8f21-f72fbb37b6a8" (UID: "e1c742d2-d065-434d-8f21-f72fbb37b6a8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:17:58.463878 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.463844    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1c742d2-d065-434d-8f21-f72fbb37b6a8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e1c742d2-d065-434d-8f21-f72fbb37b6a8" (UID: "e1c742d2-d065-434d-8f21-f72fbb37b6a8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:17:58.463878 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.463860    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1c742d2-d065-434d-8f21-f72fbb37b6a8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e1c742d2-d065-434d-8f21-f72fbb37b6a8" (UID: "e1c742d2-d065-434d-8f21-f72fbb37b6a8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:17:58.464002 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.463881    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1c742d2-d065-434d-8f21-f72fbb37b6a8-kube-api-access-bdzkl" (OuterVolumeSpecName: "kube-api-access-bdzkl") pod "e1c742d2-d065-434d-8f21-f72fbb37b6a8" (UID: "e1c742d2-d065-434d-8f21-f72fbb37b6a8"). InnerVolumeSpecName "kube-api-access-bdzkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:17:58.562755 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.562703    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-console-config\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:17:58.562755 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.562747    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1c742d2-d065-434d-8f21-f72fbb37b6a8-console-serving-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:17:58.562755 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.562757    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-oauth-serving-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:17:58.562755 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.562765    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1c742d2-d065-434d-8f21-f72fbb37b6a8-console-oauth-config\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:17:58.562755 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.562774    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdzkl\" (UniqueName: \"kubernetes.io/projected/e1c742d2-d065-434d-8f21-f72fbb37b6a8-kube-api-access-bdzkl\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:17:58.563039 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.562783    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-service-ca\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:17:58.563039 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.562790    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1c742d2-d065-434d-8f21-f72fbb37b6a8-trusted-ca-bundle\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:17:58.824776 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.824746    2567 generic.go:356] "Generic (PLEG): container finished" podID="e1c742d2-d065-434d-8f21-f72fbb37b6a8" containerID="61be3886d6c697a605406bd5be8908f3c7252130a60e098ff49106ee1aa73080" exitCode=0
Jul 28 17:17:58.824938 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.824790    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-778689875f-px9hf"
Jul 28 17:17:58.824938 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.824802    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-778689875f-px9hf" event={"ID":"e1c742d2-d065-434d-8f21-f72fbb37b6a8","Type":"ContainerDied","Data":"61be3886d6c697a605406bd5be8908f3c7252130a60e098ff49106ee1aa73080"}
Jul 28 17:17:58.824938 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.824843    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-778689875f-px9hf" event={"ID":"e1c742d2-d065-434d-8f21-f72fbb37b6a8","Type":"ContainerDied","Data":"af912c579d14ca0a87dea240adabb31a918c92a6cf584e51d7276f92b3646c4c"}
Jul 28 17:17:58.824938 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.824868    2567 scope.go:117] "RemoveContainer" containerID="61be3886d6c697a605406bd5be8908f3c7252130a60e098ff49106ee1aa73080"
Jul 28 17:17:58.833484 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.833467    2567 scope.go:117] "RemoveContainer" containerID="61be3886d6c697a605406bd5be8908f3c7252130a60e098ff49106ee1aa73080"
Jul 28 17:17:58.833749 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:17:58.833727    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61be3886d6c697a605406bd5be8908f3c7252130a60e098ff49106ee1aa73080\": container with ID starting with 61be3886d6c697a605406bd5be8908f3c7252130a60e098ff49106ee1aa73080 not found: ID does not exist" containerID="61be3886d6c697a605406bd5be8908f3c7252130a60e098ff49106ee1aa73080"
Jul 28 17:17:58.833809 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.833758    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61be3886d6c697a605406bd5be8908f3c7252130a60e098ff49106ee1aa73080"} err="failed to get container status \"61be3886d6c697a605406bd5be8908f3c7252130a60e098ff49106ee1aa73080\": rpc error: code = NotFound desc = could not find container \"61be3886d6c697a605406bd5be8908f3c7252130a60e098ff49106ee1aa73080\": container with ID starting with 61be3886d6c697a605406bd5be8908f3c7252130a60e098ff49106ee1aa73080 not found: ID does not exist"
Jul 28 17:17:58.849604 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.849564    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-778689875f-px9hf"]
Jul 28 17:17:58.851724 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:58.851695    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-778689875f-px9hf"]
Jul 28 17:17:59.286230 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:17:59.286192    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1c742d2-d065-434d-8f21-f72fbb37b6a8" path="/var/lib/kubelet/pods/e1c742d2-d065-434d-8f21-f72fbb37b6a8/volumes"
Jul 28 17:18:03.157241 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.157202    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb"]
Jul 28 17:18:03.157663 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.157464    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1c742d2-d065-434d-8f21-f72fbb37b6a8" containerName="console"
Jul 28 17:18:03.157663 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.157479    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1c742d2-d065-434d-8f21-f72fbb37b6a8" containerName="console"
Jul 28 17:18:03.157663 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.157493    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2970347-0b69-4747-98dc-629d3198ecad" containerName="extract"
Jul 28 17:18:03.157663 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.157498    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2970347-0b69-4747-98dc-629d3198ecad" containerName="extract"
Jul 28 17:18:03.157663 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.157524    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2970347-0b69-4747-98dc-629d3198ecad" containerName="util"
Jul 28 17:18:03.157663 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.157530    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2970347-0b69-4747-98dc-629d3198ecad" containerName="util"
Jul 28 17:18:03.157663 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.157535    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2970347-0b69-4747-98dc-629d3198ecad" containerName="pull"
Jul 28 17:18:03.157663 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.157541    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2970347-0b69-4747-98dc-629d3198ecad" containerName="pull"
Jul 28 17:18:03.157663 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.157584    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1c742d2-d065-434d-8f21-f72fbb37b6a8" containerName="console"
Jul 28 17:18:03.157663 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.157591    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2970347-0b69-4747-98dc-629d3198ecad" containerName="extract"
Jul 28 17:18:03.162620 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.162601    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb"
Jul 28 17:18:03.165615 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.165591    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 17:18:03.167000 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.166958    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 17:18:03.167000 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.166976    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-22pvl\""
Jul 28 17:18:03.171704 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.171680    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb"]
Jul 28 17:18:03.296624 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.296585    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb\" (UID: \"bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb"
Jul 28 17:18:03.296802 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.296635    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb\" (UID: \"bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb"
Jul 28 17:18:03.296802 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.296693    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvjf8\" (UniqueName: \"kubernetes.io/projected/bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3-kube-api-access-pvjf8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb\" (UID: \"bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb"
Jul 28 17:18:03.397102 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.397058    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb\" (UID: \"bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb"
Jul 28 17:18:03.397270 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.397125    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb\" (UID: \"bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb"
Jul 28 17:18:03.397270 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.397150    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvjf8\" (UniqueName: \"kubernetes.io/projected/bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3-kube-api-access-pvjf8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb\" (UID: \"bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb"
Jul 28 17:18:03.397541 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.397490    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb\" (UID: \"bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb"
Jul 28 17:18:03.397618 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.397560    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb\" (UID: \"bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb"
Jul 28 17:18:03.408227 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.408147    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvjf8\" (UniqueName: \"kubernetes.io/projected/bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3-kube-api-access-pvjf8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb\" (UID: \"bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb"
Jul 28 17:18:03.472005 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.471962    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb"
Jul 28 17:18:03.621853 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.621776    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb"]
Jul 28 17:18:03.624766 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:18:03.624726    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfeb46c5_6d4b_4e1a_b004_6adefcd0b4f3.slice/crio-b8992c838c43a17af1ec2e8220d69acf397c0c22ce4418cd170cf6cc7bb2e85e WatchSource:0}: Error finding container b8992c838c43a17af1ec2e8220d69acf397c0c22ce4418cd170cf6cc7bb2e85e: Status 404 returned error can't find the container with id b8992c838c43a17af1ec2e8220d69acf397c0c22ce4418cd170cf6cc7bb2e85e
Jul 28 17:18:03.847925 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.847890    2567 generic.go:356] "Generic (PLEG): container finished" podID="bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3" containerID="bdc2f462455661a59d0361f681ea7390cada1e0ec8ff2505ada6aef6307c0fb1" exitCode=0
Jul 28 17:18:03.848212 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.847987    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb" event={"ID":"bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3","Type":"ContainerDied","Data":"bdc2f462455661a59d0361f681ea7390cada1e0ec8ff2505ada6aef6307c0fb1"}
Jul 28 17:18:03.848212 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:03.848052    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb" event={"ID":"bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3","Type":"ContainerStarted","Data":"b8992c838c43a17af1ec2e8220d69acf397c0c22ce4418cd170cf6cc7bb2e85e"}
Jul 28 17:18:06.862874 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:06.862791    2567 generic.go:356] "Generic (PLEG): container finished" podID="bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3" containerID="e31a568c1312c5a14dc3e82da715c2052ddb3cf23f7e8040f4c4bf2b50561127" exitCode=0
Jul 28 17:18:06.863208 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:06.862881    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb" event={"ID":"bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3","Type":"ContainerDied","Data":"e31a568c1312c5a14dc3e82da715c2052ddb3cf23f7e8040f4c4bf2b50561127"}
Jul 28 17:18:07.868739 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:07.868707    2567 generic.go:356] "Generic (PLEG): container finished" podID="bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3" containerID="00ca4a91abb4a879e2307e056e1b917f35e4c0a6b6b31e5810c23f7a131c6d20" exitCode=0
Jul 28 17:18:07.869135 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:07.868780    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb" event={"ID":"bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3","Type":"ContainerDied","Data":"00ca4a91abb4a879e2307e056e1b917f35e4c0a6b6b31e5810c23f7a131c6d20"}
Jul 28 17:18:08.987153 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:08.987131    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb"
Jul 28 17:18:09.140594 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:09.140478    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3-bundle\") pod \"bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3\" (UID: \"bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3\") "
Jul 28 17:18:09.140594 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:09.140553    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3-util\") pod \"bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3\" (UID: \"bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3\") "
Jul 28 17:18:09.140836 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:09.140642    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvjf8\" (UniqueName: \"kubernetes.io/projected/bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3-kube-api-access-pvjf8\") pod \"bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3\" (UID: \"bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3\") "
Jul 28 17:18:09.140962 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:09.140941    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3-bundle" (OuterVolumeSpecName: "bundle") pod "bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3" (UID: "bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:18:09.142543 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:09.142473    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3-kube-api-access-pvjf8" (OuterVolumeSpecName: "kube-api-access-pvjf8") pod "bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3" (UID: "bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3"). InnerVolumeSpecName "kube-api-access-pvjf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:18:09.145909 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:09.145886    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3-util" (OuterVolumeSpecName: "util") pod "bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3" (UID: "bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:18:09.241991 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:09.241940    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3-bundle\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:18:09.241991 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:09.241981    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3-util\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:18:09.241991 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:09.241990    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvjf8\" (UniqueName: \"kubernetes.io/projected/bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3-kube-api-access-pvjf8\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:18:09.877609 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:09.877580    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb"
Jul 28 17:18:09.877851 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:09.877574    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq8xtb" event={"ID":"bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3","Type":"ContainerDied","Data":"b8992c838c43a17af1ec2e8220d69acf397c0c22ce4418cd170cf6cc7bb2e85e"}
Jul 28 17:18:09.877851 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:09.877701    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b8992c838c43a17af1ec2e8220d69acf397c0c22ce4418cd170cf6cc7bb2e85e"
Jul 28 17:18:30.948398 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:30.948364    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"]
Jul 28 17:18:30.948771 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:30.948650    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3" containerName="pull"
Jul 28 17:18:30.948771 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:30.948663    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3" containerName="pull"
Jul 28 17:18:30.948771 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:30.948678    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3" containerName="util"
Jul 28 17:18:30.948771 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:30.948683    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3" containerName="util"
Jul 28 17:18:30.948771 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:30.948690    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3" containerName="extract"
Jul 28 17:18:30.948771 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:30.948696    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3" containerName="extract"
Jul 28 17:18:30.948771 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:30.948741    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfeb46c5-6d4b-4e1a-b004-6adefcd0b4f3" containerName="extract"
Jul 28 17:18:30.954447 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:30.954428    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:30.957821 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:30.957794    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 17:18:30.957821 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:30.957808    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 28 17:18:30.957986 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:30.957815    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 17:18:30.957986 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:30.957794    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 28 17:18:30.957986 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:30.957799    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 28 17:18:30.958261 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:30.958248    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 17:18:30.971467 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:30.971442    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-lrhhd\""
Jul 28 17:18:30.973430 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:30.973402    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"]
Jul 28 17:18:31.088023 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.087988    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54a9bb44-0a50-48f3-a005-f9f3fc84c9e6-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-x4v6g\" (UID: \"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.088192 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.088031    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/54a9bb44-0a50-48f3-a005-f9f3fc84c9e6-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-x4v6g\" (UID: \"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.088192 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.088052    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/54a9bb44-0a50-48f3-a005-f9f3fc84c9e6-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-x4v6g\" (UID: \"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.088192 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.088145    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/54a9bb44-0a50-48f3-a005-f9f3fc84c9e6-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-x4v6g\" (UID: \"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.088192 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.088187    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/54a9bb44-0a50-48f3-a005-f9f3fc84c9e6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-x4v6g\" (UID: \"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.088322 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.088216    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/54a9bb44-0a50-48f3-a005-f9f3fc84c9e6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-x4v6g\" (UID: \"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.088322 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.088250    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t276\" (UniqueName: \"kubernetes.io/projected/54a9bb44-0a50-48f3-a005-f9f3fc84c9e6-kube-api-access-7t276\") pod \"istiod-openshift-gateway-5677ccb49b-x4v6g\" (UID: \"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.189313 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.189255    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/54a9bb44-0a50-48f3-a005-f9f3fc84c9e6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-x4v6g\" (UID: \"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.189313 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.189316    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t276\" (UniqueName: \"kubernetes.io/projected/54a9bb44-0a50-48f3-a005-f9f3fc84c9e6-kube-api-access-7t276\") pod \"istiod-openshift-gateway-5677ccb49b-x4v6g\" (UID: \"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.189583 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.189341    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54a9bb44-0a50-48f3-a005-f9f3fc84c9e6-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-x4v6g\" (UID: \"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.189583 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.189367    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/54a9bb44-0a50-48f3-a005-f9f3fc84c9e6-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-x4v6g\" (UID: \"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.189583 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.189385    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/54a9bb44-0a50-48f3-a005-f9f3fc84c9e6-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-x4v6g\" (UID: \"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.189583 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.189427    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/54a9bb44-0a50-48f3-a005-f9f3fc84c9e6-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-x4v6g\" (UID: \"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.189583 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.189461    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/54a9bb44-0a50-48f3-a005-f9f3fc84c9e6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-x4v6g\" (UID: \"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.190263 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.190236    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/54a9bb44-0a50-48f3-a005-f9f3fc84c9e6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-x4v6g\" (UID: \"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.191811 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.191785    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/54a9bb44-0a50-48f3-a005-f9f3fc84c9e6-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-x4v6g\" (UID: \"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.191811 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.191796    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/54a9bb44-0a50-48f3-a005-f9f3fc84c9e6-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-x4v6g\" (UID: \"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.191929 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.191786    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/54a9bb44-0a50-48f3-a005-f9f3fc84c9e6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-x4v6g\" (UID: \"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.191929 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.191857    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/54a9bb44-0a50-48f3-a005-f9f3fc84c9e6-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-x4v6g\" (UID: \"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.197688 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.197667    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54a9bb44-0a50-48f3-a005-f9f3fc84c9e6-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-x4v6g\" (UID: \"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.197817 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.197792    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t276\" (UniqueName: \"kubernetes.io/projected/54a9bb44-0a50-48f3-a005-f9f3fc84c9e6-kube-api-access-7t276\") pod \"istiod-openshift-gateway-5677ccb49b-x4v6g\" (UID: \"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.263753 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.263723    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:31.401044 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.401020    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"]
Jul 28 17:18:31.402995 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:18:31.402969    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54a9bb44_0a50_48f3_a005_f9f3fc84c9e6.slice/crio-0714a8e445a22a92c446909deae09231d811a093aa0d87d7e4e68b58b4ed28a3 WatchSource:0}: Error finding container 0714a8e445a22a92c446909deae09231d811a093aa0d87d7e4e68b58b4ed28a3: Status 404 returned error can't find the container with id 0714a8e445a22a92c446909deae09231d811a093aa0d87d7e4e68b58b4ed28a3
Jul 28 17:18:31.953548 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:31.953501    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g" event={"ID":"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6","Type":"ContainerStarted","Data":"0714a8e445a22a92c446909deae09231d811a093aa0d87d7e4e68b58b4ed28a3"}
Jul 28 17:18:33.762044 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:33.762005    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 17:18:33.762331 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:33.762073    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 17:18:33.963754 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:33.963716    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g" event={"ID":"54a9bb44-0a50-48f3-a005-f9f3fc84c9e6","Type":"ContainerStarted","Data":"c9dcdfee26d1f3615394617abd91297e1506be3923f735308c8bf7e5f31c9336"}
Jul 28 17:18:33.963907 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:33.963836    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:33.987176 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:33.987122    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g" podStartSLOduration=1.6301684440000002 podStartE2EDuration="3.987105414s" totalImagesPullingTime="2.35693697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:33.984718953 +0000 UTC m=+469.230821159" watchObservedRunningTime="2026-07-28 17:18:33.987105414 +0000 UTC m=+469.233207619"
Jul 28 17:18:34.969677 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:34.969643    2567 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-x4v6g container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 17:18:34.970165 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:34.969702    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g" podUID="54a9bb44-0a50-48f3-a005-f9f3fc84c9e6" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 17:18:37.968902 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:37.968869    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x4v6g"
Jul 28 17:18:38.443759 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.443729    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"]
Jul 28 17:18:38.446158 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.446138    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.452714 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.452531    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-default-dockercfg-5kmqk\""
Jul 28 17:18:38.480740 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.480709    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"]
Jul 28 17:18:38.546730 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.546692    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be331145-6d91-44a5-aa68-247f0ab1262b-workload-certs\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.546730 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.546729    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be331145-6d91-44a5-aa68-247f0ab1262b-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.546927 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.546765    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be331145-6d91-44a5-aa68-247f0ab1262b-istio-data\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.546927 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.546815    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be331145-6d91-44a5-aa68-247f0ab1262b-credential-socket\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.546927 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.546848    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9fqf\" (UniqueName: \"kubernetes.io/projected/be331145-6d91-44a5-aa68-247f0ab1262b-kube-api-access-r9fqf\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.546927 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.546876    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be331145-6d91-44a5-aa68-247f0ab1262b-istio-envoy\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.546927 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.546895    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be331145-6d91-44a5-aa68-247f0ab1262b-istio-token\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.546927 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.546922    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be331145-6d91-44a5-aa68-247f0ab1262b-workload-socket\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.547129 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.547008    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be331145-6d91-44a5-aa68-247f0ab1262b-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.647482 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.647444    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be331145-6d91-44a5-aa68-247f0ab1262b-istio-envoy\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.647482 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.647485    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be331145-6d91-44a5-aa68-247f0ab1262b-istio-token\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.647722 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.647540    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be331145-6d91-44a5-aa68-247f0ab1262b-workload-socket\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.647722 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.647574    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be331145-6d91-44a5-aa68-247f0ab1262b-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.647722 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.647599    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be331145-6d91-44a5-aa68-247f0ab1262b-workload-certs\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.647722 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.647622    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be331145-6d91-44a5-aa68-247f0ab1262b-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.647722 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.647660    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be331145-6d91-44a5-aa68-247f0ab1262b-istio-data\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.647722 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.647720    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be331145-6d91-44a5-aa68-247f0ab1262b-credential-socket\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.647997 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.647815    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9fqf\" (UniqueName: \"kubernetes.io/projected/be331145-6d91-44a5-aa68-247f0ab1262b-kube-api-access-r9fqf\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.648054 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.648012    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be331145-6d91-44a5-aa68-247f0ab1262b-workload-certs\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.648137 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.648107    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be331145-6d91-44a5-aa68-247f0ab1262b-istio-data\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.648274 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.648210    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be331145-6d91-44a5-aa68-247f0ab1262b-credential-socket\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.648274 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.648254    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be331145-6d91-44a5-aa68-247f0ab1262b-workload-socket\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.648414 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.648390    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be331145-6d91-44a5-aa68-247f0ab1262b-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.649933 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.649910    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be331145-6d91-44a5-aa68-247f0ab1262b-istio-envoy\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.650013 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.649979    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be331145-6d91-44a5-aa68-247f0ab1262b-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.662987 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.662962    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9fqf\" (UniqueName: \"kubernetes.io/projected/be331145-6d91-44a5-aa68-247f0ab1262b-kube-api-access-r9fqf\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.664483 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.664459    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be331145-6d91-44a5-aa68-247f0ab1262b-istio-token\") pod \"openshift-ai-inference-openshift-default-8699b59bbc-nvf9s\" (UID: \"be331145-6d91-44a5-aa68-247f0ab1262b\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.756271 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.756179    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:38.886574 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.886549    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"]
Jul 28 17:18:38.888488 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:18:38.888461    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe331145_6d91_44a5_aa68_247f0ab1262b.slice/crio-1a4329b3d63da1b568803af19971cf3ec3cb23e36d81bd64fcf1801af0388d24 WatchSource:0}: Error finding container 1a4329b3d63da1b568803af19971cf3ec3cb23e36d81bd64fcf1801af0388d24: Status 404 returned error can't find the container with id 1a4329b3d63da1b568803af19971cf3ec3cb23e36d81bd64fcf1801af0388d24
Jul 28 17:18:38.982743 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:38.982711    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s" event={"ID":"be331145-6d91-44a5-aa68-247f0ab1262b","Type":"ContainerStarted","Data":"1a4329b3d63da1b568803af19971cf3ec3cb23e36d81bd64fcf1801af0388d24"}
Jul 28 17:18:43.271814 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:43.271755    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 17:18:43.272192 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:43.271871    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 17:18:43.272192 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:43.271921    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 17:18:44.003898 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:44.003862    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s" event={"ID":"be331145-6d91-44a5-aa68-247f0ab1262b","Type":"ContainerStarted","Data":"20695c6d5dfe34149fafbd719b65165de7feacab5170c94a1a10c59410b43f29"}
Jul 28 17:18:44.028141 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:44.028082    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s" podStartSLOduration=1.647048544 podStartE2EDuration="6.02806652s" totalImagesPullingTime="4.381017976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:44.025721736 +0000 UTC m=+479.271823943" watchObservedRunningTime="2026-07-28 17:18:44.02806652 +0000 UTC m=+479.274168725"
Jul 28 17:18:44.756878 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:44.756833    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:44.762069 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:44.762039    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:45.007752 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:45.007722    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:45.008809 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:45.008788    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:18:48.042586 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.042548    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf"]
Jul 28 17:18:48.047285 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.047259    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf"
Jul 28 17:18:48.050437 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.050400    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 17:18:48.050607 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.050489    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 17:18:48.050657 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.050615    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-22pvl\""
Jul 28 17:18:48.055565 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.055538    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf"]
Jul 28 17:18:48.123670 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.123624    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7ff6796-b305-4ab4-879b-24332ca96f8c-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf\" (UID: \"d7ff6796-b305-4ab4-879b-24332ca96f8c\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf"
Jul 28 17:18:48.123856 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.123724    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj9nm\" (UniqueName: \"kubernetes.io/projected/d7ff6796-b305-4ab4-879b-24332ca96f8c-kube-api-access-cj9nm\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf\" (UID: \"d7ff6796-b305-4ab4-879b-24332ca96f8c\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf"
Jul 28 17:18:48.123856 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.123779    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7ff6796-b305-4ab4-879b-24332ca96f8c-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf\" (UID: \"d7ff6796-b305-4ab4-879b-24332ca96f8c\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf"
Jul 28 17:18:48.142747 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.142705    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg"]
Jul 28 17:18:48.146067 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.146047    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg"
Jul 28 17:18:48.154954 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.154919    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg"]
Jul 28 17:18:48.224936 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.224887    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg\" (UID: \"35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg"
Jul 28 17:18:48.224936 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.224937    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7ff6796-b305-4ab4-879b-24332ca96f8c-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf\" (UID: \"d7ff6796-b305-4ab4-879b-24332ca96f8c\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf"
Jul 28 17:18:48.225155 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.224983    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7ff6796-b305-4ab4-879b-24332ca96f8c-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf\" (UID: \"d7ff6796-b305-4ab4-879b-24332ca96f8c\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf"
Jul 28 17:18:48.225155 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.225143    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zt5k\" (UniqueName: \"kubernetes.io/projected/35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd-kube-api-access-2zt5k\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg\" (UID: \"35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg"
Jul 28 17:18:48.225226 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.225204    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj9nm\" (UniqueName: \"kubernetes.io/projected/d7ff6796-b305-4ab4-879b-24332ca96f8c-kube-api-access-cj9nm\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf\" (UID: \"d7ff6796-b305-4ab4-879b-24332ca96f8c\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf"
Jul 28 17:18:48.225275 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.225224    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg\" (UID: \"35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg"
Jul 28 17:18:48.225351 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.225323    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7ff6796-b305-4ab4-879b-24332ca96f8c-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf\" (UID: \"d7ff6796-b305-4ab4-879b-24332ca96f8c\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf"
Jul 28 17:18:48.225479 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.225413    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7ff6796-b305-4ab4-879b-24332ca96f8c-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf\" (UID: \"d7ff6796-b305-4ab4-879b-24332ca96f8c\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf"
Jul 28 17:18:48.235937 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.235900    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj9nm\" (UniqueName: \"kubernetes.io/projected/d7ff6796-b305-4ab4-879b-24332ca96f8c-kube-api-access-cj9nm\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf\" (UID: \"d7ff6796-b305-4ab4-879b-24332ca96f8c\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf"
Jul 28 17:18:48.241898 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.241835    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv"]
Jul 28 17:18:48.245362 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.245333    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv"
Jul 28 17:18:48.254044 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.254016    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv"]
Jul 28 17:18:48.326318 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.326220    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg\" (UID: \"35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg"
Jul 28 17:18:48.326318 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.326268    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfe19b1b-8b03-48b2-825a-785bd2a1b50b-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv\" (UID: \"dfe19b1b-8b03-48b2-825a-785bd2a1b50b\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv"
Jul 28 17:18:48.326551 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.326386    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zt5k\" (UniqueName: \"kubernetes.io/projected/35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd-kube-api-access-2zt5k\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg\" (UID: \"35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg"
Jul 28 17:18:48.326551 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.326405    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfe19b1b-8b03-48b2-825a-785bd2a1b50b-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv\" (UID: \"dfe19b1b-8b03-48b2-825a-785bd2a1b50b\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv"
Jul 28 17:18:48.326551 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.326527    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cn7h\" (UniqueName: \"kubernetes.io/projected/dfe19b1b-8b03-48b2-825a-785bd2a1b50b-kube-api-access-8cn7h\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv\" (UID: \"dfe19b1b-8b03-48b2-825a-785bd2a1b50b\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv"
Jul 28 17:18:48.326684 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.326558    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg\" (UID: \"35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg"
Jul 28 17:18:48.326748 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.326714    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg\" (UID: \"35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg"
Jul 28 17:18:48.326823 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.326809    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg\" (UID: \"35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg"
Jul 28 17:18:48.338412 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.338376    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zt5k\" (UniqueName: \"kubernetes.io/projected/35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd-kube-api-access-2zt5k\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg\" (UID: \"35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg"
Jul 28 17:18:48.345243 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.345198    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj"]
Jul 28 17:18:48.348714 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.348692    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj"
Jul 28 17:18:48.357175 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.357149    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf"
Jul 28 17:18:48.359870 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.359839    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj"]
Jul 28 17:18:48.427904 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.427874    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efa6f5e2-f821-4866-aa3a-892660c5a3c7-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj\" (UID: \"efa6f5e2-f821-4866-aa3a-892660c5a3c7\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj"
Jul 28 17:18:48.428058 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.427914    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfe19b1b-8b03-48b2-825a-785bd2a1b50b-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv\" (UID: \"dfe19b1b-8b03-48b2-825a-785bd2a1b50b\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv"
Jul 28 17:18:48.428058 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.427991    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfe19b1b-8b03-48b2-825a-785bd2a1b50b-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv\" (UID: \"dfe19b1b-8b03-48b2-825a-785bd2a1b50b\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv"
Jul 28 17:18:48.428058 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.428029    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74wls\" (UniqueName: \"kubernetes.io/projected/efa6f5e2-f821-4866-aa3a-892660c5a3c7-kube-api-access-74wls\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj\" (UID: \"efa6f5e2-f821-4866-aa3a-892660c5a3c7\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj"
Jul 28 17:18:48.428195 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.428070    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efa6f5e2-f821-4866-aa3a-892660c5a3c7-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj\" (UID: \"efa6f5e2-f821-4866-aa3a-892660c5a3c7\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj"
Jul 28 17:18:48.428195 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.428099    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cn7h\" (UniqueName: \"kubernetes.io/projected/dfe19b1b-8b03-48b2-825a-785bd2a1b50b-kube-api-access-8cn7h\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv\" (UID: \"dfe19b1b-8b03-48b2-825a-785bd2a1b50b\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv"
Jul 28 17:18:48.428322 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.428305    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfe19b1b-8b03-48b2-825a-785bd2a1b50b-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv\" (UID: \"dfe19b1b-8b03-48b2-825a-785bd2a1b50b\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv"
Jul 28 17:18:48.428403 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.428382    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfe19b1b-8b03-48b2-825a-785bd2a1b50b-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv\" (UID: \"dfe19b1b-8b03-48b2-825a-785bd2a1b50b\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv"
Jul 28 17:18:48.438644 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.438616    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cn7h\" (UniqueName: \"kubernetes.io/projected/dfe19b1b-8b03-48b2-825a-785bd2a1b50b-kube-api-access-8cn7h\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv\" (UID: \"dfe19b1b-8b03-48b2-825a-785bd2a1b50b\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv"
Jul 28 17:18:48.456665 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.456633    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg"
Jul 28 17:18:48.494725 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.494699    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf"]
Jul 28 17:18:48.496089 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:18:48.496061    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7ff6796_b305_4ab4_879b_24332ca96f8c.slice/crio-8fc49ffd72da941ee520a4631ee1fdf6e6990d07e8afca190130c29d80d6a301 WatchSource:0}: Error finding container 8fc49ffd72da941ee520a4631ee1fdf6e6990d07e8afca190130c29d80d6a301: Status 404 returned error can't find the container with id 8fc49ffd72da941ee520a4631ee1fdf6e6990d07e8afca190130c29d80d6a301
Jul 28 17:18:48.529616 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.529581    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efa6f5e2-f821-4866-aa3a-892660c5a3c7-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj\" (UID: \"efa6f5e2-f821-4866-aa3a-892660c5a3c7\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj"
Jul 28 17:18:48.529777 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.529668    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74wls\" (UniqueName: \"kubernetes.io/projected/efa6f5e2-f821-4866-aa3a-892660c5a3c7-kube-api-access-74wls\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj\" (UID: \"efa6f5e2-f821-4866-aa3a-892660c5a3c7\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj"
Jul 28 17:18:48.529777 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.529715    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efa6f5e2-f821-4866-aa3a-892660c5a3c7-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj\" (UID: \"efa6f5e2-f821-4866-aa3a-892660c5a3c7\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj"
Jul 28 17:18:48.530051 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.530010    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efa6f5e2-f821-4866-aa3a-892660c5a3c7-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj\" (UID: \"efa6f5e2-f821-4866-aa3a-892660c5a3c7\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj"
Jul 28 17:18:48.530198 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.530180    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efa6f5e2-f821-4866-aa3a-892660c5a3c7-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj\" (UID: \"efa6f5e2-f821-4866-aa3a-892660c5a3c7\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj"
Jul 28 17:18:48.540204 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.540161    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74wls\" (UniqueName: \"kubernetes.io/projected/efa6f5e2-f821-4866-aa3a-892660c5a3c7-kube-api-access-74wls\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj\" (UID: \"efa6f5e2-f821-4866-aa3a-892660c5a3c7\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj"
Jul 28 17:18:48.556133 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.555933    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv"
Jul 28 17:18:48.616429 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.616395    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg"]
Jul 28 17:18:48.649931 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:18:48.649880    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35fd3801_61d9_4efd_8bbd_bf7fefe2a4dd.slice/crio-b344394321443427ab8054c52e33018523cf79c9f2784dc5df85d9fc5459831a WatchSource:0}: Error finding container b344394321443427ab8054c52e33018523cf79c9f2784dc5df85d9fc5459831a: Status 404 returned error can't find the container with id b344394321443427ab8054c52e33018523cf79c9f2784dc5df85d9fc5459831a
Jul 28 17:18:48.658170 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.658136    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj"
Jul 28 17:18:48.697599 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.697520    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv"]
Jul 28 17:18:48.700383 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:18:48.700356    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfe19b1b_8b03_48b2_825a_785bd2a1b50b.slice/crio-38b3cf0d69b7ef24a4c082a636c89d0691acf790390b85c3cf7f880f91c72721 WatchSource:0}: Error finding container 38b3cf0d69b7ef24a4c082a636c89d0691acf790390b85c3cf7f880f91c72721: Status 404 returned error can't find the container with id 38b3cf0d69b7ef24a4c082a636c89d0691acf790390b85c3cf7f880f91c72721
Jul 28 17:18:48.807667 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:48.807631    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj"]
Jul 28 17:18:48.815514 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:18:48.815465    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefa6f5e2_f821_4866_aa3a_892660c5a3c7.slice/crio-8c690a6727cafbdfe59bdae6f274058ee4bb41e32fce444659bc2fcba197e7ac WatchSource:0}: Error finding container 8c690a6727cafbdfe59bdae6f274058ee4bb41e32fce444659bc2fcba197e7ac: Status 404 returned error can't find the container with id 8c690a6727cafbdfe59bdae6f274058ee4bb41e32fce444659bc2fcba197e7ac
Jul 28 17:18:49.023211 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:49.023170    2567 generic.go:356] "Generic (PLEG): container finished" podID="efa6f5e2-f821-4866-aa3a-892660c5a3c7" containerID="ed57279844e00665bf6e316ac30eb8d8fb2b536f5011e4fe8782d0df57083e21" exitCode=0
Jul 28 17:18:49.023373 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:49.023206    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj" event={"ID":"efa6f5e2-f821-4866-aa3a-892660c5a3c7","Type":"ContainerDied","Data":"ed57279844e00665bf6e316ac30eb8d8fb2b536f5011e4fe8782d0df57083e21"}
Jul 28 17:18:49.023373 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:49.023266    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj" event={"ID":"efa6f5e2-f821-4866-aa3a-892660c5a3c7","Type":"ContainerStarted","Data":"8c690a6727cafbdfe59bdae6f274058ee4bb41e32fce444659bc2fcba197e7ac"}
Jul 28 17:18:49.024761 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:49.024660    2567 generic.go:356] "Generic (PLEG): container finished" podID="d7ff6796-b305-4ab4-879b-24332ca96f8c" containerID="1192d5e6fd34515970ac81103077e6ff206f3148a2864c10b8f7c758c9ce2b63" exitCode=0
Jul 28 17:18:49.024828 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:49.024760    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf" event={"ID":"d7ff6796-b305-4ab4-879b-24332ca96f8c","Type":"ContainerDied","Data":"1192d5e6fd34515970ac81103077e6ff206f3148a2864c10b8f7c758c9ce2b63"}
Jul 28 17:18:49.024828 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:49.024796    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf" event={"ID":"d7ff6796-b305-4ab4-879b-24332ca96f8c","Type":"ContainerStarted","Data":"8fc49ffd72da941ee520a4631ee1fdf6e6990d07e8afca190130c29d80d6a301"}
Jul 28 17:18:49.026292 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:49.026260    2567 generic.go:356] "Generic (PLEG): container finished" podID="35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd" containerID="392372255b91eff6dfa8a4295bd9780100d4dfab615b3c50d5aa6d3082ce5816" exitCode=0
Jul 28 17:18:49.026360 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:49.026305    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg" event={"ID":"35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd","Type":"ContainerDied","Data":"392372255b91eff6dfa8a4295bd9780100d4dfab615b3c50d5aa6d3082ce5816"}
Jul 28 17:18:49.026409 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:49.026378    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg" event={"ID":"35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd","Type":"ContainerStarted","Data":"b344394321443427ab8054c52e33018523cf79c9f2784dc5df85d9fc5459831a"}
Jul 28 17:18:49.027995 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:49.027976    2567 generic.go:356] "Generic (PLEG): container finished" podID="dfe19b1b-8b03-48b2-825a-785bd2a1b50b" containerID="d91e9c0dc214be47c7f8daf1b5620a533afd95e6d47b325c2b6aac4a1734f69f" exitCode=0
Jul 28 17:18:49.028108 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:49.028005    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv" event={"ID":"dfe19b1b-8b03-48b2-825a-785bd2a1b50b","Type":"ContainerDied","Data":"d91e9c0dc214be47c7f8daf1b5620a533afd95e6d47b325c2b6aac4a1734f69f"}
Jul 28 17:18:49.028108 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:49.028037    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv" event={"ID":"dfe19b1b-8b03-48b2-825a-785bd2a1b50b","Type":"ContainerStarted","Data":"38b3cf0d69b7ef24a4c082a636c89d0691acf790390b85c3cf7f880f91c72721"}
Jul 28 17:18:50.034783 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:50.034743    2567 generic.go:356] "Generic (PLEG): container finished" podID="efa6f5e2-f821-4866-aa3a-892660c5a3c7" containerID="ae6bb39f29d88ca3c37ed25450f4ae69c5d45310e972216307777e85545af543" exitCode=0
Jul 28 17:18:50.035174 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:50.034815    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj" event={"ID":"efa6f5e2-f821-4866-aa3a-892660c5a3c7","Type":"ContainerDied","Data":"ae6bb39f29d88ca3c37ed25450f4ae69c5d45310e972216307777e85545af543"}
Jul 28 17:18:51.041311 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:51.041276    2567 generic.go:356] "Generic (PLEG): container finished" podID="dfe19b1b-8b03-48b2-825a-785bd2a1b50b" containerID="fed905fdbb78bd35db2218e398524f503906ea63a309ca873b596730c12afaa0" exitCode=0
Jul 28 17:18:51.041788 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:51.041364    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv" event={"ID":"dfe19b1b-8b03-48b2-825a-785bd2a1b50b","Type":"ContainerDied","Data":"fed905fdbb78bd35db2218e398524f503906ea63a309ca873b596730c12afaa0"}
Jul 28 17:18:51.043635 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:51.043608    2567 generic.go:356] "Generic (PLEG): container finished" podID="efa6f5e2-f821-4866-aa3a-892660c5a3c7" containerID="5fbce89509cedaee7fa4da543d1c9b87d3136ea5a13a963072be5d2b1cced90f" exitCode=0
Jul 28 17:18:51.043749 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:51.043696    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj" event={"ID":"efa6f5e2-f821-4866-aa3a-892660c5a3c7","Type":"ContainerDied","Data":"5fbce89509cedaee7fa4da543d1c9b87d3136ea5a13a963072be5d2b1cced90f"}
Jul 28 17:18:51.045294 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:51.045268    2567 generic.go:356] "Generic (PLEG): container finished" podID="d7ff6796-b305-4ab4-879b-24332ca96f8c" containerID="78180d43d7dcfce1364889d5933a9370983cd0223973235d1b30dc52f6c69ba4" exitCode=0
Jul 28 17:18:51.045415 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:51.045353    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf" event={"ID":"d7ff6796-b305-4ab4-879b-24332ca96f8c","Type":"ContainerDied","Data":"78180d43d7dcfce1364889d5933a9370983cd0223973235d1b30dc52f6c69ba4"}
Jul 28 17:18:52.051374 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:52.051334    2567 generic.go:356] "Generic (PLEG): container finished" podID="d7ff6796-b305-4ab4-879b-24332ca96f8c" containerID="06487f3b94327e70fb2f2e10eaf904b4bb02cccd0f95b2f0bb9113aa35c98f91" exitCode=0
Jul 28 17:18:52.051837 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:52.051418    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf" event={"ID":"d7ff6796-b305-4ab4-879b-24332ca96f8c","Type":"ContainerDied","Data":"06487f3b94327e70fb2f2e10eaf904b4bb02cccd0f95b2f0bb9113aa35c98f91"}
Jul 28 17:18:52.053538 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:52.053497    2567 generic.go:356] "Generic (PLEG): container finished" podID="dfe19b1b-8b03-48b2-825a-785bd2a1b50b" containerID="5ad55b837729088f659d9a8bec25ffe294d7537dc7918121399216abd6998741" exitCode=0
Jul 28 17:18:52.053703 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:52.053610    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv" event={"ID":"dfe19b1b-8b03-48b2-825a-785bd2a1b50b","Type":"ContainerDied","Data":"5ad55b837729088f659d9a8bec25ffe294d7537dc7918121399216abd6998741"}
Jul 28 17:18:52.187928 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:52.187894    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj"
Jul 28 17:18:52.257566 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:52.257529    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efa6f5e2-f821-4866-aa3a-892660c5a3c7-bundle\") pod \"efa6f5e2-f821-4866-aa3a-892660c5a3c7\" (UID: \"efa6f5e2-f821-4866-aa3a-892660c5a3c7\") "
Jul 28 17:18:52.257763 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:52.257576    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74wls\" (UniqueName: \"kubernetes.io/projected/efa6f5e2-f821-4866-aa3a-892660c5a3c7-kube-api-access-74wls\") pod \"efa6f5e2-f821-4866-aa3a-892660c5a3c7\" (UID: \"efa6f5e2-f821-4866-aa3a-892660c5a3c7\") "
Jul 28 17:18:52.257763 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:52.257609    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efa6f5e2-f821-4866-aa3a-892660c5a3c7-util\") pod \"efa6f5e2-f821-4866-aa3a-892660c5a3c7\" (UID: \"efa6f5e2-f821-4866-aa3a-892660c5a3c7\") "
Jul 28 17:18:52.258086 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:52.258053    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa6f5e2-f821-4866-aa3a-892660c5a3c7-bundle" (OuterVolumeSpecName: "bundle") pod "efa6f5e2-f821-4866-aa3a-892660c5a3c7" (UID: "efa6f5e2-f821-4866-aa3a-892660c5a3c7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:18:52.259909 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:52.259876    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efa6f5e2-f821-4866-aa3a-892660c5a3c7-kube-api-access-74wls" (OuterVolumeSpecName: "kube-api-access-74wls") pod "efa6f5e2-f821-4866-aa3a-892660c5a3c7" (UID: "efa6f5e2-f821-4866-aa3a-892660c5a3c7"). InnerVolumeSpecName "kube-api-access-74wls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:18:52.262847 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:52.262806    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa6f5e2-f821-4866-aa3a-892660c5a3c7-util" (OuterVolumeSpecName: "util") pod "efa6f5e2-f821-4866-aa3a-892660c5a3c7" (UID: "efa6f5e2-f821-4866-aa3a-892660c5a3c7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:18:52.358482 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:52.358380    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efa6f5e2-f821-4866-aa3a-892660c5a3c7-util\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:18:52.358482 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:52.358412    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efa6f5e2-f821-4866-aa3a-892660c5a3c7-bundle\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:18:52.358482 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:52.358422    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74wls\" (UniqueName: \"kubernetes.io/projected/efa6f5e2-f821-4866-aa3a-892660c5a3c7-kube-api-access-74wls\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:18:53.059636 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.059607    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj"
Jul 28 17:18:53.060037 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.059640    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2c8rj" event={"ID":"efa6f5e2-f821-4866-aa3a-892660c5a3c7","Type":"ContainerDied","Data":"8c690a6727cafbdfe59bdae6f274058ee4bb41e32fce444659bc2fcba197e7ac"}
Jul 28 17:18:53.060037 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.059695    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c690a6727cafbdfe59bdae6f274058ee4bb41e32fce444659bc2fcba197e7ac"
Jul 28 17:18:53.193444 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.193418    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf"
Jul 28 17:18:53.224921 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.224890    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv"
Jul 28 17:18:53.265267 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.265224    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cn7h\" (UniqueName: \"kubernetes.io/projected/dfe19b1b-8b03-48b2-825a-785bd2a1b50b-kube-api-access-8cn7h\") pod \"dfe19b1b-8b03-48b2-825a-785bd2a1b50b\" (UID: \"dfe19b1b-8b03-48b2-825a-785bd2a1b50b\") "
Jul 28 17:18:53.265477 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.265379    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfe19b1b-8b03-48b2-825a-785bd2a1b50b-util\") pod \"dfe19b1b-8b03-48b2-825a-785bd2a1b50b\" (UID: \"dfe19b1b-8b03-48b2-825a-785bd2a1b50b\") "
Jul 28 17:18:53.265477 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.265421    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7ff6796-b305-4ab4-879b-24332ca96f8c-util\") pod \"d7ff6796-b305-4ab4-879b-24332ca96f8c\" (UID: \"d7ff6796-b305-4ab4-879b-24332ca96f8c\") "
Jul 28 17:18:53.265477 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.265460    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cj9nm\" (UniqueName: \"kubernetes.io/projected/d7ff6796-b305-4ab4-879b-24332ca96f8c-kube-api-access-cj9nm\") pod \"d7ff6796-b305-4ab4-879b-24332ca96f8c\" (UID: \"d7ff6796-b305-4ab4-879b-24332ca96f8c\") "
Jul 28 17:18:53.265634 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.265488    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7ff6796-b305-4ab4-879b-24332ca96f8c-bundle\") pod \"d7ff6796-b305-4ab4-879b-24332ca96f8c\" (UID: \"d7ff6796-b305-4ab4-879b-24332ca96f8c\") "
Jul 28 17:18:53.265634 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.265570    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfe19b1b-8b03-48b2-825a-785bd2a1b50b-bundle\") pod \"dfe19b1b-8b03-48b2-825a-785bd2a1b50b\" (UID: \"dfe19b1b-8b03-48b2-825a-785bd2a1b50b\") "
Jul 28 17:18:53.266336 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.266281    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfe19b1b-8b03-48b2-825a-785bd2a1b50b-bundle" (OuterVolumeSpecName: "bundle") pod "dfe19b1b-8b03-48b2-825a-785bd2a1b50b" (UID: "dfe19b1b-8b03-48b2-825a-785bd2a1b50b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:18:53.266539 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.266481    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ff6796-b305-4ab4-879b-24332ca96f8c-bundle" (OuterVolumeSpecName: "bundle") pod "d7ff6796-b305-4ab4-879b-24332ca96f8c" (UID: "d7ff6796-b305-4ab4-879b-24332ca96f8c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:18:53.267668 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.267637    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfe19b1b-8b03-48b2-825a-785bd2a1b50b-kube-api-access-8cn7h" (OuterVolumeSpecName: "kube-api-access-8cn7h") pod "dfe19b1b-8b03-48b2-825a-785bd2a1b50b" (UID: "dfe19b1b-8b03-48b2-825a-785bd2a1b50b"). InnerVolumeSpecName "kube-api-access-8cn7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:18:53.267771 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.267677    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7ff6796-b305-4ab4-879b-24332ca96f8c-kube-api-access-cj9nm" (OuterVolumeSpecName: "kube-api-access-cj9nm") pod "d7ff6796-b305-4ab4-879b-24332ca96f8c" (UID: "d7ff6796-b305-4ab4-879b-24332ca96f8c"). InnerVolumeSpecName "kube-api-access-cj9nm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:18:53.271892 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.271828    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ff6796-b305-4ab4-879b-24332ca96f8c-util" (OuterVolumeSpecName: "util") pod "d7ff6796-b305-4ab4-879b-24332ca96f8c" (UID: "d7ff6796-b305-4ab4-879b-24332ca96f8c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:18:53.272152 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.272126    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfe19b1b-8b03-48b2-825a-785bd2a1b50b-util" (OuterVolumeSpecName: "util") pod "dfe19b1b-8b03-48b2-825a-785bd2a1b50b" (UID: "dfe19b1b-8b03-48b2-825a-785bd2a1b50b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:18:53.366807 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.366700    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfe19b1b-8b03-48b2-825a-785bd2a1b50b-util\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:18:53.366807 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.366737    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7ff6796-b305-4ab4-879b-24332ca96f8c-util\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:18:53.366807 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.366747    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cj9nm\" (UniqueName: \"kubernetes.io/projected/d7ff6796-b305-4ab4-879b-24332ca96f8c-kube-api-access-cj9nm\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:18:53.366807 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.366756    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7ff6796-b305-4ab4-879b-24332ca96f8c-bundle\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:18:53.366807 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.366766    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfe19b1b-8b03-48b2-825a-785bd2a1b50b-bundle\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:18:53.366807 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:53.366774    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cn7h\" (UniqueName: \"kubernetes.io/projected/dfe19b1b-8b03-48b2-825a-785bd2a1b50b-kube-api-access-8cn7h\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:18:54.066023 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:54.065989    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf"
Jul 28 17:18:54.066445 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:54.066025    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053c4vkf" event={"ID":"d7ff6796-b305-4ab4-879b-24332ca96f8c","Type":"ContainerDied","Data":"8fc49ffd72da941ee520a4631ee1fdf6e6990d07e8afca190130c29d80d6a301"}
Jul 28 17:18:54.066445 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:54.066073    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8fc49ffd72da941ee520a4631ee1fdf6e6990d07e8afca190130c29d80d6a301"
Jul 28 17:18:54.068027 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:54.067995    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv" event={"ID":"dfe19b1b-8b03-48b2-825a-785bd2a1b50b","Type":"ContainerDied","Data":"38b3cf0d69b7ef24a4c082a636c89d0691acf790390b85c3cf7f880f91c72721"}
Jul 28 17:18:54.068149 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:54.068069    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="38b3cf0d69b7ef24a4c082a636c89d0691acf790390b85c3cf7f880f91c72721"
Jul 28 17:18:54.068149 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:54.068090    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ckvrlv"
Jul 28 17:18:55.073526 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:55.073471    2567 generic.go:356] "Generic (PLEG): container finished" podID="35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd" containerID="b6985113d9833355dbdd6e326fe2a6e58ec6ab6d9b27d3fe1a5cec63dde8c928" exitCode=0
Jul 28 17:18:55.073909 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:55.073557    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg" event={"ID":"35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd","Type":"ContainerDied","Data":"b6985113d9833355dbdd6e326fe2a6e58ec6ab6d9b27d3fe1a5cec63dde8c928"}
Jul 28 17:18:56.079652 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:56.079612    2567 generic.go:356] "Generic (PLEG): container finished" podID="35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd" containerID="6593ad37f9760d50d7bd7d83acc12000a5b4349099c84f7f1f51dccbffefce8a" exitCode=0
Jul 28 17:18:56.079652 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:56.079655    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg" event={"ID":"35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd","Type":"ContainerDied","Data":"6593ad37f9760d50d7bd7d83acc12000a5b4349099c84f7f1f51dccbffefce8a"}
Jul 28 17:18:57.217099 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:57.217063    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg"
Jul 28 17:18:57.295807 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:57.295767    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zt5k\" (UniqueName: \"kubernetes.io/projected/35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd-kube-api-access-2zt5k\") pod \"35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd\" (UID: \"35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd\") "
Jul 28 17:18:57.295975 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:57.295874    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd-bundle\") pod \"35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd\" (UID: \"35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd\") "
Jul 28 17:18:57.295975 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:57.295895    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd-util\") pod \"35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd\" (UID: \"35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd\") "
Jul 28 17:18:57.296358 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:57.296333    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd-bundle" (OuterVolumeSpecName: "bundle") pod "35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd" (UID: "35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:18:57.297988 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:57.297964    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd-kube-api-access-2zt5k" (OuterVolumeSpecName: "kube-api-access-2zt5k") pod "35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd" (UID: "35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd"). InnerVolumeSpecName "kube-api-access-2zt5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:18:57.302595 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:57.302554    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd-util" (OuterVolumeSpecName: "util") pod "35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd" (UID: "35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:18:57.397203 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:57.397092    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd-bundle\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:18:57.397203 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:57.397139    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd-util\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:18:57.397203 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:57.397150    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zt5k\" (UniqueName: \"kubernetes.io/projected/35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd-kube-api-access-2zt5k\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:18:58.090924 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:58.090890    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg"
Jul 28 17:18:58.091115 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:58.090891    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0vgqvg" event={"ID":"35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd","Type":"ContainerDied","Data":"b344394321443427ab8054c52e33018523cf79c9f2784dc5df85d9fc5459831a"}
Jul 28 17:18:58.091115 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:18:58.091052    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b344394321443427ab8054c52e33018523cf79c9f2784dc5df85d9fc5459831a"
Jul 28 17:19:08.846059 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846025    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-66866f5fcc-7phb4"]
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846278    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ff6796-b305-4ab4-879b-24332ca96f8c" containerName="extract"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846289    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ff6796-b305-4ab4-879b-24332ca96f8c" containerName="extract"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846298    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efa6f5e2-f821-4866-aa3a-892660c5a3c7" containerName="extract"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846304    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="efa6f5e2-f821-4866-aa3a-892660c5a3c7" containerName="extract"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846309    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd" containerName="pull"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846316    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd" containerName="pull"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846326    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd" containerName="util"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846335    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd" containerName="util"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846342    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efa6f5e2-f821-4866-aa3a-892660c5a3c7" containerName="pull"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846349    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="efa6f5e2-f821-4866-aa3a-892660c5a3c7" containerName="pull"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846357    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd" containerName="extract"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846364    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd" containerName="extract"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846377    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfe19b1b-8b03-48b2-825a-785bd2a1b50b" containerName="extract"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846384    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfe19b1b-8b03-48b2-825a-785bd2a1b50b" containerName="extract"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846393    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ff6796-b305-4ab4-879b-24332ca96f8c" containerName="util"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846401    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ff6796-b305-4ab4-879b-24332ca96f8c" containerName="util"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846411    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ff6796-b305-4ab4-879b-24332ca96f8c" containerName="pull"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846417    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ff6796-b305-4ab4-879b-24332ca96f8c" containerName="pull"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846434    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfe19b1b-8b03-48b2-825a-785bd2a1b50b" containerName="util"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846441    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfe19b1b-8b03-48b2-825a-785bd2a1b50b" containerName="util"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846451    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efa6f5e2-f821-4866-aa3a-892660c5a3c7" containerName="util"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846458    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="efa6f5e2-f821-4866-aa3a-892660c5a3c7" containerName="util"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846475    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfe19b1b-8b03-48b2-825a-785bd2a1b50b" containerName="pull"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846482    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfe19b1b-8b03-48b2-825a-785bd2a1b50b" containerName="pull"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846564    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfe19b1b-8b03-48b2-825a-785bd2a1b50b" containerName="extract"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846579    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7ff6796-b305-4ab4-879b-24332ca96f8c" containerName="extract"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846588    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="efa6f5e2-f821-4866-aa3a-892660c5a3c7" containerName="extract"
Jul 28 17:19:08.846610 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.846597    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="35fd3801-61d9-4efd-8bbd-bf7fefe2a4dd" containerName="extract"
Jul 28 17:19:08.848673 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.848654    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-66866f5fcc-7phb4"
Jul 28 17:19:08.852416 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.852383    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 17:19:08.852842 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.852825    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4k6x8\""
Jul 28 17:19:08.853185 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.853168    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 17:19:08.878477 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.878438    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-66866f5fcc-7phb4"]
Jul 28 17:19:08.888239 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.888205    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9985ba3-0799-40cb-8e84-d747050273d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-66866f5fcc-7phb4\" (UID: \"b9985ba3-0799-40cb-8e84-d747050273d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-66866f5fcc-7phb4"
Jul 28 17:19:08.888430 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.888259    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xd78\" (UniqueName: \"kubernetes.io/projected/b9985ba3-0799-40cb-8e84-d747050273d6-kube-api-access-2xd78\") pod \"kuadrant-operator-controller-manager-66866f5fcc-7phb4\" (UID: \"b9985ba3-0799-40cb-8e84-d747050273d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-66866f5fcc-7phb4"
Jul 28 17:19:08.988834 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.988796    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xd78\" (UniqueName: \"kubernetes.io/projected/b9985ba3-0799-40cb-8e84-d747050273d6-kube-api-access-2xd78\") pod \"kuadrant-operator-controller-manager-66866f5fcc-7phb4\" (UID: \"b9985ba3-0799-40cb-8e84-d747050273d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-66866f5fcc-7phb4"
Jul 28 17:19:08.989007 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.988857    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9985ba3-0799-40cb-8e84-d747050273d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-66866f5fcc-7phb4\" (UID: \"b9985ba3-0799-40cb-8e84-d747050273d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-66866f5fcc-7phb4"
Jul 28 17:19:08.989211 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:08.989191    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9985ba3-0799-40cb-8e84-d747050273d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-66866f5fcc-7phb4\" (UID: \"b9985ba3-0799-40cb-8e84-d747050273d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-66866f5fcc-7phb4"
Jul 28 17:19:09.022030 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:09.021987    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xd78\" (UniqueName: \"kubernetes.io/projected/b9985ba3-0799-40cb-8e84-d747050273d6-kube-api-access-2xd78\") pod \"kuadrant-operator-controller-manager-66866f5fcc-7phb4\" (UID: \"b9985ba3-0799-40cb-8e84-d747050273d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-66866f5fcc-7phb4"
Jul 28 17:19:09.158100 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:09.158004    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-66866f5fcc-7phb4"
Jul 28 17:19:09.309279 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:09.307404    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-66866f5fcc-7phb4"]
Jul 28 17:19:09.312451 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:19:09.312418    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9985ba3_0799_40cb_8e84_d747050273d6.slice/crio-98e34aec8ec8ae9db2d99626fa757f78fc738d2132c83a294237de030a4ffe36 WatchSource:0}: Error finding container 98e34aec8ec8ae9db2d99626fa757f78fc738d2132c83a294237de030a4ffe36: Status 404 returned error can't find the container with id 98e34aec8ec8ae9db2d99626fa757f78fc738d2132c83a294237de030a4ffe36
Jul 28 17:19:10.139431 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:10.139390    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-66866f5fcc-7phb4" event={"ID":"b9985ba3-0799-40cb-8e84-d747050273d6","Type":"ContainerStarted","Data":"98e34aec8ec8ae9db2d99626fa757f78fc738d2132c83a294237de030a4ffe36"}
Jul 28 17:19:13.555039 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:13.554997    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-56fb6f6856-nq65w"]
Jul 28 17:19:13.557847 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:13.557823    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-56fb6f6856-nq65w"
Jul 28 17:19:13.560826 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:13.560800    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-7q9wg\""
Jul 28 17:19:13.570108 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:13.570072    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-56fb6f6856-nq65w"]
Jul 28 17:19:13.630646 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:13.630609    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zkzg\" (UniqueName: \"kubernetes.io/projected/1ae9718c-eecf-4360-afd2-76105c0ef626-kube-api-access-9zkzg\") pod \"authorino-operator-56fb6f6856-nq65w\" (UID: \"1ae9718c-eecf-4360-afd2-76105c0ef626\") " pod="kuadrant-system/authorino-operator-56fb6f6856-nq65w"
Jul 28 17:19:13.731969 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:13.731931    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zkzg\" (UniqueName: \"kubernetes.io/projected/1ae9718c-eecf-4360-afd2-76105c0ef626-kube-api-access-9zkzg\") pod \"authorino-operator-56fb6f6856-nq65w\" (UID: \"1ae9718c-eecf-4360-afd2-76105c0ef626\") " pod="kuadrant-system/authorino-operator-56fb6f6856-nq65w"
Jul 28 17:19:13.745874 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:13.745839    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zkzg\" (UniqueName: \"kubernetes.io/projected/1ae9718c-eecf-4360-afd2-76105c0ef626-kube-api-access-9zkzg\") pod \"authorino-operator-56fb6f6856-nq65w\" (UID: \"1ae9718c-eecf-4360-afd2-76105c0ef626\") " pod="kuadrant-system/authorino-operator-56fb6f6856-nq65w"
Jul 28 17:19:13.871905 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:13.871806    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-56fb6f6856-nq65w"
Jul 28 17:19:14.081807 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:14.081779    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-56fb6f6856-nq65w"]
Jul 28 17:19:14.085050 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:19:14.085011    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ae9718c_eecf_4360_afd2_76105c0ef626.slice/crio-5db1d105aeaa8fed86d15c723a3b6d5e5b6bf2d7dacc6bb84442d31dd2f9e958 WatchSource:0}: Error finding container 5db1d105aeaa8fed86d15c723a3b6d5e5b6bf2d7dacc6bb84442d31dd2f9e958: Status 404 returned error can't find the container with id 5db1d105aeaa8fed86d15c723a3b6d5e5b6bf2d7dacc6bb84442d31dd2f9e958
Jul 28 17:19:14.162095 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:14.161998    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-66866f5fcc-7phb4" event={"ID":"b9985ba3-0799-40cb-8e84-d747050273d6","Type":"ContainerStarted","Data":"96c2c59ca03fddf904e2e73230911af7ca7d0c6bcd4f7221cc0e868ddc458da5"}
Jul 28 17:19:14.162095 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:14.162090    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-66866f5fcc-7phb4"
Jul 28 17:19:14.163322 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:14.163298    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-56fb6f6856-nq65w" event={"ID":"1ae9718c-eecf-4360-afd2-76105c0ef626","Type":"ContainerStarted","Data":"5db1d105aeaa8fed86d15c723a3b6d5e5b6bf2d7dacc6bb84442d31dd2f9e958"}
Jul 28 17:19:14.185495 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:14.185428    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-66866f5fcc-7phb4" podStartSLOduration=1.508420584 podStartE2EDuration="6.185405415s" totalImagesPullingTime="4.676984831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:19:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:19:14.183919734 +0000 UTC m=+509.430021943" watchObservedRunningTime="2026-07-28 17:19:14.185405415 +0000 UTC m=+509.431507618"
Jul 28 17:19:21.198783 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:21.198691    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-56fb6f6856-nq65w" event={"ID":"1ae9718c-eecf-4360-afd2-76105c0ef626","Type":"ContainerStarted","Data":"627df4fc4aa87a8bc978f6a7fd0131d2bbcb27b7dfbdc07e9460fec5ce7afdaa"}
Jul 28 17:19:21.199215 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:21.198916    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-56fb6f6856-nq65w"
Jul 28 17:19:21.225148 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:21.225087    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-56fb6f6856-nq65w" podStartSLOduration=1.440662084 podStartE2EDuration="8.225070888s" totalImagesPullingTime="6.784408804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:19:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:19:21.224406674 +0000 UTC m=+516.470508880" watchObservedRunningTime="2026-07-28 17:19:21.225070888 +0000 UTC m=+516.471173093"
Jul 28 17:19:25.169815 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:25.169781    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-66866f5fcc-7phb4"
Jul 28 17:19:32.205434 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:32.205383    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-56fb6f6856-nq65w"
Jul 28 17:19:58.083561 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:58.083523    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-glmwj"]
Jul 28 17:19:58.088541 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:58.088521    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-glmwj"
Jul 28 17:19:58.091423 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:58.091400    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 28 17:19:58.091543 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:58.091474    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hg4cd\""
Jul 28 17:19:58.097153 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:58.097132    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-glmwj"]
Jul 28 17:19:58.112577 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:58.112547    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-glmwj"]
Jul 28 17:19:58.174309 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:58.174284    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b-config-file\") pod \"limitador-limitador-85d9db4d5d-glmwj\" (UID: \"a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-glmwj"
Jul 28 17:19:58.174418 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:58.174312    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz5mn\" (UniqueName: \"kubernetes.io/projected/a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b-kube-api-access-vz5mn\") pod \"limitador-limitador-85d9db4d5d-glmwj\" (UID: \"a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-glmwj"
Jul 28 17:19:58.275283 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:58.275254    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b-config-file\") pod \"limitador-limitador-85d9db4d5d-glmwj\" (UID: \"a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-glmwj"
Jul 28 17:19:58.275283 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:58.275284    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz5mn\" (UniqueName: \"kubernetes.io/projected/a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b-kube-api-access-vz5mn\") pod \"limitador-limitador-85d9db4d5d-glmwj\" (UID: \"a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-glmwj"
Jul 28 17:19:58.275950 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:58.275932    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b-config-file\") pod \"limitador-limitador-85d9db4d5d-glmwj\" (UID: \"a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-glmwj"
Jul 28 17:19:58.283757 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:58.283736    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz5mn\" (UniqueName: \"kubernetes.io/projected/a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b-kube-api-access-vz5mn\") pod \"limitador-limitador-85d9db4d5d-glmwj\" (UID: \"a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-glmwj"
Jul 28 17:19:58.399332 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:58.399262    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-glmwj"
Jul 28 17:19:58.525793 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:58.525643    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-glmwj"]
Jul 28 17:19:58.527986 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:19:58.527962    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4e05dcd_6cb8_4e63_8c53_3ff7b4bf520b.slice/crio-835a38db794ddc67ff71b2d2bbc66e5c55678d1572aebbe4a4063c1c55f553bc WatchSource:0}: Error finding container 835a38db794ddc67ff71b2d2bbc66e5c55678d1572aebbe4a4063c1c55f553bc: Status 404 returned error can't find the container with id 835a38db794ddc67ff71b2d2bbc66e5c55678d1572aebbe4a4063c1c55f553bc
Jul 28 17:19:59.363893 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:19:59.363863    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-glmwj" event={"ID":"a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b","Type":"ContainerStarted","Data":"835a38db794ddc67ff71b2d2bbc66e5c55678d1572aebbe4a4063c1c55f553bc"}
Jul 28 17:20:00.370064 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:20:00.370030    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-glmwj" event={"ID":"a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b","Type":"ContainerStarted","Data":"ea057be244520acfdbf8dfbfe9d8d41bf371776d4681c569e40154edd8ec6cad"}
Jul 28 17:20:00.370457 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:20:00.370161    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-glmwj"
Jul 28 17:20:00.391336 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:20:00.391237    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-glmwj" podStartSLOduration=0.785520249 podStartE2EDuration="2.391221955s" totalImagesPullingTime="1.605701706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:20:00.390314168 +0000 UTC m=+555.636416375" watchObservedRunningTime="2026-07-28 17:20:00.391221955 +0000 UTC m=+555.637324161"
Jul 28 17:20:11.375835 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:20:11.375804    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-glmwj"
Jul 28 17:20:45.950188 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:20:45.950026    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:20:45.953688 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:20:45.950554    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:23:26.297950 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:23:26.297912    2567 generic.go:356] "Generic (PLEG): container finished" podID="be331145-6d91-44a5-aa68-247f0ab1262b" containerID="20695c6d5dfe34149fafbd719b65165de7feacab5170c94a1a10c59410b43f29" exitCode=0
Jul 28 17:23:26.298373 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:23:26.297958    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s" event={"ID":"be331145-6d91-44a5-aa68-247f0ab1262b","Type":"ContainerDied","Data":"20695c6d5dfe34149fafbd719b65165de7feacab5170c94a1a10c59410b43f29"}
Jul 28 17:23:26.298373 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:23:26.298301    2567 scope.go:117] "RemoveContainer" containerID="20695c6d5dfe34149fafbd719b65165de7feacab5170c94a1a10c59410b43f29"
Jul 28 17:23:26.299197 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:23:26.299159    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 17:23:26.299250 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:23:26.299234    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 17:23:26.299283 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:23:26.299266    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 17:23:27.308425 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:23:27.308380    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s" event={"ID":"be331145-6d91-44a5-aa68-247f0ab1262b","Type":"ContainerStarted","Data":"a3dce7d8c1179b812ee039266d3ed8d2e3e80c5a97a0ea7205b5166897940665"}
Jul 28 17:23:27.756799 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:23:27.756690    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:23:27.757919 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:23:27.757889    2567 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" start-of-body=
Jul 28 17:23:27.758053 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:23:27.757941    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s" podUID="be331145-6d91-44a5-aa68-247f0ab1262b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused"
Jul 28 17:23:28.756909 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:23:28.756870    2567 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" start-of-body=
Jul 28 17:23:28.757349 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:23:28.756928    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s" podUID="be331145-6d91-44a5-aa68-247f0ab1262b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused"
Jul 28 17:23:29.756800 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:23:29.756754    2567 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" start-of-body=
Jul 28 17:23:29.756987 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:23:29.756840    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s" podUID="be331145-6d91-44a5-aa68-247f0ab1262b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused"
Jul 28 17:23:30.008531 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:23:30.008480    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:23:30.757641 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:23:30.757603    2567 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" start-of-body=
Jul 28 17:23:30.758073 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:23:30.757666    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s" podUID="be331145-6d91-44a5-aa68-247f0ab1262b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused"
Jul 28 17:23:31.756604 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:23:31.756568    2567 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" start-of-body=
Jul 28 17:23:31.756866 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:23:31.756637    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s" podUID="be331145-6d91-44a5-aa68-247f0ab1262b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused"
Jul 28 17:23:33.491284 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:23:33.491239    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:23:33.492916 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:23:33.492885    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-8699b59bbc-nvf9s"
Jul 28 17:24:42.186567 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:24:42.186483    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht"]
Jul 28 17:24:42.189413 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:24:42.189376    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht"
Jul 28 17:24:42.194345 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:24:42.194310    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"fc-concurrency-test-epp-sa-dockercfg-srrx8\""
Jul 28 17:24:42.194345 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:24:42.194335    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"openshift-service-ca.crt\""
Jul 28 17:24:42.194577 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:24:42.194354    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"default-dockercfg-b4zzn\""
Jul 28 17:24:42.194577 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:24:42.194318    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"kube-root-ca.crt\""
Jul 28 17:24:42.194577 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:24:42.194310    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"fc-concurrency-test-kserve-self-signed-certs\""
Jul 28 17:24:42.198679 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:24:42.198652    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht"]
Jul 28 17:24:42.319258 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:24:42.319218    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a311593-adff-4532-92d3-68549e2d6193-tls-certs\") pod \"fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht\" (UID: \"7a311593-adff-4532-92d3-68549e2d6193\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht"
Jul 28 17:24:42.319443 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:24:42.319278    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2f5p\" (UniqueName: \"kubernetes.io/projected/7a311593-adff-4532-92d3-68549e2d6193-kube-api-access-h2f5p\") pod \"fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht\" (UID: \"7a311593-adff-4532-92d3-68549e2d6193\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht"
Jul 28 17:24:42.420259 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:24:42.420224    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a311593-adff-4532-92d3-68549e2d6193-tls-certs\") pod \"fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht\" (UID: \"7a311593-adff-4532-92d3-68549e2d6193\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht"
Jul 28 17:24:42.420433 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:24:42.420278    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2f5p\" (UniqueName: \"kubernetes.io/projected/7a311593-adff-4532-92d3-68549e2d6193-kube-api-access-h2f5p\") pod \"fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht\" (UID: \"7a311593-adff-4532-92d3-68549e2d6193\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht"
Jul 28 17:24:42.422830 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:24:42.422795    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a311593-adff-4532-92d3-68549e2d6193-tls-certs\") pod \"fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht\" (UID: \"7a311593-adff-4532-92d3-68549e2d6193\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht"
Jul 28 17:24:42.430438 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:24:42.430408    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2f5p\" (UniqueName: \"kubernetes.io/projected/7a311593-adff-4532-92d3-68549e2d6193-kube-api-access-h2f5p\") pod \"fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht\" (UID: \"7a311593-adff-4532-92d3-68549e2d6193\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht"
Jul 28 17:24:42.502064 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:24:42.501963    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht"
Jul 28 17:24:42.644414 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:24:42.644385    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht"]
Jul 28 17:24:42.646288 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:24:42.646261    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a311593_adff_4532_92d3_68549e2d6193.slice/crio-50d3d7ece1afc6345cb9a6b47e2b07e44c78ace06ca3cd9a392535cd13d77fe9 WatchSource:0}: Error finding container 50d3d7ece1afc6345cb9a6b47e2b07e44c78ace06ca3cd9a392535cd13d77fe9: Status 404 returned error can't find the container with id 50d3d7ece1afc6345cb9a6b47e2b07e44c78ace06ca3cd9a392535cd13d77fe9
Jul 28 17:24:42.648280 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:24:42.648262    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 17:24:42.655237 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:24:42.655204    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht" event={"ID":"7a311593-adff-4532-92d3-68549e2d6193","Type":"ContainerStarted","Data":"50d3d7ece1afc6345cb9a6b47e2b07e44c78ace06ca3cd9a392535cd13d77fe9"}
Jul 28 17:24:45.683719 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:24:45.683674    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht" event={"ID":"7a311593-adff-4532-92d3-68549e2d6193","Type":"ContainerStarted","Data":"ebdb498bdfb739f8c5c6198cd35b7cb3fe955753bf01f11e2a0840b02e219371"}
Jul 28 17:24:45.684181 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:24:45.683820    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht"
Jul 28 17:24:45.702340 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:24:45.702275    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht" podStartSLOduration=1.410167444 podStartE2EDuration="3.702258967s" totalImagesPullingTime="2.292091523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:24:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:24:45.701265399 +0000 UTC m=+840.947367620" watchObservedRunningTime="2026-07-28 17:24:45.702258967 +0000 UTC m=+840.948361172"
Jul 28 17:25:16.691571 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:16.691534    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht"
Jul 28 17:25:23.064178 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:23.064145    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht"]
Jul 28 17:25:26.691697 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:26.691657    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht" podUID="7a311593-adff-4532-92d3-68549e2d6193" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 17:25:36.691766 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:36.691723    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht" podUID="7a311593-adff-4532-92d3-68549e2d6193" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 17:25:38.084631 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:38.084581    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht" podUID="7a311593-adff-4532-92d3-68549e2d6193" containerName="main" containerID="cri-o://ebdb498bdfb739f8c5c6198cd35b7cb3fe955753bf01f11e2a0840b02e219371" gracePeriod=45
Jul 28 17:25:38.335161 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:38.335077    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht"
Jul 28 17:25:38.453762 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:38.453712    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a311593-adff-4532-92d3-68549e2d6193-tls-certs\") pod \"7a311593-adff-4532-92d3-68549e2d6193\" (UID: \"7a311593-adff-4532-92d3-68549e2d6193\") "
Jul 28 17:25:38.453962 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:38.453793    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2f5p\" (UniqueName: \"kubernetes.io/projected/7a311593-adff-4532-92d3-68549e2d6193-kube-api-access-h2f5p\") pod \"7a311593-adff-4532-92d3-68549e2d6193\" (UID: \"7a311593-adff-4532-92d3-68549e2d6193\") "
Jul 28 17:25:38.455927 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:38.455893    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a311593-adff-4532-92d3-68549e2d6193-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7a311593-adff-4532-92d3-68549e2d6193" (UID: "7a311593-adff-4532-92d3-68549e2d6193"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:25:38.456048 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:38.455972    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a311593-adff-4532-92d3-68549e2d6193-kube-api-access-h2f5p" (OuterVolumeSpecName: "kube-api-access-h2f5p") pod "7a311593-adff-4532-92d3-68549e2d6193" (UID: "7a311593-adff-4532-92d3-68549e2d6193"). InnerVolumeSpecName "kube-api-access-h2f5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:25:38.555250 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:38.555205    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2f5p\" (UniqueName: \"kubernetes.io/projected/7a311593-adff-4532-92d3-68549e2d6193-kube-api-access-h2f5p\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:25:38.555250 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:38.555241    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a311593-adff-4532-92d3-68549e2d6193-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:25:38.927472 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:38.927433    2567 generic.go:356] "Generic (PLEG): container finished" podID="7a311593-adff-4532-92d3-68549e2d6193" containerID="ebdb498bdfb739f8c5c6198cd35b7cb3fe955753bf01f11e2a0840b02e219371" exitCode=0
Jul 28 17:25:38.927671 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:38.927477    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht"
Jul 28 17:25:38.927671 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:38.927538    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht" event={"ID":"7a311593-adff-4532-92d3-68549e2d6193","Type":"ContainerDied","Data":"ebdb498bdfb739f8c5c6198cd35b7cb3fe955753bf01f11e2a0840b02e219371"}
Jul 28 17:25:38.927671 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:38.927600    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht" event={"ID":"7a311593-adff-4532-92d3-68549e2d6193","Type":"ContainerDied","Data":"50d3d7ece1afc6345cb9a6b47e2b07e44c78ace06ca3cd9a392535cd13d77fe9"}
Jul 28 17:25:38.927671 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:38.927633    2567 scope.go:117] "RemoveContainer" containerID="ebdb498bdfb739f8c5c6198cd35b7cb3fe955753bf01f11e2a0840b02e219371"
Jul 28 17:25:38.936419 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:38.936401    2567 scope.go:117] "RemoveContainer" containerID="ebdb498bdfb739f8c5c6198cd35b7cb3fe955753bf01f11e2a0840b02e219371"
Jul 28 17:25:38.936717 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:25:38.936698    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebdb498bdfb739f8c5c6198cd35b7cb3fe955753bf01f11e2a0840b02e219371\": container with ID starting with ebdb498bdfb739f8c5c6198cd35b7cb3fe955753bf01f11e2a0840b02e219371 not found: ID does not exist" containerID="ebdb498bdfb739f8c5c6198cd35b7cb3fe955753bf01f11e2a0840b02e219371"
Jul 28 17:25:38.936788 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:38.936727    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebdb498bdfb739f8c5c6198cd35b7cb3fe955753bf01f11e2a0840b02e219371"} err="failed to get container status \"ebdb498bdfb739f8c5c6198cd35b7cb3fe955753bf01f11e2a0840b02e219371\": rpc error: code = NotFound desc = could not find container \"ebdb498bdfb739f8c5c6198cd35b7cb3fe955753bf01f11e2a0840b02e219371\": container with ID starting with ebdb498bdfb739f8c5c6198cd35b7cb3fe955753bf01f11e2a0840b02e219371 not found: ID does not exist"
Jul 28 17:25:38.950607 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:38.950570    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht"]
Jul 28 17:25:38.954424 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:38.954392    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-nzpht"]
Jul 28 17:25:39.286697 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:39.286666    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a311593-adff-4532-92d3-68549e2d6193" path="/var/lib/kubelet/pods/7a311593-adff-4532-92d3-68549e2d6193/volumes"
Jul 28 17:25:43.196889 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.196852    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"]
Jul 28 17:25:43.197266 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.197227    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a311593-adff-4532-92d3-68549e2d6193" containerName="main"
Jul 28 17:25:43.197266 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.197241    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a311593-adff-4532-92d3-68549e2d6193" containerName="main"
Jul 28 17:25:43.197343 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.197292    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a311593-adff-4532-92d3-68549e2d6193" containerName="main"
Jul 28 17:25:43.200075 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.200048    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.203311 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.203283    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"openshift-service-ca.crt\""
Jul 28 17:25:43.203311 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.203283    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"router-gateway-1-openshift-default-dockercfg-hw4v2\""
Jul 28 17:25:43.203543 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.203391    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"openshift-gw-ca-root-cert\""
Jul 28 17:25:43.203543 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.203437    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"kube-root-ca.crt\""
Jul 28 17:25:43.218019 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.217990    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"]
Jul 28 17:25:43.294750 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.294716    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2821fe2a-b965-4079-a276-7c91eacc4a2e-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.294927 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.294755    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6tvp\" (UniqueName: \"kubernetes.io/projected/2821fe2a-b965-4079-a276-7c91eacc4a2e-kube-api-access-n6tvp\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.294927 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.294780    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-workload-socket\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.294927 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.294838    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-token\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.294927 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.294885    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-workload-certs\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.295074 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.294944    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-credential-socket\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.295074 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.294962    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-envoy\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.295074 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.294978    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-podinfo\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.295074 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.294997    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-data\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.395474 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.395431    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6tvp\" (UniqueName: \"kubernetes.io/projected/2821fe2a-b965-4079-a276-7c91eacc4a2e-kube-api-access-n6tvp\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.395709 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.395485    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-workload-socket\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.395709 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.395685    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-token\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.395840 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.395775    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-workload-certs\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.395935 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.395918    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-credential-socket\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.396045 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.396019    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-envoy\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.396045 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.395961    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-workload-socket\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.396235 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.396053    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-podinfo\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.396235 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.396078    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-data\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.396235 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.396182    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2821fe2a-b965-4079-a276-7c91eacc4a2e-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.396235 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.396188    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-workload-certs\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.396426 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.396260    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-credential-socket\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.396495 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.396469    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-data\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.396933 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.396907    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2821fe2a-b965-4079-a276-7c91eacc4a2e-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.398825 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.398796    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-envoy\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.399067 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.399048    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-podinfo\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.404807 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.404779    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6tvp\" (UniqueName: \"kubernetes.io/projected/2821fe2a-b965-4079-a276-7c91eacc4a2e-kube-api-access-n6tvp\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.404951 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.404826    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-token\") pod \"router-gateway-1-openshift-default-5b4d9458bd-nbgb9\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.510647 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.510606    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:43.661357 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.661331    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"]
Jul 28 17:25:43.663559 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:25:43.663500    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2821fe2a_b965_4079_a276_7c91eacc4a2e.slice/crio-1bfd518f54849b51dc9db17127f82eb6465e9c616aa0aa38847d3f01a7da18da WatchSource:0}: Error finding container 1bfd518f54849b51dc9db17127f82eb6465e9c616aa0aa38847d3f01a7da18da: Status 404 returned error can't find the container with id 1bfd518f54849b51dc9db17127f82eb6465e9c616aa0aa38847d3f01a7da18da
Jul 28 17:25:43.665562 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.665525    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 17:25:43.665666 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.665609    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 17:25:43.665666 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.665646    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 17:25:43.953542 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.953419    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9" event={"ID":"2821fe2a-b965-4079-a276-7c91eacc4a2e","Type":"ContainerStarted","Data":"8c32f0ee3a30ae5b2868d48f6c86e0ebba469f57aba41f99a1ec423284369a45"}
Jul 28 17:25:43.953542 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.953492    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9" event={"ID":"2821fe2a-b965-4079-a276-7c91eacc4a2e","Type":"ContainerStarted","Data":"1bfd518f54849b51dc9db17127f82eb6465e9c616aa0aa38847d3f01a7da18da"}
Jul 28 17:25:43.980307 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:43.980231    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9" podStartSLOduration=0.980209379 podStartE2EDuration="980.209379ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:25:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:25:43.976189826 +0000 UTC m=+899.222292033" watchObservedRunningTime="2026-07-28 17:25:43.980209379 +0000 UTC m=+899.226311587"
Jul 28 17:25:44.511604 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:44.511559    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:44.517039 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:44.517006    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:44.958457 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:44.958367    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:44.959384 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:44.959364    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:25:45.979266 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:45.979240    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:25:45.981121 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:45.981096    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:25:54.173517 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.173441    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"]
Jul 28 17:25:54.179103 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.179062    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.189473 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.185333    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"kube-root-ca.crt\""
Jul 28 17:25:54.189473 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.185659    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"openshift-service-ca.crt\""
Jul 28 17:25:54.189473 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.185899    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"openshift-gw-ca-root-cert\""
Jul 28 17:25:54.189473 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.186192    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"router-gateway-1-openshift-default-dockercfg-pxbgr\""
Jul 28 17:25:54.207538 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.207422    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"]
Jul 28 17:25:54.218165 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.217702    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-token\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.218165 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.217746    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.218165 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.217778    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-podinfo\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.218165 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.217812    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-workload-socket\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.218165 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.217856    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-credential-socket\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.218165 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.217888    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-workload-certs\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.218165 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.217914    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-data\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.218165 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.217953    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rppkv\" (UniqueName: \"kubernetes.io/projected/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-kube-api-access-rppkv\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.218165 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.217991    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-envoy\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.318271 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.318222    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rppkv\" (UniqueName: \"kubernetes.io/projected/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-kube-api-access-rppkv\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.318271 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.318273    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-envoy\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.318593 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.318301    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-token\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.318593 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.318315    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.318593 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.318333    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-podinfo\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.318593 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.318354    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-workload-socket\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.318593 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.318390    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-credential-socket\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.318873 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.318676    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-workload-certs\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.318873 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.318710    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-data\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.318873 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.318774    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-credential-socket\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.319158 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.319074    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-workload-certs\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.319158 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.319078    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-data\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.319318 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.319242    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.319375 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.319310    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-workload-socket\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.321014 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.320991    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-podinfo\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.322084 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.322066    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-envoy\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.334208 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.334174    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rppkv\" (UniqueName: \"kubernetes.io/projected/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-kube-api-access-rppkv\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.334462 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.334439    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-token\") pod \"router-gateway-1-openshift-default-66bbccc5fb-k2dbv\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.518586 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.518548    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:54.688323 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.686455    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"]
Jul 28 17:25:54.715232 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:25:54.715194    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96bbdcc3_c7d9_4910_8b69_69f6dcf75277.slice/crio-7b745afc828cd78da616737ffc1f7ac858e160236f7eacf40d29ae2d67935020 WatchSource:0}: Error finding container 7b745afc828cd78da616737ffc1f7ac858e160236f7eacf40d29ae2d67935020: Status 404 returned error can't find the container with id 7b745afc828cd78da616737ffc1f7ac858e160236f7eacf40d29ae2d67935020
Jul 28 17:25:54.717244 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.717207    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 17:25:54.717344 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.717286    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 17:25:54.717344 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:54.717324    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 17:25:55.013965 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:55.013920    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv" event={"ID":"96bbdcc3-c7d9-4910-8b69-69f6dcf75277","Type":"ContainerStarted","Data":"ad0d5c0bb13c9222ab4805e9c4e6e43b8cd86a72de9a6758dd665fe47884f9c0"}
Jul 28 17:25:55.014172 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:55.013980    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv" event={"ID":"96bbdcc3-c7d9-4910-8b69-69f6dcf75277","Type":"ContainerStarted","Data":"7b745afc828cd78da616737ffc1f7ac858e160236f7eacf40d29ae2d67935020"}
Jul 28 17:25:55.038485 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:55.038426    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv" podStartSLOduration=1.038406952 podStartE2EDuration="1.038406952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:25:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:25:55.036803523 +0000 UTC m=+910.282905765" watchObservedRunningTime="2026-07-28 17:25:55.038406952 +0000 UTC m=+910.284509158"
Jul 28 17:25:55.519411 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:55.519368    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:55.524704 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:55.524658    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:56.019920 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:56.019895    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:56.021123 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:56.021097    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:25:58.869189 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:58.869143    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"]
Jul 28 17:25:58.869717 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:25:58.869433    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9" podUID="2821fe2a-b965-4079-a276-7c91eacc4a2e" containerName="istio-proxy" containerID="cri-o://8c32f0ee3a30ae5b2868d48f6c86e0ebba469f57aba41f99a1ec423284369a45" gracePeriod=30
Jul 28 17:26:00.999750 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:00.999690    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9" podUID="2821fe2a-b965-4079-a276-7c91eacc4a2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 17:26:02.052658 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.052621    2567 generic.go:356] "Generic (PLEG): container finished" podID="2821fe2a-b965-4079-a276-7c91eacc4a2e" containerID="8c32f0ee3a30ae5b2868d48f6c86e0ebba469f57aba41f99a1ec423284369a45" exitCode=0
Jul 28 17:26:02.053079 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.052691    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9" event={"ID":"2821fe2a-b965-4079-a276-7c91eacc4a2e","Type":"ContainerDied","Data":"8c32f0ee3a30ae5b2868d48f6c86e0ebba469f57aba41f99a1ec423284369a45"}
Jul 28 17:26:02.134786 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.134758    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:26:02.192181 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.192073    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-workload-socket\") pod \"2821fe2a-b965-4079-a276-7c91eacc4a2e\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") "
Jul 28 17:26:02.192181 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.192140    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-podinfo\") pod \"2821fe2a-b965-4079-a276-7c91eacc4a2e\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") "
Jul 28 17:26:02.192404 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.192189    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6tvp\" (UniqueName: \"kubernetes.io/projected/2821fe2a-b965-4079-a276-7c91eacc4a2e-kube-api-access-n6tvp\") pod \"2821fe2a-b965-4079-a276-7c91eacc4a2e\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") "
Jul 28 17:26:02.192404 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.192215    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-token\") pod \"2821fe2a-b965-4079-a276-7c91eacc4a2e\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") "
Jul 28 17:26:02.192404 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.192246    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2821fe2a-b965-4079-a276-7c91eacc4a2e-istiod-ca-cert\") pod \"2821fe2a-b965-4079-a276-7c91eacc4a2e\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") "
Jul 28 17:26:02.192404 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.192268    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-data\") pod \"2821fe2a-b965-4079-a276-7c91eacc4a2e\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") "
Jul 28 17:26:02.192404 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.192294    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-workload-certs\") pod \"2821fe2a-b965-4079-a276-7c91eacc4a2e\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") "
Jul 28 17:26:02.192404 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.192363    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2821fe2a-b965-4079-a276-7c91eacc4a2e" (UID: "2821fe2a-b965-4079-a276-7c91eacc4a2e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:26:02.192404 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.192377    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-envoy\") pod \"2821fe2a-b965-4079-a276-7c91eacc4a2e\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") "
Jul 28 17:26:02.192774 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.192457    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-credential-socket\") pod \"2821fe2a-b965-4079-a276-7c91eacc4a2e\" (UID: \"2821fe2a-b965-4079-a276-7c91eacc4a2e\") "
Jul 28 17:26:02.192774 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.192552    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-data" (OuterVolumeSpecName: "istio-data") pod "2821fe2a-b965-4079-a276-7c91eacc4a2e" (UID: "2821fe2a-b965-4079-a276-7c91eacc4a2e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:26:02.192774 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.192719    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2821fe2a-b965-4079-a276-7c91eacc4a2e" (UID: "2821fe2a-b965-4079-a276-7c91eacc4a2e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:26:02.192774 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.192719    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2821fe2a-b965-4079-a276-7c91eacc4a2e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2821fe2a-b965-4079-a276-7c91eacc4a2e" (UID: "2821fe2a-b965-4079-a276-7c91eacc4a2e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:26:02.192979 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.192790    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-credential-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:26:02.192979 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.192810    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-workload-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:26:02.192979 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.192826    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2821fe2a-b965-4079-a276-7c91eacc4a2e-istiod-ca-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:26:02.192979 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.192837    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-data\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:26:02.192979 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.192863    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2821fe2a-b965-4079-a276-7c91eacc4a2e" (UID: "2821fe2a-b965-4079-a276-7c91eacc4a2e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:26:02.195053 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.195017    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2821fe2a-b965-4079-a276-7c91eacc4a2e" (UID: "2821fe2a-b965-4079-a276-7c91eacc4a2e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:26:02.195204 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.195102    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2821fe2a-b965-4079-a276-7c91eacc4a2e" (UID: "2821fe2a-b965-4079-a276-7c91eacc4a2e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 17:26:02.195275 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.195253    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2821fe2a-b965-4079-a276-7c91eacc4a2e-kube-api-access-n6tvp" (OuterVolumeSpecName: "kube-api-access-n6tvp") pod "2821fe2a-b965-4079-a276-7c91eacc4a2e" (UID: "2821fe2a-b965-4079-a276-7c91eacc4a2e"). InnerVolumeSpecName "kube-api-access-n6tvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:26:02.195331 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.195292    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-token" (OuterVolumeSpecName: "istio-token") pod "2821fe2a-b965-4079-a276-7c91eacc4a2e" (UID: "2821fe2a-b965-4079-a276-7c91eacc4a2e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:26:02.293252 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.293201    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-envoy\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:26:02.293252 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.293239    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-podinfo\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:26:02.293252 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.293249    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6tvp\" (UniqueName: \"kubernetes.io/projected/2821fe2a-b965-4079-a276-7c91eacc4a2e-kube-api-access-n6tvp\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:26:02.293252 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.293258    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2821fe2a-b965-4079-a276-7c91eacc4a2e-istio-token\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:26:02.293252 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:02.293269    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2821fe2a-b965-4079-a276-7c91eacc4a2e-workload-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:26:03.058866 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:03.058826    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9" event={"ID":"2821fe2a-b965-4079-a276-7c91eacc4a2e","Type":"ContainerDied","Data":"1bfd518f54849b51dc9db17127f82eb6465e9c616aa0aa38847d3f01a7da18da"}
Jul 28 17:26:03.058866 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:03.058845    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"
Jul 28 17:26:03.059379 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:03.058899    2567 scope.go:117] "RemoveContainer" containerID="8c32f0ee3a30ae5b2868d48f6c86e0ebba469f57aba41f99a1ec423284369a45"
Jul 28 17:26:03.089352 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:03.089302    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"]
Jul 28 17:26:03.104436 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:03.104387    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-5b4d9458bd-nbgb9"]
Jul 28 17:26:03.287424 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:03.287380    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2821fe2a-b965-4079-a276-7c91eacc4a2e" path="/var/lib/kubelet/pods/2821fe2a-b965-4079-a276-7c91eacc4a2e/volumes"
Jul 28 17:26:19.381972 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:19.381928    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"]
Jul 28 17:26:19.382457 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:19.382193    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv" podUID="96bbdcc3-c7d9-4910-8b69-69f6dcf75277" containerName="istio-proxy" containerID="cri-o://ad0d5c0bb13c9222ab4805e9c4e6e43b8cd86a72de9a6758dd665fe47884f9c0" gracePeriod=30
Jul 28 17:26:22.336340 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.336256    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:26:22.480701 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.480651    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-podinfo\") pod \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") "
Jul 28 17:26:22.480701 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.480701    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-workload-certs\") pod \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") "
Jul 28 17:26:22.480961 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.480723    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-workload-socket\") pod \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") "
Jul 28 17:26:22.480961 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.480754    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rppkv\" (UniqueName: \"kubernetes.io/projected/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-kube-api-access-rppkv\") pod \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") "
Jul 28 17:26:22.480961 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.480877    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-envoy\") pod \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") "
Jul 28 17:26:22.480961 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.480960    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istiod-ca-cert\") pod \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") "
Jul 28 17:26:22.481163 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.481001    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-token\") pod \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") "
Jul 28 17:26:22.481163 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.481020    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "96bbdcc3-c7d9-4910-8b69-69f6dcf75277" (UID: "96bbdcc3-c7d9-4910-8b69-69f6dcf75277"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:26:22.481163 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.481039    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-data\") pod \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") "
Jul 28 17:26:22.481163 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.481061    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "96bbdcc3-c7d9-4910-8b69-69f6dcf75277" (UID: "96bbdcc3-c7d9-4910-8b69-69f6dcf75277"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:26:22.481163 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.481081    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-credential-socket\") pod \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\" (UID: \"96bbdcc3-c7d9-4910-8b69-69f6dcf75277\") "
Jul 28 17:26:22.481408 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.481328    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-workload-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:26:22.481408 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.481346    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-workload-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:26:22.481408 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.481337    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "96bbdcc3-c7d9-4910-8b69-69f6dcf75277" (UID: "96bbdcc3-c7d9-4910-8b69-69f6dcf75277"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:26:22.481602 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.481409    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-data" (OuterVolumeSpecName: "istio-data") pod "96bbdcc3-c7d9-4910-8b69-69f6dcf75277" (UID: "96bbdcc3-c7d9-4910-8b69-69f6dcf75277"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:26:22.481602 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.481568    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "96bbdcc3-c7d9-4910-8b69-69f6dcf75277" (UID: "96bbdcc3-c7d9-4910-8b69-69f6dcf75277"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:26:22.483376 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.483337    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-token" (OuterVolumeSpecName: "istio-token") pod "96bbdcc3-c7d9-4910-8b69-69f6dcf75277" (UID: "96bbdcc3-c7d9-4910-8b69-69f6dcf75277"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:26:22.483376 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.483348    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-kube-api-access-rppkv" (OuterVolumeSpecName: "kube-api-access-rppkv") pod "96bbdcc3-c7d9-4910-8b69-69f6dcf75277" (UID: "96bbdcc3-c7d9-4910-8b69-69f6dcf75277"). InnerVolumeSpecName "kube-api-access-rppkv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:26:22.483605 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.483362    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "96bbdcc3-c7d9-4910-8b69-69f6dcf75277" (UID: "96bbdcc3-c7d9-4910-8b69-69f6dcf75277"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:26:22.483605 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.483382    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "96bbdcc3-c7d9-4910-8b69-69f6dcf75277" (UID: "96bbdcc3-c7d9-4910-8b69-69f6dcf75277"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 17:26:22.582791 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.582740    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istiod-ca-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:26:22.582791 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.582780    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-token\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:26:22.582791 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.582791    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-data\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:26:22.582791 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.582799    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-credential-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:26:22.582791 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.582808    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-podinfo\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:26:22.583190 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.582818    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rppkv\" (UniqueName: \"kubernetes.io/projected/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-kube-api-access-rppkv\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:26:22.583190 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:22.582827    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96bbdcc3-c7d9-4910-8b69-69f6dcf75277-istio-envoy\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:26:23.155666 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:23.155622    2567 generic.go:356] "Generic (PLEG): container finished" podID="96bbdcc3-c7d9-4910-8b69-69f6dcf75277" containerID="ad0d5c0bb13c9222ab4805e9c4e6e43b8cd86a72de9a6758dd665fe47884f9c0" exitCode=0
Jul 28 17:26:23.155666 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:23.155678    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"
Jul 28 17:26:23.155903 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:23.155712    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv" event={"ID":"96bbdcc3-c7d9-4910-8b69-69f6dcf75277","Type":"ContainerDied","Data":"ad0d5c0bb13c9222ab4805e9c4e6e43b8cd86a72de9a6758dd665fe47884f9c0"}
Jul 28 17:26:23.155903 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:23.155770    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv" event={"ID":"96bbdcc3-c7d9-4910-8b69-69f6dcf75277","Type":"ContainerDied","Data":"7b745afc828cd78da616737ffc1f7ac858e160236f7eacf40d29ae2d67935020"}
Jul 28 17:26:23.155903 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:23.155798    2567 scope.go:117] "RemoveContainer" containerID="ad0d5c0bb13c9222ab4805e9c4e6e43b8cd86a72de9a6758dd665fe47884f9c0"
Jul 28 17:26:23.169332 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:23.165597    2567 scope.go:117] "RemoveContainer" containerID="ad0d5c0bb13c9222ab4805e9c4e6e43b8cd86a72de9a6758dd665fe47884f9c0"
Jul 28 17:26:23.169332 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:26:23.166046    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad0d5c0bb13c9222ab4805e9c4e6e43b8cd86a72de9a6758dd665fe47884f9c0\": container with ID starting with ad0d5c0bb13c9222ab4805e9c4e6e43b8cd86a72de9a6758dd665fe47884f9c0 not found: ID does not exist" containerID="ad0d5c0bb13c9222ab4805e9c4e6e43b8cd86a72de9a6758dd665fe47884f9c0"
Jul 28 17:26:23.169332 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:23.166095    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad0d5c0bb13c9222ab4805e9c4e6e43b8cd86a72de9a6758dd665fe47884f9c0"} err="failed to get container status \"ad0d5c0bb13c9222ab4805e9c4e6e43b8cd86a72de9a6758dd665fe47884f9c0\": rpc error: code = NotFound desc = could not find container \"ad0d5c0bb13c9222ab4805e9c4e6e43b8cd86a72de9a6758dd665fe47884f9c0\": container with ID starting with ad0d5c0bb13c9222ab4805e9c4e6e43b8cd86a72de9a6758dd665fe47884f9c0 not found: ID does not exist"
Jul 28 17:26:23.180662 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:23.180620    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"]
Jul 28 17:26:23.186544 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:23.186491    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-66bbccc5fb-k2dbv"]
Jul 28 17:26:23.286937 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:23.286902    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96bbdcc3-c7d9-4910-8b69-69f6dcf75277" path="/var/lib/kubelet/pods/96bbdcc3-c7d9-4910-8b69-69f6dcf75277/volumes"
Jul 28 17:26:35.638040 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:35.638000    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds"]
Jul 28 17:26:35.638566 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:35.638436    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96bbdcc3-c7d9-4910-8b69-69f6dcf75277" containerName="istio-proxy"
Jul 28 17:26:35.638566 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:35.638457    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="96bbdcc3-c7d9-4910-8b69-69f6dcf75277" containerName="istio-proxy"
Jul 28 17:26:35.638566 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:35.638485    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2821fe2a-b965-4079-a276-7c91eacc4a2e" containerName="istio-proxy"
Jul 28 17:26:35.638566 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:35.638493    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2821fe2a-b965-4079-a276-7c91eacc4a2e" containerName="istio-proxy"
Jul 28 17:26:35.638775 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:35.638580    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2821fe2a-b965-4079-a276-7c91eacc4a2e" containerName="istio-proxy"
Jul 28 17:26:35.638775 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:35.638591    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="96bbdcc3-c7d9-4910-8b69-69f6dcf75277" containerName="istio-proxy"
Jul 28 17:26:35.642206 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:35.642176    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds"
Jul 28 17:26:35.647174 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:35.647138    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"auth-enabled-test-kserve-self-signed-certs\""
Jul 28 17:26:35.647375 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:35.647181    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"auth-enabled-test-epp-sa-dockercfg-cltk9\""
Jul 28 17:26:35.647375 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:35.647154    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"default-dockercfg-29hj4\""
Jul 28 17:26:35.647375 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:35.647211    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"kube-root-ca.crt\""
Jul 28 17:26:35.647375 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:35.647138    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"openshift-service-ca.crt\""
Jul 28 17:26:35.652041 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:35.652010    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds"]
Jul 28 17:26:35.790141 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:35.790083    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52d7c93a-6af5-4175-871a-3a4d61c08957-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds\" (UID: \"52d7c93a-6af5-4175-871a-3a4d61c08957\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds"
Jul 28 17:26:35.790343 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:35.790162    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcj5z\" (UniqueName: \"kubernetes.io/projected/52d7c93a-6af5-4175-871a-3a4d61c08957-kube-api-access-vcj5z\") pod \"auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds\" (UID: \"52d7c93a-6af5-4175-871a-3a4d61c08957\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds"
Jul 28 17:26:35.891609 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:35.891446    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcj5z\" (UniqueName: \"kubernetes.io/projected/52d7c93a-6af5-4175-871a-3a4d61c08957-kube-api-access-vcj5z\") pod \"auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds\" (UID: \"52d7c93a-6af5-4175-871a-3a4d61c08957\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds"
Jul 28 17:26:35.891909 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:35.891849    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52d7c93a-6af5-4175-871a-3a4d61c08957-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds\" (UID: \"52d7c93a-6af5-4175-871a-3a4d61c08957\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds"
Jul 28 17:26:35.894556 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:35.894525    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52d7c93a-6af5-4175-871a-3a4d61c08957-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds\" (UID: \"52d7c93a-6af5-4175-871a-3a4d61c08957\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds"
Jul 28 17:26:35.901832 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:35.901783    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcj5z\" (UniqueName: \"kubernetes.io/projected/52d7c93a-6af5-4175-871a-3a4d61c08957-kube-api-access-vcj5z\") pod \"auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds\" (UID: \"52d7c93a-6af5-4175-871a-3a4d61c08957\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds"
Jul 28 17:26:35.956169 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:35.956123    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds"
Jul 28 17:26:36.105832 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:36.105801    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds"]
Jul 28 17:26:36.108358 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:26:36.108325    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52d7c93a_6af5_4175_871a_3a4d61c08957.slice/crio-97c2ddf0305f686ecdc01603426f4bf2ca4193af1eed3c6ae69f63c186381ccd WatchSource:0}: Error finding container 97c2ddf0305f686ecdc01603426f4bf2ca4193af1eed3c6ae69f63c186381ccd: Status 404 returned error can't find the container with id 97c2ddf0305f686ecdc01603426f4bf2ca4193af1eed3c6ae69f63c186381ccd
Jul 28 17:26:36.220602 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:36.220563    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds" event={"ID":"52d7c93a-6af5-4175-871a-3a4d61c08957","Type":"ContainerStarted","Data":"1216bf84d2c2a32bf2b2920c7fa68483b5b83f90900104f7a6abbe36f9ecf764"}
Jul 28 17:26:36.220814 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:36.220631    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds" event={"ID":"52d7c93a-6af5-4175-871a-3a4d61c08957","Type":"ContainerStarted","Data":"97c2ddf0305f686ecdc01603426f4bf2ca4193af1eed3c6ae69f63c186381ccd"}
Jul 28 17:26:36.220814 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:36.220673    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds"
Jul 28 17:26:36.237909 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:26:36.237838    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds" podStartSLOduration=1.237819095 podStartE2EDuration="1.237819095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:26:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:26:36.237469536 +0000 UTC m=+951.483571746" watchObservedRunningTime="2026-07-28 17:26:36.237819095 +0000 UTC m=+951.483921303"
Jul 28 17:27:07.228204 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:27:07.228120    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds"
Jul 28 17:30:10.458727 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:10.458495    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds"]
Jul 28 17:30:17.228289 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:17.228242    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds" podUID="52d7c93a-6af5-4175-871a-3a4d61c08957" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 17:30:25.477755 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:25.477686    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds" podUID="52d7c93a-6af5-4175-871a-3a4d61c08957" containerName="main" containerID="cri-o://1216bf84d2c2a32bf2b2920c7fa68483b5b83f90900104f7a6abbe36f9ecf764" gracePeriod=45
Jul 28 17:30:25.720665 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:25.720638    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds"
Jul 28 17:30:25.828322 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:25.828283    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcj5z\" (UniqueName: \"kubernetes.io/projected/52d7c93a-6af5-4175-871a-3a4d61c08957-kube-api-access-vcj5z\") pod \"52d7c93a-6af5-4175-871a-3a4d61c08957\" (UID: \"52d7c93a-6af5-4175-871a-3a4d61c08957\") "
Jul 28 17:30:25.828541 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:25.828408    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52d7c93a-6af5-4175-871a-3a4d61c08957-tls-certs\") pod \"52d7c93a-6af5-4175-871a-3a4d61c08957\" (UID: \"52d7c93a-6af5-4175-871a-3a4d61c08957\") "
Jul 28 17:30:25.830673 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:25.830637    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52d7c93a-6af5-4175-871a-3a4d61c08957-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "52d7c93a-6af5-4175-871a-3a4d61c08957" (UID: "52d7c93a-6af5-4175-871a-3a4d61c08957"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:30:25.830673 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:25.830645    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52d7c93a-6af5-4175-871a-3a4d61c08957-kube-api-access-vcj5z" (OuterVolumeSpecName: "kube-api-access-vcj5z") pod "52d7c93a-6af5-4175-871a-3a4d61c08957" (UID: "52d7c93a-6af5-4175-871a-3a4d61c08957"). InnerVolumeSpecName "kube-api-access-vcj5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:30:25.929176 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:25.929134    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52d7c93a-6af5-4175-871a-3a4d61c08957-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:30:25.929176 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:25.929170    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcj5z\" (UniqueName: \"kubernetes.io/projected/52d7c93a-6af5-4175-871a-3a4d61c08957-kube-api-access-vcj5z\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:30:26.299981 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:26.299948    2567 generic.go:356] "Generic (PLEG): container finished" podID="52d7c93a-6af5-4175-871a-3a4d61c08957" containerID="1216bf84d2c2a32bf2b2920c7fa68483b5b83f90900104f7a6abbe36f9ecf764" exitCode=0
Jul 28 17:30:26.300178 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:26.299987    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds"
Jul 28 17:30:26.300178 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:26.300030    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds" event={"ID":"52d7c93a-6af5-4175-871a-3a4d61c08957","Type":"ContainerDied","Data":"1216bf84d2c2a32bf2b2920c7fa68483b5b83f90900104f7a6abbe36f9ecf764"}
Jul 28 17:30:26.300178 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:26.300085    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds" event={"ID":"52d7c93a-6af5-4175-871a-3a4d61c08957","Type":"ContainerDied","Data":"97c2ddf0305f686ecdc01603426f4bf2ca4193af1eed3c6ae69f63c186381ccd"}
Jul 28 17:30:26.300178 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:26.300108    2567 scope.go:117] "RemoveContainer" containerID="1216bf84d2c2a32bf2b2920c7fa68483b5b83f90900104f7a6abbe36f9ecf764"
Jul 28 17:30:26.308733 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:26.308714    2567 scope.go:117] "RemoveContainer" containerID="1216bf84d2c2a32bf2b2920c7fa68483b5b83f90900104f7a6abbe36f9ecf764"
Jul 28 17:30:26.309015 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:30:26.308996    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1216bf84d2c2a32bf2b2920c7fa68483b5b83f90900104f7a6abbe36f9ecf764\": container with ID starting with 1216bf84d2c2a32bf2b2920c7fa68483b5b83f90900104f7a6abbe36f9ecf764 not found: ID does not exist" containerID="1216bf84d2c2a32bf2b2920c7fa68483b5b83f90900104f7a6abbe36f9ecf764"
Jul 28 17:30:26.309053 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:26.309027    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1216bf84d2c2a32bf2b2920c7fa68483b5b83f90900104f7a6abbe36f9ecf764"} err="failed to get container status \"1216bf84d2c2a32bf2b2920c7fa68483b5b83f90900104f7a6abbe36f9ecf764\": rpc error: code = NotFound desc = could not find container \"1216bf84d2c2a32bf2b2920c7fa68483b5b83f90900104f7a6abbe36f9ecf764\": container with ID starting with 1216bf84d2c2a32bf2b2920c7fa68483b5b83f90900104f7a6abbe36f9ecf764 not found: ID does not exist"
Jul 28 17:30:26.323473 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:26.323445    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds"]
Jul 28 17:30:26.327125 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:26.327101    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-2ftds"]
Jul 28 17:30:27.286707 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:27.286673    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52d7c93a-6af5-4175-871a-3a4d61c08957" path="/var/lib/kubelet/pods/52d7c93a-6af5-4175-871a-3a4d61c08957/volumes"
Jul 28 17:30:43.982765 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:43.982731    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q"]
Jul 28 17:30:43.983308 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:43.983097    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52d7c93a-6af5-4175-871a-3a4d61c08957" containerName="main"
Jul 28 17:30:43.983308 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:43.983111    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="52d7c93a-6af5-4175-871a-3a4d61c08957" containerName="main"
Jul 28 17:30:43.983308 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:43.983231    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="52d7c93a-6af5-4175-871a-3a4d61c08957" containerName="main"
Jul 28 17:30:43.985394 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:43.985372    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q"
Jul 28 17:30:43.990432 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:43.990404    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"default-dockercfg-5svxx\""
Jul 28 17:30:43.990588 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:43.990404    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"auth-invalid-token-test-epp-sa-dockercfg-cqmzx\""
Jul 28 17:30:43.990588 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:43.990403    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"kube-root-ca.crt\""
Jul 28 17:30:43.990588 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:43.990415    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"openshift-service-ca.crt\""
Jul 28 17:30:43.990588 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:43.990490    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"auth-invalid-token-test-kserve-self-signed-certs\""
Jul 28 17:30:44.000802 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:44.000775    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q"]
Jul 28 17:30:44.077079 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:44.077043    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74660d04-9529-4f20-b4a8-f877de2307e7-tls-certs\") pod \"auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q\" (UID: \"74660d04-9529-4f20-b4a8-f877de2307e7\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q"
Jul 28 17:30:44.077258 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:44.077091    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk6p9\" (UniqueName: \"kubernetes.io/projected/74660d04-9529-4f20-b4a8-f877de2307e7-kube-api-access-zk6p9\") pod \"auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q\" (UID: \"74660d04-9529-4f20-b4a8-f877de2307e7\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q"
Jul 28 17:30:44.178398 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:44.178348    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74660d04-9529-4f20-b4a8-f877de2307e7-tls-certs\") pod \"auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q\" (UID: \"74660d04-9529-4f20-b4a8-f877de2307e7\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q"
Jul 28 17:30:44.178398 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:44.178402    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk6p9\" (UniqueName: \"kubernetes.io/projected/74660d04-9529-4f20-b4a8-f877de2307e7-kube-api-access-zk6p9\") pod \"auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q\" (UID: \"74660d04-9529-4f20-b4a8-f877de2307e7\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q"
Jul 28 17:30:44.180891 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:44.180861    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74660d04-9529-4f20-b4a8-f877de2307e7-tls-certs\") pod \"auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q\" (UID: \"74660d04-9529-4f20-b4a8-f877de2307e7\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q"
Jul 28 17:30:44.189051 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:44.189025    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk6p9\" (UniqueName: \"kubernetes.io/projected/74660d04-9529-4f20-b4a8-f877de2307e7-kube-api-access-zk6p9\") pod \"auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q\" (UID: \"74660d04-9529-4f20-b4a8-f877de2307e7\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q"
Jul 28 17:30:44.296400 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:44.296362    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q"
Jul 28 17:30:44.433934 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:44.433910    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q"]
Jul 28 17:30:44.435518 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:30:44.435482    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74660d04_9529_4f20_b4a8_f877de2307e7.slice/crio-94d040bb5ab65d2b130bd20e645d9b59bb7439fdc23a6574616382e4d9641f8e WatchSource:0}: Error finding container 94d040bb5ab65d2b130bd20e645d9b59bb7439fdc23a6574616382e4d9641f8e: Status 404 returned error can't find the container with id 94d040bb5ab65d2b130bd20e645d9b59bb7439fdc23a6574616382e4d9641f8e
Jul 28 17:30:45.395823 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:45.395786    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q" event={"ID":"74660d04-9529-4f20-b4a8-f877de2307e7","Type":"ContainerStarted","Data":"49d42dd9fcb70c46883333f99ba1d55c0c6cef1fa608c51ef016f8d8023ac15a"}
Jul 28 17:30:45.396274 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:45.395865    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q"
Jul 28 17:30:45.396274 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:45.395881    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q" event={"ID":"74660d04-9529-4f20-b4a8-f877de2307e7","Type":"ContainerStarted","Data":"94d040bb5ab65d2b130bd20e645d9b59bb7439fdc23a6574616382e4d9641f8e"}
Jul 28 17:30:45.417328 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:45.417273    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q" podStartSLOduration=2.41725537 podStartE2EDuration="2.41725537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:30:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:30:45.415453832 +0000 UTC m=+1200.661556039" watchObservedRunningTime="2026-07-28 17:30:45.41725537 +0000 UTC m=+1200.663357629"
Jul 28 17:30:46.016105 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:46.016071    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:30:46.018131 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:30:46.018102    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:31:16.403764 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:31:16.403727    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q"
Jul 28 17:32:18.272729 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:18.272687    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn"]
Jul 28 17:32:18.275084 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:18.275056    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn"
Jul 28 17:32:18.279847 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:18.279813    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"default-dockercfg-5g8rv\""
Jul 28 17:32:18.280022 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:18.279863    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"kube-root-ca.crt\""
Jul 28 17:32:18.280022 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:18.279814    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs\""
Jul 28 17:32:18.280022 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:18.279864    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-sa-dockercfg-2gsgd\""
Jul 28 17:32:18.280022 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:18.279813    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"openshift-service-ca.crt\""
Jul 28 17:32:18.287291 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:18.287258    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn"]
Jul 28 17:32:18.384338 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:18.384284    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76fd0ec9-874b-4ebc-bed7-1ad3ac608d72-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn\" (UID: \"76fd0ec9-874b-4ebc-bed7-1ad3ac608d72\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn"
Jul 28 17:32:18.384572 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:18.384410    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8bl7\" (UniqueName: \"kubernetes.io/projected/76fd0ec9-874b-4ebc-bed7-1ad3ac608d72-kube-api-access-j8bl7\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn\" (UID: \"76fd0ec9-874b-4ebc-bed7-1ad3ac608d72\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn"
Jul 28 17:32:18.485138 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:18.485083    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8bl7\" (UniqueName: \"kubernetes.io/projected/76fd0ec9-874b-4ebc-bed7-1ad3ac608d72-kube-api-access-j8bl7\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn\" (UID: \"76fd0ec9-874b-4ebc-bed7-1ad3ac608d72\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn"
Jul 28 17:32:18.485329 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:18.485224    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76fd0ec9-874b-4ebc-bed7-1ad3ac608d72-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn\" (UID: \"76fd0ec9-874b-4ebc-bed7-1ad3ac608d72\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn"
Jul 28 17:32:18.487880 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:18.487843    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76fd0ec9-874b-4ebc-bed7-1ad3ac608d72-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn\" (UID: \"76fd0ec9-874b-4ebc-bed7-1ad3ac608d72\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn"
Jul 28 17:32:18.505656 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:18.505625    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8bl7\" (UniqueName: \"kubernetes.io/projected/76fd0ec9-874b-4ebc-bed7-1ad3ac608d72-kube-api-access-j8bl7\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn\" (UID: \"76fd0ec9-874b-4ebc-bed7-1ad3ac608d72\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn"
Jul 28 17:32:18.587454 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:18.587356    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn"
Jul 28 17:32:18.744005 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:32:18.743968    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76fd0ec9_874b_4ebc_bed7_1ad3ac608d72.slice/crio-230f868f05cbd3f10f519fb8f7cd3ef43bcda9b7e51e6bd86a62fe2c80438ed3 WatchSource:0}: Error finding container 230f868f05cbd3f10f519fb8f7cd3ef43bcda9b7e51e6bd86a62fe2c80438ed3: Status 404 returned error can't find the container with id 230f868f05cbd3f10f519fb8f7cd3ef43bcda9b7e51e6bd86a62fe2c80438ed3
Jul 28 17:32:18.753365 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:18.753284    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn"]
Jul 28 17:32:18.853998 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:18.853895    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn" event={"ID":"76fd0ec9-874b-4ebc-bed7-1ad3ac608d72","Type":"ContainerStarted","Data":"6778d4c479198b5b4f70d52699dac52cd73ae4cabbbe982b2ac2fcb6cf84c513"}
Jul 28 17:32:18.853998 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:18.853955    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn" event={"ID":"76fd0ec9-874b-4ebc-bed7-1ad3ac608d72","Type":"ContainerStarted","Data":"230f868f05cbd3f10f519fb8f7cd3ef43bcda9b7e51e6bd86a62fe2c80438ed3"}
Jul 28 17:32:18.854239 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:18.854093    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn"
Jul 28 17:32:18.873019 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:18.872960    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn" podStartSLOduration=0.872941311 podStartE2EDuration="872.941311ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:32:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:32:18.871883459 +0000 UTC m=+1294.117985666" watchObservedRunningTime="2026-07-28 17:32:18.872941311 +0000 UTC m=+1294.119043516"
Jul 28 17:32:49.860655 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:49.860620    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn"
Jul 28 17:32:51.706390 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:51.706350    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn"]
Jul 28 17:32:59.860456 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:32:59.860416    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn" podUID="76fd0ec9-874b-4ebc-bed7-1ad3ac608d72" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 17:33:06.726149 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:33:06.726044    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn" podUID="76fd0ec9-874b-4ebc-bed7-1ad3ac608d72" containerName="main" containerID="cri-o://6778d4c479198b5b4f70d52699dac52cd73ae4cabbbe982b2ac2fcb6cf84c513" gracePeriod=45
Jul 28 17:33:06.977164 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:33:06.977090    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn"
Jul 28 17:33:07.080162 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:33:07.080120    2567 generic.go:356] "Generic (PLEG): container finished" podID="76fd0ec9-874b-4ebc-bed7-1ad3ac608d72" containerID="6778d4c479198b5b4f70d52699dac52cd73ae4cabbbe982b2ac2fcb6cf84c513" exitCode=0
Jul 28 17:33:07.080162 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:33:07.080162    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn"
Jul 28 17:33:07.080373 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:33:07.080186    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn" event={"ID":"76fd0ec9-874b-4ebc-bed7-1ad3ac608d72","Type":"ContainerDied","Data":"6778d4c479198b5b4f70d52699dac52cd73ae4cabbbe982b2ac2fcb6cf84c513"}
Jul 28 17:33:07.080373 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:33:07.080235    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn" event={"ID":"76fd0ec9-874b-4ebc-bed7-1ad3ac608d72","Type":"ContainerDied","Data":"230f868f05cbd3f10f519fb8f7cd3ef43bcda9b7e51e6bd86a62fe2c80438ed3"}
Jul 28 17:33:07.080373 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:33:07.080258    2567 scope.go:117] "RemoveContainer" containerID="6778d4c479198b5b4f70d52699dac52cd73ae4cabbbe982b2ac2fcb6cf84c513"
Jul 28 17:33:07.088858 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:33:07.088787    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76fd0ec9-874b-4ebc-bed7-1ad3ac608d72-tls-certs\") pod \"76fd0ec9-874b-4ebc-bed7-1ad3ac608d72\" (UID: \"76fd0ec9-874b-4ebc-bed7-1ad3ac608d72\") "
Jul 28 17:33:07.088950 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:33:07.088864    2567 scope.go:117] "RemoveContainer" containerID="6778d4c479198b5b4f70d52699dac52cd73ae4cabbbe982b2ac2fcb6cf84c513"
Jul 28 17:33:07.088950 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:33:07.088898    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8bl7\" (UniqueName: \"kubernetes.io/projected/76fd0ec9-874b-4ebc-bed7-1ad3ac608d72-kube-api-access-j8bl7\") pod \"76fd0ec9-874b-4ebc-bed7-1ad3ac608d72\" (UID: \"76fd0ec9-874b-4ebc-bed7-1ad3ac608d72\") "
Jul 28 17:33:07.089207 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:33:07.089182    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6778d4c479198b5b4f70d52699dac52cd73ae4cabbbe982b2ac2fcb6cf84c513\": container with ID starting with 6778d4c479198b5b4f70d52699dac52cd73ae4cabbbe982b2ac2fcb6cf84c513 not found: ID does not exist" containerID="6778d4c479198b5b4f70d52699dac52cd73ae4cabbbe982b2ac2fcb6cf84c513"
Jul 28 17:33:07.089303 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:33:07.089211    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6778d4c479198b5b4f70d52699dac52cd73ae4cabbbe982b2ac2fcb6cf84c513"} err="failed to get container status \"6778d4c479198b5b4f70d52699dac52cd73ae4cabbbe982b2ac2fcb6cf84c513\": rpc error: code = NotFound desc = could not find container \"6778d4c479198b5b4f70d52699dac52cd73ae4cabbbe982b2ac2fcb6cf84c513\": container with ID starting with 6778d4c479198b5b4f70d52699dac52cd73ae4cabbbe982b2ac2fcb6cf84c513 not found: ID does not exist"
Jul 28 17:33:07.091013 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:33:07.090981    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76fd0ec9-874b-4ebc-bed7-1ad3ac608d72-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "76fd0ec9-874b-4ebc-bed7-1ad3ac608d72" (UID: "76fd0ec9-874b-4ebc-bed7-1ad3ac608d72"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:33:07.091097 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:33:07.091048    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76fd0ec9-874b-4ebc-bed7-1ad3ac608d72-kube-api-access-j8bl7" (OuterVolumeSpecName: "kube-api-access-j8bl7") pod "76fd0ec9-874b-4ebc-bed7-1ad3ac608d72" (UID: "76fd0ec9-874b-4ebc-bed7-1ad3ac608d72"). InnerVolumeSpecName "kube-api-access-j8bl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:33:07.190305 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:33:07.190247    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8bl7\" (UniqueName: \"kubernetes.io/projected/76fd0ec9-874b-4ebc-bed7-1ad3ac608d72-kube-api-access-j8bl7\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:33:07.190305 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:33:07.190296    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76fd0ec9-874b-4ebc-bed7-1ad3ac608d72-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:33:07.398457 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:33:07.398391    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn"]
Jul 28 17:33:07.403386 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:33:07.403327    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche822pn"]
Jul 28 17:33:09.286705 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:33:09.286667    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76fd0ec9-874b-4ebc-bed7-1ad3ac608d72" path="/var/lib/kubelet/pods/76fd0ec9-874b-4ebc-bed7-1ad3ac608d72/volumes"
Jul 28 17:34:22.496573 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:22.496528    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc"]
Jul 28 17:34:22.497231 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:22.496996    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76fd0ec9-874b-4ebc-bed7-1ad3ac608d72" containerName="main"
Jul 28 17:34:22.497231 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:22.497048    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="76fd0ec9-874b-4ebc-bed7-1ad3ac608d72" containerName="main"
Jul 28 17:34:22.497231 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:22.497142    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="76fd0ec9-874b-4ebc-bed7-1ad3ac608d72" containerName="main"
Jul 28 17:34:22.500193 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:22.500168    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc"
Jul 28 17:34:22.503984 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:22.503957    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"kube-root-ca.crt\""
Jul 28 17:34:22.505700 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:22.505677    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"openshift-service-ca.crt\""
Jul 28 17:34:22.505835 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:22.505794    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\""
Jul 28 17:34:22.506439 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:22.506420    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"default-dockercfg-g922c\""
Jul 28 17:34:22.506571 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:22.506450    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"scheduler-ha-replicas-test-epp-sa-dockercfg-9wb52\""
Jul 28 17:34:22.509901 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:22.509877    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc"]
Jul 28 17:34:22.617793 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:22.617749    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns567\" (UniqueName: \"kubernetes.io/projected/884c8648-7e97-47e8-90dc-74e752b71963-kube-api-access-ns567\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc\" (UID: \"884c8648-7e97-47e8-90dc-74e752b71963\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc"
Jul 28 17:34:22.617793 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:22.617798    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/884c8648-7e97-47e8-90dc-74e752b71963-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc\" (UID: \"884c8648-7e97-47e8-90dc-74e752b71963\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc"
Jul 28 17:34:22.719090 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:22.719039    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/884c8648-7e97-47e8-90dc-74e752b71963-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc\" (UID: \"884c8648-7e97-47e8-90dc-74e752b71963\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc"
Jul 28 17:34:22.719261 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:22.719130    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns567\" (UniqueName: \"kubernetes.io/projected/884c8648-7e97-47e8-90dc-74e752b71963-kube-api-access-ns567\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc\" (UID: \"884c8648-7e97-47e8-90dc-74e752b71963\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc"
Jul 28 17:34:22.721761 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:22.721725    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/884c8648-7e97-47e8-90dc-74e752b71963-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc\" (UID: \"884c8648-7e97-47e8-90dc-74e752b71963\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc"
Jul 28 17:34:22.728309 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:22.728272    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns567\" (UniqueName: \"kubernetes.io/projected/884c8648-7e97-47e8-90dc-74e752b71963-kube-api-access-ns567\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc\" (UID: \"884c8648-7e97-47e8-90dc-74e752b71963\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc"
Jul 28 17:34:22.812454 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:22.812412    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc"
Jul 28 17:34:22.957302 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:22.957255    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc"]
Jul 28 17:34:22.959679 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:34:22.959647    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod884c8648_7e97_47e8_90dc_74e752b71963.slice/crio-5fd1278ec072ca9c63e48f5f35f1901e05a2910a76a96863c2315cbb6133e842 WatchSource:0}: Error finding container 5fd1278ec072ca9c63e48f5f35f1901e05a2910a76a96863c2315cbb6133e842: Status 404 returned error can't find the container with id 5fd1278ec072ca9c63e48f5f35f1901e05a2910a76a96863c2315cbb6133e842
Jul 28 17:34:23.446942 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:23.446902    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc" event={"ID":"884c8648-7e97-47e8-90dc-74e752b71963","Type":"ContainerStarted","Data":"851da6ef9433e52347da1eaa0b76ea8ebb037d5c71538c22c7da6c996fcd60da"}
Jul 28 17:34:23.447133 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:23.446970    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc" event={"ID":"884c8648-7e97-47e8-90dc-74e752b71963","Type":"ContainerStarted","Data":"5fd1278ec072ca9c63e48f5f35f1901e05a2910a76a96863c2315cbb6133e842"}
Jul 28 17:34:23.447133 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:23.447035    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc"
Jul 28 17:34:23.467850 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:23.467792    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc" podStartSLOduration=1.467775322 podStartE2EDuration="1.467775322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:34:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:34:23.466766192 +0000 UTC m=+1418.712868390" watchObservedRunningTime="2026-07-28 17:34:23.467775322 +0000 UTC m=+1418.713877529"
Jul 28 17:34:54.453858 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:54.453779    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc"
Jul 28 17:34:55.775077 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:34:55.775034    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc"]
Jul 28 17:35:04.454296 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:04.454248    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc" podUID="884c8648-7e97-47e8-90dc-74e752b71963" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 17:35:07.268962 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:07.268912    2567 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc" message="" err="command '/bin/sleep 15' exited with 137: "
Jul 28 17:35:07.268962 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:35:07.268963    2567 kuberuntime_container.go:755] "PreStop hook failed" err="command '/bin/sleep 15' exited with 137: " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc" podUID="884c8648-7e97-47e8-90dc-74e752b71963" containerName="main" containerID="cri-o://851da6ef9433e52347da1eaa0b76ea8ebb037d5c71538c22c7da6c996fcd60da"
Jul 28 17:35:07.269413 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:07.269018    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc" podUID="884c8648-7e97-47e8-90dc-74e752b71963" containerName="main" containerID="cri-o://851da6ef9433e52347da1eaa0b76ea8ebb037d5c71538c22c7da6c996fcd60da" gracePeriod=49
Jul 28 17:35:07.425767 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:07.425739    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-llm-inference-service-2541cd65_scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc_884c8648-7e97-47e8-90dc-74e752b71963/main/0.log"
Jul 28 17:35:07.428044 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:07.428026    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc"
Jul 28 17:35:07.498988 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:07.498943    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ns567\" (UniqueName: \"kubernetes.io/projected/884c8648-7e97-47e8-90dc-74e752b71963-kube-api-access-ns567\") pod \"884c8648-7e97-47e8-90dc-74e752b71963\" (UID: \"884c8648-7e97-47e8-90dc-74e752b71963\") "
Jul 28 17:35:07.499173 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:07.498998    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/884c8648-7e97-47e8-90dc-74e752b71963-tls-certs\") pod \"884c8648-7e97-47e8-90dc-74e752b71963\" (UID: \"884c8648-7e97-47e8-90dc-74e752b71963\") "
Jul 28 17:35:07.501265 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:07.501230    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/884c8648-7e97-47e8-90dc-74e752b71963-kube-api-access-ns567" (OuterVolumeSpecName: "kube-api-access-ns567") pod "884c8648-7e97-47e8-90dc-74e752b71963" (UID: "884c8648-7e97-47e8-90dc-74e752b71963"). InnerVolumeSpecName "kube-api-access-ns567". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:35:07.501390 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:07.501324    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/884c8648-7e97-47e8-90dc-74e752b71963-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "884c8648-7e97-47e8-90dc-74e752b71963" (UID: "884c8648-7e97-47e8-90dc-74e752b71963"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:35:07.600531 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:07.600405    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ns567\" (UniqueName: \"kubernetes.io/projected/884c8648-7e97-47e8-90dc-74e752b71963-kube-api-access-ns567\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:35:07.600531 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:07.600441    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/884c8648-7e97-47e8-90dc-74e752b71963-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:35:07.663595 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:07.663566    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-llm-inference-service-2541cd65_scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc_884c8648-7e97-47e8-90dc-74e752b71963/main/0.log"
Jul 28 17:35:07.663758 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:07.663633    2567 generic.go:356] "Generic (PLEG): container finished" podID="884c8648-7e97-47e8-90dc-74e752b71963" containerID="851da6ef9433e52347da1eaa0b76ea8ebb037d5c71538c22c7da6c996fcd60da" exitCode=1
Jul 28 17:35:07.663758 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:07.663685    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc"
Jul 28 17:35:07.663758 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:07.663707    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc" event={"ID":"884c8648-7e97-47e8-90dc-74e752b71963","Type":"ContainerDied","Data":"851da6ef9433e52347da1eaa0b76ea8ebb037d5c71538c22c7da6c996fcd60da"}
Jul 28 17:35:07.663758 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:07.663752    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc" event={"ID":"884c8648-7e97-47e8-90dc-74e752b71963","Type":"ContainerDied","Data":"5fd1278ec072ca9c63e48f5f35f1901e05a2910a76a96863c2315cbb6133e842"}
Jul 28 17:35:07.663891 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:07.663781    2567 scope.go:117] "RemoveContainer" containerID="851da6ef9433e52347da1eaa0b76ea8ebb037d5c71538c22c7da6c996fcd60da"
Jul 28 17:35:07.672753 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:07.672732    2567 scope.go:117] "RemoveContainer" containerID="851da6ef9433e52347da1eaa0b76ea8ebb037d5c71538c22c7da6c996fcd60da"
Jul 28 17:35:07.673068 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:35:07.673048    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"851da6ef9433e52347da1eaa0b76ea8ebb037d5c71538c22c7da6c996fcd60da\": container with ID starting with 851da6ef9433e52347da1eaa0b76ea8ebb037d5c71538c22c7da6c996fcd60da not found: ID does not exist" containerID="851da6ef9433e52347da1eaa0b76ea8ebb037d5c71538c22c7da6c996fcd60da"
Jul 28 17:35:07.673131 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:07.673081    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"851da6ef9433e52347da1eaa0b76ea8ebb037d5c71538c22c7da6c996fcd60da"} err="failed to get container status \"851da6ef9433e52347da1eaa0b76ea8ebb037d5c71538c22c7da6c996fcd60da\": rpc error: code = NotFound desc = could not find container \"851da6ef9433e52347da1eaa0b76ea8ebb037d5c71538c22c7da6c996fcd60da\": container with ID starting with 851da6ef9433e52347da1eaa0b76ea8ebb037d5c71538c22c7da6c996fcd60da not found: ID does not exist"
Jul 28 17:35:07.687007 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:07.686964    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc"]
Jul 28 17:35:07.690135 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:07.690095    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfwc4hc"]
Jul 28 17:35:09.287237 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:09.287193    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="884c8648-7e97-47e8-90dc-74e752b71963" path="/var/lib/kubelet/pods/884c8648-7e97-47e8-90dc-74e752b71963/volumes"
Jul 28 17:35:46.047372 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:46.047330    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:35:46.051262 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:35:46.051234    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:39:47.210837 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.210793    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"]
Jul 28 17:39:47.211426 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.211257    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="884c8648-7e97-47e8-90dc-74e752b71963" containerName="main"
Jul 28 17:39:47.211426 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.211278    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="884c8648-7e97-47e8-90dc-74e752b71963" containerName="main"
Jul 28 17:39:47.211426 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.211359    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="884c8648-7e97-47e8-90dc-74e752b71963" containerName="main"
Jul 28 17:39:47.214381 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.214353    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:47.217915 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.217883    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"openshift-service-ca.crt\""
Jul 28 17:39:47.219064 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.219031    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"default-dockercfg-pcqc7\""
Jul 28 17:39:47.219238 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.219037    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"kube-root-ca.crt\""
Jul 28 17:39:47.219418 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.219380    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"scheduler-v06-threshold-migration-test-kserve-self-signed-certs\""
Jul 28 17:39:47.226883 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.226848    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"]
Jul 28 17:39:47.322254 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.322197    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:47.322254 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.322253    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/579e8f01-8db8-4e5c-891b-40f5ef0dce19-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:47.322584 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.322311    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-home\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:47.322584 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.322354    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:47.322584 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.322382    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkl6c\" (UniqueName: \"kubernetes.io/projected/579e8f01-8db8-4e5c-891b-40f5ef0dce19-kube-api-access-mkl6c\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:47.322584 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.322490    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:47.423852 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.423805    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:47.424028 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.423891    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:47.424028 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.423923    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/579e8f01-8db8-4e5c-891b-40f5ef0dce19-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:47.424028 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.423964    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-home\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:47.424028 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.423990    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:47.424028 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.424018    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkl6c\" (UniqueName: \"kubernetes.io/projected/579e8f01-8db8-4e5c-891b-40f5ef0dce19-kube-api-access-mkl6c\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:47.424383 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.424355    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:47.424468 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.424428    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-home\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:47.424641 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.424616    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:47.426702 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.426660    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:47.426904 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.426880    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/579e8f01-8db8-4e5c-891b-40f5ef0dce19-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:47.434204 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.434163    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkl6c\" (UniqueName: \"kubernetes.io/projected/579e8f01-8db8-4e5c-891b-40f5ef0dce19-kube-api-access-mkl6c\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:47.527172 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.527126    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:47.679797 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.679766    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"]
Jul 28 17:39:47.682049 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:39:47.682015    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod579e8f01_8db8_4e5c_891b_40f5ef0dce19.slice/crio-d0bf8ced106070955648b3afaae45cbfee67346ef9d5f9ab6f66226eed537f87 WatchSource:0}: Error finding container d0bf8ced106070955648b3afaae45cbfee67346ef9d5f9ab6f66226eed537f87: Status 404 returned error can't find the container with id d0bf8ced106070955648b3afaae45cbfee67346ef9d5f9ab6f66226eed537f87
Jul 28 17:39:47.684514 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:47.684479    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 17:39:48.025291 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:48.025252    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x" event={"ID":"579e8f01-8db8-4e5c-891b-40f5ef0dce19","Type":"ContainerStarted","Data":"d0bf8ced106070955648b3afaae45cbfee67346ef9d5f9ab6f66226eed537f87"}
Jul 28 17:39:50.038018 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:50.037975    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x" event={"ID":"579e8f01-8db8-4e5c-891b-40f5ef0dce19","Type":"ContainerStarted","Data":"77685bc18e8848c3711a447203f5a8882c1b15bff63093355b7fb377e25d51c0"}
Jul 28 17:39:50.058945 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:50.058876    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x" podStartSLOduration=1.526171459 podStartE2EDuration="3.058857191s" totalImagesPullingTime="1.532685732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:39:50.057379504 +0000 UTC m=+1745.303481724" watchObservedRunningTime="2026-07-28 17:39:50.058857191 +0000 UTC m=+1745.304959454"
Jul 28 17:39:50.528266 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:50.528225    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:57.528304 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:57.528254    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:57.540836 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:57.540799    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:39:58.097655 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:39:58.097618    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:40:21.524614 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:21.524570    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"]
Jul 28 17:40:21.534059 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:40:21.534004    2567 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 28 17:40:21.534059 ip-10-0-133-215 kubenswrapper[2567]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 17:40:21.534059 ip-10-0-133-215 kubenswrapper[2567]:         , stdout: , stderr: , exit code -1
Jul 28 17:40:21.534059 ip-10-0-133-215 kubenswrapper[2567]:  > containerID="77685bc18e8848c3711a447203f5a8882c1b15bff63093355b7fb377e25d51c0" cmd=["/bin/sleep","15"]
Jul 28 17:40:21.534405 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:21.534083    2567 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x" message="" err=<
Jul 28 17:40:21.534405 ip-10-0-133-215 kubenswrapper[2567]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 17:40:21.534405 ip-10-0-133-215 kubenswrapper[2567]:         , stdout: , stderr: , exit code -1
Jul 28 17:40:21.534405 ip-10-0-133-215 kubenswrapper[2567]:  >
Jul 28 17:40:21.534405 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:40:21.534101    2567 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 28 17:40:21.534405 ip-10-0-133-215 kubenswrapper[2567]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 17:40:21.534405 ip-10-0-133-215 kubenswrapper[2567]:         , stdout: , stderr: , exit code -1
Jul 28 17:40:21.534405 ip-10-0-133-215 kubenswrapper[2567]:  > pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x" podUID="579e8f01-8db8-4e5c-891b-40f5ef0dce19" containerName="main" containerID="cri-o://77685bc18e8848c3711a447203f5a8882c1b15bff63093355b7fb377e25d51c0"
Jul 28 17:40:21.534405 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:21.534162    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x" podUID="579e8f01-8db8-4e5c-891b-40f5ef0dce19" containerName="main" containerID="cri-o://77685bc18e8848c3711a447203f5a8882c1b15bff63093355b7fb377e25d51c0" gracePeriod=60
Jul 28 17:40:21.904085 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:21.904050    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:40:22.033348 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.033292    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/579e8f01-8db8-4e5c-891b-40f5ef0dce19-tls-certs\") pod \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") "
Jul 28 17:40:22.033575 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.033369    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-model-cache\") pod \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") "
Jul 28 17:40:22.033575 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.033396    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-home\") pod \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") "
Jul 28 17:40:22.033575 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.033418    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-dshm\") pod \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") "
Jul 28 17:40:22.033575 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.033442    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkl6c\" (UniqueName: \"kubernetes.io/projected/579e8f01-8db8-4e5c-891b-40f5ef0dce19-kube-api-access-mkl6c\") pod \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") "
Jul 28 17:40:22.033575 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.033479    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-tmp-dir\") pod \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\" (UID: \"579e8f01-8db8-4e5c-891b-40f5ef0dce19\") "
Jul 28 17:40:22.033842 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.033695    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-home" (OuterVolumeSpecName: "home") pod "579e8f01-8db8-4e5c-891b-40f5ef0dce19" (UID: "579e8f01-8db8-4e5c-891b-40f5ef0dce19"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:40:22.033842 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.033732    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-model-cache" (OuterVolumeSpecName: "model-cache") pod "579e8f01-8db8-4e5c-891b-40f5ef0dce19" (UID: "579e8f01-8db8-4e5c-891b-40f5ef0dce19"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:40:22.033842 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.033801    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-model-cache\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:40:22.033842 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.033806    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "579e8f01-8db8-4e5c-891b-40f5ef0dce19" (UID: "579e8f01-8db8-4e5c-891b-40f5ef0dce19"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:40:22.033842 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.033818    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-home\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:40:22.035760 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.035726    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/579e8f01-8db8-4e5c-891b-40f5ef0dce19-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "579e8f01-8db8-4e5c-891b-40f5ef0dce19" (UID: "579e8f01-8db8-4e5c-891b-40f5ef0dce19"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:40:22.035897 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.035763    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-dshm" (OuterVolumeSpecName: "dshm") pod "579e8f01-8db8-4e5c-891b-40f5ef0dce19" (UID: "579e8f01-8db8-4e5c-891b-40f5ef0dce19"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:40:22.035897 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.035868    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/579e8f01-8db8-4e5c-891b-40f5ef0dce19-kube-api-access-mkl6c" (OuterVolumeSpecName: "kube-api-access-mkl6c") pod "579e8f01-8db8-4e5c-891b-40f5ef0dce19" (UID: "579e8f01-8db8-4e5c-891b-40f5ef0dce19"). InnerVolumeSpecName "kube-api-access-mkl6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:40:22.134941 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.134843    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/579e8f01-8db8-4e5c-891b-40f5ef0dce19-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:40:22.134941 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.134877    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-dshm\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:40:22.134941 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.134886    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkl6c\" (UniqueName: \"kubernetes.io/projected/579e8f01-8db8-4e5c-891b-40f5ef0dce19-kube-api-access-mkl6c\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:40:22.134941 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.134897    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/579e8f01-8db8-4e5c-891b-40f5ef0dce19-tmp-dir\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:40:22.203972 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.203935    2567 generic.go:356] "Generic (PLEG): container finished" podID="579e8f01-8db8-4e5c-891b-40f5ef0dce19" containerID="77685bc18e8848c3711a447203f5a8882c1b15bff63093355b7fb377e25d51c0" exitCode=0
Jul 28 17:40:22.204180 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.203984    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"
Jul 28 17:40:22.204180 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.204007    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x" event={"ID":"579e8f01-8db8-4e5c-891b-40f5ef0dce19","Type":"ContainerDied","Data":"77685bc18e8848c3711a447203f5a8882c1b15bff63093355b7fb377e25d51c0"}
Jul 28 17:40:22.204180 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.204069    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x" event={"ID":"579e8f01-8db8-4e5c-891b-40f5ef0dce19","Type":"ContainerDied","Data":"d0bf8ced106070955648b3afaae45cbfee67346ef9d5f9ab6f66226eed537f87"}
Jul 28 17:40:22.204180 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.204098    2567 scope.go:117] "RemoveContainer" containerID="77685bc18e8848c3711a447203f5a8882c1b15bff63093355b7fb377e25d51c0"
Jul 28 17:40:22.215398 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.215376    2567 scope.go:117] "RemoveContainer" containerID="77685bc18e8848c3711a447203f5a8882c1b15bff63093355b7fb377e25d51c0"
Jul 28 17:40:22.215763 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:40:22.215742    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77685bc18e8848c3711a447203f5a8882c1b15bff63093355b7fb377e25d51c0\": container with ID starting with 77685bc18e8848c3711a447203f5a8882c1b15bff63093355b7fb377e25d51c0 not found: ID does not exist" containerID="77685bc18e8848c3711a447203f5a8882c1b15bff63093355b7fb377e25d51c0"
Jul 28 17:40:22.215875 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.215776    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77685bc18e8848c3711a447203f5a8882c1b15bff63093355b7fb377e25d51c0"} err="failed to get container status \"77685bc18e8848c3711a447203f5a8882c1b15bff63093355b7fb377e25d51c0\": rpc error: code = NotFound desc = could not find container \"77685bc18e8848c3711a447203f5a8882c1b15bff63093355b7fb377e25d51c0\": container with ID starting with 77685bc18e8848c3711a447203f5a8882c1b15bff63093355b7fb377e25d51c0 not found: ID does not exist"
Jul 28 17:40:22.227228 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.227190    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"]
Jul 28 17:40:22.230760 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:22.230718    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fh758x"]
Jul 28 17:40:23.287458 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:23.287418    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="579e8f01-8db8-4e5c-891b-40f5ef0dce19" path="/var/lib/kubelet/pods/579e8f01-8db8-4e5c-891b-40f5ef0dce19/volumes"
Jul 28 17:40:45.920822 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:45.920781    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb"]
Jul 28 17:40:45.921234 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:45.921220    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="579e8f01-8db8-4e5c-891b-40f5ef0dce19" containerName="main"
Jul 28 17:40:45.921234 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:45.921234    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="579e8f01-8db8-4e5c-891b-40f5ef0dce19" containerName="main"
Jul 28 17:40:45.921332 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:45.921319    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="579e8f01-8db8-4e5c-891b-40f5ef0dce19" containerName="main"
Jul 28 17:40:45.925933 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:45.925903    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb"
Jul 28 17:40:45.929016 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:45.928987    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"kube-root-ca.crt\""
Jul 28 17:40:45.930727 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:45.930698    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"openshift-service-ca.crt\""
Jul 28 17:40:45.930871 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:45.930767    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"default-dockercfg-h6ncf\""
Jul 28 17:40:45.930871 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:45.930802    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-epp-sa-dockercfg-pg5vh\""
Jul 28 17:40:45.930985 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:45.930698    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-kserve-self-signed-certs\""
Jul 28 17:40:45.936366 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:45.936335    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb"]
Jul 28 17:40:46.013053 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:46.013003    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rms9v\" (UniqueName: \"kubernetes.io/projected/7c3cfbe0-bdc2-4944-8a63-756fb5979f8c-kube-api-access-rms9v\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb\" (UID: \"7c3cfbe0-bdc2-4944-8a63-756fb5979f8c\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb"
Jul 28 17:40:46.013234 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:46.013116    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c3cfbe0-bdc2-4944-8a63-756fb5979f8c-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb\" (UID: \"7c3cfbe0-bdc2-4944-8a63-756fb5979f8c\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb"
Jul 28 17:40:46.077251 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:46.077224    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:40:46.083125 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:46.083096    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:40:46.113900 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:46.113855    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c3cfbe0-bdc2-4944-8a63-756fb5979f8c-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb\" (UID: \"7c3cfbe0-bdc2-4944-8a63-756fb5979f8c\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb"
Jul 28 17:40:46.114072 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:46.113947    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rms9v\" (UniqueName: \"kubernetes.io/projected/7c3cfbe0-bdc2-4944-8a63-756fb5979f8c-kube-api-access-rms9v\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb\" (UID: \"7c3cfbe0-bdc2-4944-8a63-756fb5979f8c\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb"
Jul 28 17:40:46.116631 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:46.116596    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c3cfbe0-bdc2-4944-8a63-756fb5979f8c-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb\" (UID: \"7c3cfbe0-bdc2-4944-8a63-756fb5979f8c\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb"
Jul 28 17:40:46.125956 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:46.125923    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rms9v\" (UniqueName: \"kubernetes.io/projected/7c3cfbe0-bdc2-4944-8a63-756fb5979f8c-kube-api-access-rms9v\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb\" (UID: \"7c3cfbe0-bdc2-4944-8a63-756fb5979f8c\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb"
Jul 28 17:40:46.238149 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:46.238043    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb"
Jul 28 17:40:46.382953 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:46.382916    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb"]
Jul 28 17:40:46.385105 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:40:46.385069    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c3cfbe0_bdc2_4944_8a63_756fb5979f8c.slice/crio-9c94a12aef44f82d8ca761a10c1a8a6ff0ff9ff0db0d421fe0d1d92ccf372211 WatchSource:0}: Error finding container 9c94a12aef44f82d8ca761a10c1a8a6ff0ff9ff0db0d421fe0d1d92ccf372211: Status 404 returned error can't find the container with id 9c94a12aef44f82d8ca761a10c1a8a6ff0ff9ff0db0d421fe0d1d92ccf372211
Jul 28 17:40:47.329868 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:47.329835    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb" event={"ID":"7c3cfbe0-bdc2-4944-8a63-756fb5979f8c","Type":"ContainerStarted","Data":"dca108d7a693e927f6e11f9a90104c6f98e1327124ed6b93c191b46e2b4180ce"}
Jul 28 17:40:47.330266 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:47.329886    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb" event={"ID":"7c3cfbe0-bdc2-4944-8a63-756fb5979f8c","Type":"ContainerStarted","Data":"9c94a12aef44f82d8ca761a10c1a8a6ff0ff9ff0db0d421fe0d1d92ccf372211"}
Jul 28 17:40:47.330266 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:47.329930    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb"
Jul 28 17:40:47.353917 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:40:47.353852    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb" podStartSLOduration=2.353834932 podStartE2EDuration="2.353834932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:40:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:40:47.351834347 +0000 UTC m=+1802.597936567" watchObservedRunningTime="2026-07-28 17:40:47.353834932 +0000 UTC m=+1802.599937188"
Jul 28 17:41:18.337240 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:41:18.337206    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb"
Jul 28 17:42:06.676105 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:06.676067    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb"]
Jul 28 17:42:08.336785 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:08.336745    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb" podUID="7c3cfbe0-bdc2-4944-8a63-756fb5979f8c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 17:42:18.337029 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:18.336987    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb" podUID="7c3cfbe0-bdc2-4944-8a63-756fb5979f8c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 17:42:21.697876 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:21.697832    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb" podUID="7c3cfbe0-bdc2-4944-8a63-756fb5979f8c" containerName="main" containerID="cri-o://dca108d7a693e927f6e11f9a90104c6f98e1327124ed6b93c191b46e2b4180ce" gracePeriod=45
Jul 28 17:42:21.964589 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:21.964557    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb"
Jul 28 17:42:22.108847 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:22.108800    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c3cfbe0-bdc2-4944-8a63-756fb5979f8c-tls-certs\") pod \"7c3cfbe0-bdc2-4944-8a63-756fb5979f8c\" (UID: \"7c3cfbe0-bdc2-4944-8a63-756fb5979f8c\") "
Jul 28 17:42:22.108847 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:22.108854    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rms9v\" (UniqueName: \"kubernetes.io/projected/7c3cfbe0-bdc2-4944-8a63-756fb5979f8c-kube-api-access-rms9v\") pod \"7c3cfbe0-bdc2-4944-8a63-756fb5979f8c\" (UID: \"7c3cfbe0-bdc2-4944-8a63-756fb5979f8c\") "
Jul 28 17:42:22.111176 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:22.111138    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c3cfbe0-bdc2-4944-8a63-756fb5979f8c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7c3cfbe0-bdc2-4944-8a63-756fb5979f8c" (UID: "7c3cfbe0-bdc2-4944-8a63-756fb5979f8c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:42:22.111312 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:22.111190    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c3cfbe0-bdc2-4944-8a63-756fb5979f8c-kube-api-access-rms9v" (OuterVolumeSpecName: "kube-api-access-rms9v") pod "7c3cfbe0-bdc2-4944-8a63-756fb5979f8c" (UID: "7c3cfbe0-bdc2-4944-8a63-756fb5979f8c"). InnerVolumeSpecName "kube-api-access-rms9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:42:22.209679 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:22.209576    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c3cfbe0-bdc2-4944-8a63-756fb5979f8c-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:42:22.209679 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:22.209611    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rms9v\" (UniqueName: \"kubernetes.io/projected/7c3cfbe0-bdc2-4944-8a63-756fb5979f8c-kube-api-access-rms9v\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:42:22.789335 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:22.789298    2567 generic.go:356] "Generic (PLEG): container finished" podID="7c3cfbe0-bdc2-4944-8a63-756fb5979f8c" containerID="dca108d7a693e927f6e11f9a90104c6f98e1327124ed6b93c191b46e2b4180ce" exitCode=0
Jul 28 17:42:22.789786 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:22.789354    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb"
Jul 28 17:42:22.789786 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:22.789380    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb" event={"ID":"7c3cfbe0-bdc2-4944-8a63-756fb5979f8c","Type":"ContainerDied","Data":"dca108d7a693e927f6e11f9a90104c6f98e1327124ed6b93c191b46e2b4180ce"}
Jul 28 17:42:22.789786 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:22.789464    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb" event={"ID":"7c3cfbe0-bdc2-4944-8a63-756fb5979f8c","Type":"ContainerDied","Data":"9c94a12aef44f82d8ca761a10c1a8a6ff0ff9ff0db0d421fe0d1d92ccf372211"}
Jul 28 17:42:22.789786 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:22.789488    2567 scope.go:117] "RemoveContainer" containerID="dca108d7a693e927f6e11f9a90104c6f98e1327124ed6b93c191b46e2b4180ce"
Jul 28 17:42:22.798650 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:22.798624    2567 scope.go:117] "RemoveContainer" containerID="dca108d7a693e927f6e11f9a90104c6f98e1327124ed6b93c191b46e2b4180ce"
Jul 28 17:42:22.798975 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:42:22.798953    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dca108d7a693e927f6e11f9a90104c6f98e1327124ed6b93c191b46e2b4180ce\": container with ID starting with dca108d7a693e927f6e11f9a90104c6f98e1327124ed6b93c191b46e2b4180ce not found: ID does not exist" containerID="dca108d7a693e927f6e11f9a90104c6f98e1327124ed6b93c191b46e2b4180ce"
Jul 28 17:42:22.799016 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:22.798988    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dca108d7a693e927f6e11f9a90104c6f98e1327124ed6b93c191b46e2b4180ce"} err="failed to get container status \"dca108d7a693e927f6e11f9a90104c6f98e1327124ed6b93c191b46e2b4180ce\": rpc error: code = NotFound desc = could not find container \"dca108d7a693e927f6e11f9a90104c6f98e1327124ed6b93c191b46e2b4180ce\": container with ID starting with dca108d7a693e927f6e11f9a90104c6f98e1327124ed6b93c191b46e2b4180ce not found: ID does not exist"
Jul 28 17:42:22.812843 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:22.812789    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb"]
Jul 28 17:42:22.816836 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:22.816787    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65597zb"]
Jul 28 17:42:23.286803 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:23.286763    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c3cfbe0-bdc2-4944-8a63-756fb5979f8c" path="/var/lib/kubelet/pods/7c3cfbe0-bdc2-4944-8a63-756fb5979f8c/volumes"
Jul 28 17:42:37.118356 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.118318    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9"]
Jul 28 17:42:37.118997 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.118957    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c3cfbe0-bdc2-4944-8a63-756fb5979f8c" containerName="main"
Jul 28 17:42:37.118997 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.118986    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c3cfbe0-bdc2-4944-8a63-756fb5979f8c" containerName="main"
Jul 28 17:42:37.119154 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.119066    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c3cfbe0-bdc2-4944-8a63-756fb5979f8c" containerName="main"
Jul 28 17:42:37.122453 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.122424    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9"
Jul 28 17:42:37.127495 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.127459    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"kube-root-ca.crt\""
Jul 28 17:42:37.129065 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.129009    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"openshift-service-ca.crt\""
Jul 28 17:42:37.129065 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.129021    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"tokenizer-migration-path-test-epp-sa-dockercfg-lssz2\""
Jul 28 17:42:37.129065 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.129041    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"default-dockercfg-5rwls\""
Jul 28 17:42:37.129343 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.129098    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"tokenizer-migration-path-test-kserve-self-signed-certs\""
Jul 28 17:42:37.138018 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.137986    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9"]
Jul 28 17:42:37.232401 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.232350    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jj4j\" (UniqueName: \"kubernetes.io/projected/735aa4be-2509-4623-8790-fcaf5b9968f4-kube-api-access-9jj4j\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9\" (UID: \"735aa4be-2509-4623-8790-fcaf5b9968f4\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9"
Jul 28 17:42:37.232401 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.232401    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/735aa4be-2509-4623-8790-fcaf5b9968f4-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9\" (UID: \"735aa4be-2509-4623-8790-fcaf5b9968f4\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9"
Jul 28 17:42:37.332916 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.332862    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jj4j\" (UniqueName: \"kubernetes.io/projected/735aa4be-2509-4623-8790-fcaf5b9968f4-kube-api-access-9jj4j\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9\" (UID: \"735aa4be-2509-4623-8790-fcaf5b9968f4\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9"
Jul 28 17:42:37.332916 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.332916    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/735aa4be-2509-4623-8790-fcaf5b9968f4-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9\" (UID: \"735aa4be-2509-4623-8790-fcaf5b9968f4\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9"
Jul 28 17:42:37.335668 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.335627    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/735aa4be-2509-4623-8790-fcaf5b9968f4-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9\" (UID: \"735aa4be-2509-4623-8790-fcaf5b9968f4\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9"
Jul 28 17:42:37.341880 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.341847    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jj4j\" (UniqueName: \"kubernetes.io/projected/735aa4be-2509-4623-8790-fcaf5b9968f4-kube-api-access-9jj4j\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9\" (UID: \"735aa4be-2509-4623-8790-fcaf5b9968f4\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9"
Jul 28 17:42:37.435680 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.435580    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9"
Jul 28 17:42:37.581762 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.581720    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9"]
Jul 28 17:42:37.583716 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:42:37.583680    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod735aa4be_2509_4623_8790_fcaf5b9968f4.slice/crio-15f79ca1631aceaf68f1da3e27b7426be9b7b91529bf8aeb49de1a463b177126 WatchSource:0}: Error finding container 15f79ca1631aceaf68f1da3e27b7426be9b7b91529bf8aeb49de1a463b177126: Status 404 returned error can't find the container with id 15f79ca1631aceaf68f1da3e27b7426be9b7b91529bf8aeb49de1a463b177126
Jul 28 17:42:37.860232 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.860180    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9" event={"ID":"735aa4be-2509-4623-8790-fcaf5b9968f4","Type":"ContainerStarted","Data":"bfaba55d5631ebef0366e3f40b852d06a97ed786550f683185944527645e7747"}
Jul 28 17:42:37.860444 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.860241    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9" event={"ID":"735aa4be-2509-4623-8790-fcaf5b9968f4","Type":"ContainerStarted","Data":"15f79ca1631aceaf68f1da3e27b7426be9b7b91529bf8aeb49de1a463b177126"}
Jul 28 17:42:37.860444 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.860353    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9"
Jul 28 17:42:37.878072 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:42:37.878010    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9" podStartSLOduration=0.877989567 podStartE2EDuration="877.989567ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:42:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:42:37.876901363 +0000 UTC m=+1913.123003569" watchObservedRunningTime="2026-07-28 17:42:37.877989567 +0000 UTC m=+1913.124091774"
Jul 28 17:43:08.866994 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:43:08.866959    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9"
Jul 28 17:43:22.512633 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:43:22.512594    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9"]
Jul 28 17:43:28.867430 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:43:28.867386    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9" podUID="735aa4be-2509-4623-8790-fcaf5b9968f4" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 17:43:37.532843 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:43:37.532753    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9" podUID="735aa4be-2509-4623-8790-fcaf5b9968f4" containerName="main" containerID="cri-o://bfaba55d5631ebef0366e3f40b852d06a97ed786550f683185944527645e7747" gracePeriod=45
Jul 28 17:43:37.782340 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:43:37.782313    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9"
Jul 28 17:43:37.952415 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:43:37.952304    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/735aa4be-2509-4623-8790-fcaf5b9968f4-tls-certs\") pod \"735aa4be-2509-4623-8790-fcaf5b9968f4\" (UID: \"735aa4be-2509-4623-8790-fcaf5b9968f4\") "
Jul 28 17:43:37.952415 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:43:37.952375    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jj4j\" (UniqueName: \"kubernetes.io/projected/735aa4be-2509-4623-8790-fcaf5b9968f4-kube-api-access-9jj4j\") pod \"735aa4be-2509-4623-8790-fcaf5b9968f4\" (UID: \"735aa4be-2509-4623-8790-fcaf5b9968f4\") "
Jul 28 17:43:37.954640 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:43:37.954611    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/735aa4be-2509-4623-8790-fcaf5b9968f4-kube-api-access-9jj4j" (OuterVolumeSpecName: "kube-api-access-9jj4j") pod "735aa4be-2509-4623-8790-fcaf5b9968f4" (UID: "735aa4be-2509-4623-8790-fcaf5b9968f4"). InnerVolumeSpecName "kube-api-access-9jj4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:43:37.954761 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:43:37.954681    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/735aa4be-2509-4623-8790-fcaf5b9968f4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "735aa4be-2509-4623-8790-fcaf5b9968f4" (UID: "735aa4be-2509-4623-8790-fcaf5b9968f4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:43:38.053830 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:43:38.053792    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/735aa4be-2509-4623-8790-fcaf5b9968f4-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:43:38.053830 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:43:38.053822    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jj4j\" (UniqueName: \"kubernetes.io/projected/735aa4be-2509-4623-8790-fcaf5b9968f4-kube-api-access-9jj4j\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:43:38.152699 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:43:38.152658    2567 generic.go:356] "Generic (PLEG): container finished" podID="735aa4be-2509-4623-8790-fcaf5b9968f4" containerID="bfaba55d5631ebef0366e3f40b852d06a97ed786550f683185944527645e7747" exitCode=0
Jul 28 17:43:38.152864 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:43:38.152707    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9"
Jul 28 17:43:38.152864 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:43:38.152747    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9" event={"ID":"735aa4be-2509-4623-8790-fcaf5b9968f4","Type":"ContainerDied","Data":"bfaba55d5631ebef0366e3f40b852d06a97ed786550f683185944527645e7747"}
Jul 28 17:43:38.152864 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:43:38.152804    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9" event={"ID":"735aa4be-2509-4623-8790-fcaf5b9968f4","Type":"ContainerDied","Data":"15f79ca1631aceaf68f1da3e27b7426be9b7b91529bf8aeb49de1a463b177126"}
Jul 28 17:43:38.152864 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:43:38.152829    2567 scope.go:117] "RemoveContainer" containerID="bfaba55d5631ebef0366e3f40b852d06a97ed786550f683185944527645e7747"
Jul 28 17:43:38.161524 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:43:38.161488    2567 scope.go:117] "RemoveContainer" containerID="bfaba55d5631ebef0366e3f40b852d06a97ed786550f683185944527645e7747"
Jul 28 17:43:38.161803 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:43:38.161784    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfaba55d5631ebef0366e3f40b852d06a97ed786550f683185944527645e7747\": container with ID starting with bfaba55d5631ebef0366e3f40b852d06a97ed786550f683185944527645e7747 not found: ID does not exist" containerID="bfaba55d5631ebef0366e3f40b852d06a97ed786550f683185944527645e7747"
Jul 28 17:43:38.161856 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:43:38.161811    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfaba55d5631ebef0366e3f40b852d06a97ed786550f683185944527645e7747"} err="failed to get container status \"bfaba55d5631ebef0366e3f40b852d06a97ed786550f683185944527645e7747\": rpc error: code = NotFound desc = could not find container \"bfaba55d5631ebef0366e3f40b852d06a97ed786550f683185944527645e7747\": container with ID starting with bfaba55d5631ebef0366e3f40b852d06a97ed786550f683185944527645e7747 not found: ID does not exist"
Jul 28 17:43:38.174363 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:43:38.174338    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9"]
Jul 28 17:43:38.178439 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:43:38.178415    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9rrwz9"]
Jul 28 17:43:39.287007 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:43:39.286968    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="735aa4be-2509-4623-8790-fcaf5b9968f4" path="/var/lib/kubelet/pods/735aa4be-2509-4623-8790-fcaf5b9968f4/volumes"
Jul 28 17:45:31.262958 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:31.262919    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6_auth-invalid-token-test-kserve-router-scheduler-6d845bb7dclxd9q_74660d04-9529-4f20-b4a8-f877de2307e7/main/0.log"
Jul 28 17:45:39.264986 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:39.264952    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k"]
Jul 28 17:45:39.265568 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:39.265252    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="735aa4be-2509-4623-8790-fcaf5b9968f4" containerName="main"
Jul 28 17:45:39.265568 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:39.265268    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="735aa4be-2509-4623-8790-fcaf5b9968f4" containerName="main"
Jul 28 17:45:39.265568 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:39.265349    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="735aa4be-2509-4623-8790-fcaf5b9968f4" containerName="main"
Jul 28 17:45:39.268325 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:39.268301    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k"
Jul 28 17:45:39.271373 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:39.271353    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"openshift-service-ca.crt\""
Jul 28 17:45:39.272966 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:39.272931    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"auth-disabled-test-kserve-self-signed-certs\""
Jul 28 17:45:39.272966 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:39.272950    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"auth-disabled-test-epp-sa-dockercfg-8bxl6\""
Jul 28 17:45:39.272966 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:39.272941    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"kube-root-ca.crt\""
Jul 28 17:45:39.273192 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:39.272995    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"default-dockercfg-8twpr\""
Jul 28 17:45:39.279763 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:39.279743    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k"]
Jul 28 17:45:39.453258 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:39.453220    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33f98552-a235-4394-a477-f72a9e944f99-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k\" (UID: \"33f98552-a235-4394-a477-f72a9e944f99\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k"
Jul 28 17:45:39.453443 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:39.453278    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrn8l\" (UniqueName: \"kubernetes.io/projected/33f98552-a235-4394-a477-f72a9e944f99-kube-api-access-zrn8l\") pod \"auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k\" (UID: \"33f98552-a235-4394-a477-f72a9e944f99\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k"
Jul 28 17:45:39.554577 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:39.554473    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrn8l\" (UniqueName: \"kubernetes.io/projected/33f98552-a235-4394-a477-f72a9e944f99-kube-api-access-zrn8l\") pod \"auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k\" (UID: \"33f98552-a235-4394-a477-f72a9e944f99\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k"
Jul 28 17:45:39.554731 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:39.554627    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33f98552-a235-4394-a477-f72a9e944f99-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k\" (UID: \"33f98552-a235-4394-a477-f72a9e944f99\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k"
Jul 28 17:45:39.557003 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:39.556975    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33f98552-a235-4394-a477-f72a9e944f99-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k\" (UID: \"33f98552-a235-4394-a477-f72a9e944f99\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k"
Jul 28 17:45:39.563843 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:39.563819    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrn8l\" (UniqueName: \"kubernetes.io/projected/33f98552-a235-4394-a477-f72a9e944f99-kube-api-access-zrn8l\") pod \"auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k\" (UID: \"33f98552-a235-4394-a477-f72a9e944f99\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k"
Jul 28 17:45:39.579641 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:39.579617    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k"
Jul 28 17:45:39.714327 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:39.714298    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k"]
Jul 28 17:45:39.715648 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:45:39.715620    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33f98552_a235_4394_a477_f72a9e944f99.slice/crio-ede4f8b2a6b19f1550a52d41c54da1c0d09e67408a3093dc39d6b91b744a7a7e WatchSource:0}: Error finding container ede4f8b2a6b19f1550a52d41c54da1c0d09e67408a3093dc39d6b91b744a7a7e: Status 404 returned error can't find the container with id ede4f8b2a6b19f1550a52d41c54da1c0d09e67408a3093dc39d6b91b744a7a7e
Jul 28 17:45:40.723334 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:40.723295    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k" event={"ID":"33f98552-a235-4394-a477-f72a9e944f99","Type":"ContainerStarted","Data":"aec745a8f24d247d7b876420ef25a44cd444b4fab93d4a18d7059b89c15c11a1"}
Jul 28 17:45:40.723776 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:40.723353    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k" event={"ID":"33f98552-a235-4394-a477-f72a9e944f99","Type":"ContainerStarted","Data":"ede4f8b2a6b19f1550a52d41c54da1c0d09e67408a3093dc39d6b91b744a7a7e"}
Jul 28 17:45:40.723776 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:40.723387    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k"
Jul 28 17:45:40.742241 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:40.742193    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k" podStartSLOduration=1.742176786 podStartE2EDuration="1.742176786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:45:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:45:40.741483302 +0000 UTC m=+2095.987585508" watchObservedRunningTime="2026-07-28 17:45:40.742176786 +0000 UTC m=+2095.988279002"
Jul 28 17:45:46.108622 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:46.108594    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:45:46.115451 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:45:46.115429    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:46:11.729830 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:46:11.729798    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k"
Jul 28 17:47:43.966296 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:47:43.966254    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k"]
Jul 28 17:47:51.729741 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:47:51.729696    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k" podUID="33f98552-a235-4394-a477-f72a9e944f99" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 17:47:58.987421 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:47:58.987353    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k" podUID="33f98552-a235-4394-a477-f72a9e944f99" containerName="main" containerID="cri-o://aec745a8f24d247d7b876420ef25a44cd444b4fab93d4a18d7059b89c15c11a1" gracePeriod=45
Jul 28 17:47:59.237162 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:47:59.237118    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k"
Jul 28 17:47:59.310768 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:47:59.310720    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrn8l\" (UniqueName: \"kubernetes.io/projected/33f98552-a235-4394-a477-f72a9e944f99-kube-api-access-zrn8l\") pod \"33f98552-a235-4394-a477-f72a9e944f99\" (UID: \"33f98552-a235-4394-a477-f72a9e944f99\") "
Jul 28 17:47:59.310768 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:47:59.310770    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33f98552-a235-4394-a477-f72a9e944f99-tls-certs\") pod \"33f98552-a235-4394-a477-f72a9e944f99\" (UID: \"33f98552-a235-4394-a477-f72a9e944f99\") "
Jul 28 17:47:59.313050 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:47:59.313014    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33f98552-a235-4394-a477-f72a9e944f99-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "33f98552-a235-4394-a477-f72a9e944f99" (UID: "33f98552-a235-4394-a477-f72a9e944f99"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:47:59.313050 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:47:59.313022    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33f98552-a235-4394-a477-f72a9e944f99-kube-api-access-zrn8l" (OuterVolumeSpecName: "kube-api-access-zrn8l") pod "33f98552-a235-4394-a477-f72a9e944f99" (UID: "33f98552-a235-4394-a477-f72a9e944f99"). InnerVolumeSpecName "kube-api-access-zrn8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:47:59.390783 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:47:59.390746    2567 generic.go:356] "Generic (PLEG): container finished" podID="33f98552-a235-4394-a477-f72a9e944f99" containerID="aec745a8f24d247d7b876420ef25a44cd444b4fab93d4a18d7059b89c15c11a1" exitCode=0
Jul 28 17:47:59.390973 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:47:59.390793    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k"
Jul 28 17:47:59.390973 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:47:59.390827    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k" event={"ID":"33f98552-a235-4394-a477-f72a9e944f99","Type":"ContainerDied","Data":"aec745a8f24d247d7b876420ef25a44cd444b4fab93d4a18d7059b89c15c11a1"}
Jul 28 17:47:59.390973 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:47:59.390881    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k" event={"ID":"33f98552-a235-4394-a477-f72a9e944f99","Type":"ContainerDied","Data":"ede4f8b2a6b19f1550a52d41c54da1c0d09e67408a3093dc39d6b91b744a7a7e"}
Jul 28 17:47:59.390973 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:47:59.390903    2567 scope.go:117] "RemoveContainer" containerID="aec745a8f24d247d7b876420ef25a44cd444b4fab93d4a18d7059b89c15c11a1"
Jul 28 17:47:59.399919 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:47:59.399898    2567 scope.go:117] "RemoveContainer" containerID="aec745a8f24d247d7b876420ef25a44cd444b4fab93d4a18d7059b89c15c11a1"
Jul 28 17:47:59.400249 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:47:59.400229    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aec745a8f24d247d7b876420ef25a44cd444b4fab93d4a18d7059b89c15c11a1\": container with ID starting with aec745a8f24d247d7b876420ef25a44cd444b4fab93d4a18d7059b89c15c11a1 not found: ID does not exist" containerID="aec745a8f24d247d7b876420ef25a44cd444b4fab93d4a18d7059b89c15c11a1"
Jul 28 17:47:59.400299 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:47:59.400259    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aec745a8f24d247d7b876420ef25a44cd444b4fab93d4a18d7059b89c15c11a1"} err="failed to get container status \"aec745a8f24d247d7b876420ef25a44cd444b4fab93d4a18d7059b89c15c11a1\": rpc error: code = NotFound desc = could not find container \"aec745a8f24d247d7b876420ef25a44cd444b4fab93d4a18d7059b89c15c11a1\": container with ID starting with aec745a8f24d247d7b876420ef25a44cd444b4fab93d4a18d7059b89c15c11a1 not found: ID does not exist"
Jul 28 17:47:59.411530 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:47:59.411466    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrn8l\" (UniqueName: \"kubernetes.io/projected/33f98552-a235-4394-a477-f72a9e944f99-kube-api-access-zrn8l\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:47:59.411530 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:47:59.411522    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33f98552-a235-4394-a477-f72a9e944f99-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:47:59.413416 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:47:59.413381    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k"]
Jul 28 17:47:59.417236 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:47:59.417204    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-grm7k"]
Jul 28 17:48:01.286512 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:01.286463    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33f98552-a235-4394-a477-f72a9e944f99" path="/var/lib/kubelet/pods/33f98552-a235-4394-a477-f72a9e944f99/volumes"
Jul 28 17:48:06.425722 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.425689    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"]
Jul 28 17:48:06.426114 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.425973    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33f98552-a235-4394-a477-f72a9e944f99" containerName="main"
Jul 28 17:48:06.426114 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.425984    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="33f98552-a235-4394-a477-f72a9e944f99" containerName="main"
Jul 28 17:48:06.426114 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.426037    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="33f98552-a235-4394-a477-f72a9e944f99" containerName="main"
Jul 28 17:48:06.430306 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.430284    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:06.433632 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.433611    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"kube-root-ca.crt\""
Jul 28 17:48:06.435216 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.435196    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs\""
Jul 28 17:48:06.435367 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.435248    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"default-dockercfg-s22hh\""
Jul 28 17:48:06.435367 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.435254    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"openshift-service-ca.crt\""
Jul 28 17:48:06.440710 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.440690    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"]
Jul 28 17:48:06.463858 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.463817    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:06.463858 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.463858    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:06.464078 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.463884    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:06.464078 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.463906    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs85g\" (UniqueName: \"kubernetes.io/projected/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-kube-api-access-bs85g\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:06.464078 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.463943    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:06.464078 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.464010    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:06.565126 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.565089    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:06.565389 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.565147    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:06.565389 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.565193    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:06.565389 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.565215    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:06.565389 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.565248    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:06.565389 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.565271    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs85g\" (UniqueName: \"kubernetes.io/projected/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-kube-api-access-bs85g\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:06.565704 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.565583    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:06.565763 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.565742    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:06.565880 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.565860    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:06.567598 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.567569    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:06.567839 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.567824    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:06.575348 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.575327    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs85g\" (UniqueName: \"kubernetes.io/projected/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-kube-api-access-bs85g\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:06.741241 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.741092    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:06.883969 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:06.883936    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"]
Jul 28 17:48:06.885886 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:48:06.885854    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b7ed013_acd2_4d99_a4f3_f457cf4c6e83.slice/crio-5128070e422455c3e32c43bd71b5e1945a758a5b638c834bdc9f1e1febe72dbd WatchSource:0}: Error finding container 5128070e422455c3e32c43bd71b5e1945a758a5b638c834bdc9f1e1febe72dbd: Status 404 returned error can't find the container with id 5128070e422455c3e32c43bd71b5e1945a758a5b638c834bdc9f1e1febe72dbd
Jul 28 17:48:07.431750 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:07.431711    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh" event={"ID":"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83","Type":"ContainerStarted","Data":"3f7a63ab76d042957d5d33f33827e4297f0f000475318c020240e7bba6389d7d"}
Jul 28 17:48:07.432158 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:07.431767    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh" event={"ID":"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83","Type":"ContainerStarted","Data":"5128070e422455c3e32c43bd71b5e1945a758a5b638c834bdc9f1e1febe72dbd"}
Jul 28 17:48:07.454394 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:07.454332    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh" podStartSLOduration=1.454315447 podStartE2EDuration="1.454315447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:48:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:48:07.452607353 +0000 UTC m=+2242.698709560" watchObservedRunningTime="2026-07-28 17:48:07.454315447 +0000 UTC m=+2242.700417653"
Jul 28 17:48:07.741650 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:07.741555    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:16.741593 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:16.741550    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:16.754324 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:16.754289    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:17.310531 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:17.310467    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt"]
Jul 28 17:48:17.314065 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:17.314038    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt"
Jul 28 17:48:17.317621 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:17.317585    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4\"/\"default-dockercfg-jt48n\""
Jul 28 17:48:17.319052 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:17.319024    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4\"/\"kube-root-ca.crt\""
Jul 28 17:48:17.319052 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:17.319042    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4\"/\"autoscale-hpa-deploy-kserve-self-signed-certs\""
Jul 28 17:48:17.319257 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:17.319091    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4\"/\"openshift-service-ca.crt\""
Jul 28 17:48:17.319257 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:17.319118    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4\"/\"autoscale-hpa-deploy-epp-sa-dockercfg-gtzvl\""
Jul 28 17:48:17.325446 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:17.325414    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt"]
Jul 28 17:48:17.469171 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:17.469108    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae2cd460-12c0-468b-a51d-e4d0a32620cb-tls-certs\") pod \"autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt\" (UID: \"ae2cd460-12c0-468b-a51d-e4d0a32620cb\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt"
Jul 28 17:48:17.469348 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:17.469208    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gwqs\" (UniqueName: \"kubernetes.io/projected/ae2cd460-12c0-468b-a51d-e4d0a32620cb-kube-api-access-7gwqs\") pod \"autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt\" (UID: \"ae2cd460-12c0-468b-a51d-e4d0a32620cb\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt"
Jul 28 17:48:17.498026 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:17.497993    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:17.570199 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:17.570086    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae2cd460-12c0-468b-a51d-e4d0a32620cb-tls-certs\") pod \"autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt\" (UID: \"ae2cd460-12c0-468b-a51d-e4d0a32620cb\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt"
Jul 28 17:48:17.570375 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:17.570309    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gwqs\" (UniqueName: \"kubernetes.io/projected/ae2cd460-12c0-468b-a51d-e4d0a32620cb-kube-api-access-7gwqs\") pod \"autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt\" (UID: \"ae2cd460-12c0-468b-a51d-e4d0a32620cb\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt"
Jul 28 17:48:17.572934 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:17.572902    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae2cd460-12c0-468b-a51d-e4d0a32620cb-tls-certs\") pod \"autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt\" (UID: \"ae2cd460-12c0-468b-a51d-e4d0a32620cb\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt"
Jul 28 17:48:17.581279 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:17.581241    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gwqs\" (UniqueName: \"kubernetes.io/projected/ae2cd460-12c0-468b-a51d-e4d0a32620cb-kube-api-access-7gwqs\") pod \"autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt\" (UID: \"ae2cd460-12c0-468b-a51d-e4d0a32620cb\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt"
Jul 28 17:48:17.627061 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:17.627021    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt"
Jul 28 17:48:17.778622 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:17.778584    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt"]
Jul 28 17:48:17.781981 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:48:17.781948    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae2cd460_12c0_468b_a51d_e4d0a32620cb.slice/crio-88bee58473953aa045d5fc179b39eacc89131f0e47ea1f55c8b6244ee01c6260 WatchSource:0}: Error finding container 88bee58473953aa045d5fc179b39eacc89131f0e47ea1f55c8b6244ee01c6260: Status 404 returned error can't find the container with id 88bee58473953aa045d5fc179b39eacc89131f0e47ea1f55c8b6244ee01c6260
Jul 28 17:48:18.490790 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:18.490750    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt" event={"ID":"ae2cd460-12c0-468b-a51d-e4d0a32620cb","Type":"ContainerStarted","Data":"56ad6d1f4e3a030aae17e8c57fd092e1e4e8185400592fd6c052c352de9a0870"}
Jul 28 17:48:18.491001 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:18.490812    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt" event={"ID":"ae2cd460-12c0-468b-a51d-e4d0a32620cb","Type":"ContainerStarted","Data":"88bee58473953aa045d5fc179b39eacc89131f0e47ea1f55c8b6244ee01c6260"}
Jul 28 17:48:18.491001 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:18.490982    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt"
Jul 28 17:48:18.515854 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:18.515788    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt" podStartSLOduration=1.5157681680000001 podStartE2EDuration="1.515768168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:48:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:48:18.513348482 +0000 UTC m=+2253.759450712" watchObservedRunningTime="2026-07-28 17:48:18.515768168 +0000 UTC m=+2253.761870376"
Jul 28 17:48:48.908055 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:48.908021    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"]
Jul 28 17:48:48.917409 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:48:48.917351    2567 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 28 17:48:48.917409 ip-10-0-133-215 kubenswrapper[2567]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 17:48:48.917409 ip-10-0-133-215 kubenswrapper[2567]:         , stdout: , stderr: , exit code -1
Jul 28 17:48:48.917409 ip-10-0-133-215 kubenswrapper[2567]:  > containerID="3f7a63ab76d042957d5d33f33827e4297f0f000475318c020240e7bba6389d7d" cmd=["/bin/sleep","15"]
Jul 28 17:48:48.917749 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:48.917428    2567 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh" message="" err=<
Jul 28 17:48:48.917749 ip-10-0-133-215 kubenswrapper[2567]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 17:48:48.917749 ip-10-0-133-215 kubenswrapper[2567]:         , stdout: , stderr: , exit code -1
Jul 28 17:48:48.917749 ip-10-0-133-215 kubenswrapper[2567]:  >
Jul 28 17:48:48.917749 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:48:48.917447    2567 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 28 17:48:48.917749 ip-10-0-133-215 kubenswrapper[2567]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 17:48:48.917749 ip-10-0-133-215 kubenswrapper[2567]:         , stdout: , stderr: , exit code -1
Jul 28 17:48:48.917749 ip-10-0-133-215 kubenswrapper[2567]:  > pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh" podUID="6b7ed013-acd2-4d99-a4f3-f457cf4c6e83" containerName="main" containerID="cri-o://3f7a63ab76d042957d5d33f33827e4297f0f000475318c020240e7bba6389d7d"
Jul 28 17:48:48.917749 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:48.917529    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh" podUID="6b7ed013-acd2-4d99-a4f3-f457cf4c6e83" containerName="main" containerID="cri-o://3f7a63ab76d042957d5d33f33827e4297f0f000475318c020240e7bba6389d7d" gracePeriod=60
Jul 28 17:48:49.267070 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.267039    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:49.347455 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.347417    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-dshm\") pod \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") "
Jul 28 17:48:49.347675 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.347473    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-home\") pod \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") "
Jul 28 17:48:49.347675 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.347564    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-tmp-dir\") pod \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") "
Jul 28 17:48:49.347675 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.347598    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs85g\" (UniqueName: \"kubernetes.io/projected/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-kube-api-access-bs85g\") pod \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") "
Jul 28 17:48:49.347675 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.347624    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-tls-certs\") pod \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") "
Jul 28 17:48:49.347675 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.347655    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-model-cache\") pod \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\" (UID: \"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83\") "
Jul 28 17:48:49.347930 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.347785    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-home" (OuterVolumeSpecName: "home") pod "6b7ed013-acd2-4d99-a4f3-f457cf4c6e83" (UID: "6b7ed013-acd2-4d99-a4f3-f457cf4c6e83"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:48:49.347930 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.347829    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6b7ed013-acd2-4d99-a4f3-f457cf4c6e83" (UID: "6b7ed013-acd2-4d99-a4f3-f457cf4c6e83"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:48:49.347930 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.347913    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-tmp-dir\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:48:49.348087 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.347933    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-home\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:48:49.348087 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.348016    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-model-cache" (OuterVolumeSpecName: "model-cache") pod "6b7ed013-acd2-4d99-a4f3-f457cf4c6e83" (UID: "6b7ed013-acd2-4d99-a4f3-f457cf4c6e83"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:48:49.350457 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.350239    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-dshm" (OuterVolumeSpecName: "dshm") pod "6b7ed013-acd2-4d99-a4f3-f457cf4c6e83" (UID: "6b7ed013-acd2-4d99-a4f3-f457cf4c6e83"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:48:49.350615 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.350462    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-kube-api-access-bs85g" (OuterVolumeSpecName: "kube-api-access-bs85g") pod "6b7ed013-acd2-4d99-a4f3-f457cf4c6e83" (UID: "6b7ed013-acd2-4d99-a4f3-f457cf4c6e83"). InnerVolumeSpecName "kube-api-access-bs85g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:48:49.350615 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.350536    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6b7ed013-acd2-4d99-a4f3-f457cf4c6e83" (UID: "6b7ed013-acd2-4d99-a4f3-f457cf4c6e83"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:48:49.449348 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.449252    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs85g\" (UniqueName: \"kubernetes.io/projected/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-kube-api-access-bs85g\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:48:49.449348 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.449285    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:48:49.449348 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.449295    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-model-cache\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:48:49.449348 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.449303    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83-dshm\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 17:48:49.498607 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.498576    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt"
Jul 28 17:48:49.648293 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.648258    2567 generic.go:356] "Generic (PLEG): container finished" podID="6b7ed013-acd2-4d99-a4f3-f457cf4c6e83" containerID="3f7a63ab76d042957d5d33f33827e4297f0f000475318c020240e7bba6389d7d" exitCode=0
Jul 28 17:48:49.648481 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.648304    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"
Jul 28 17:48:49.648481 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.648344    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh" event={"ID":"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83","Type":"ContainerDied","Data":"3f7a63ab76d042957d5d33f33827e4297f0f000475318c020240e7bba6389d7d"}
Jul 28 17:48:49.648481 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.648397    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh" event={"ID":"6b7ed013-acd2-4d99-a4f3-f457cf4c6e83","Type":"ContainerDied","Data":"5128070e422455c3e32c43bd71b5e1945a758a5b638c834bdc9f1e1febe72dbd"}
Jul 28 17:48:49.648481 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.648419    2567 scope.go:117] "RemoveContainer" containerID="3f7a63ab76d042957d5d33f33827e4297f0f000475318c020240e7bba6389d7d"
Jul 28 17:48:49.657467 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.657447    2567 scope.go:117] "RemoveContainer" containerID="3f7a63ab76d042957d5d33f33827e4297f0f000475318c020240e7bba6389d7d"
Jul 28 17:48:49.657746 ip-10-0-133-215 kubenswrapper[2567]: E0728 17:48:49.657729    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f7a63ab76d042957d5d33f33827e4297f0f000475318c020240e7bba6389d7d\": container with ID starting with 3f7a63ab76d042957d5d33f33827e4297f0f000475318c020240e7bba6389d7d not found: ID does not exist" containerID="3f7a63ab76d042957d5d33f33827e4297f0f000475318c020240e7bba6389d7d"
Jul 28 17:48:49.657799 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.657753    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f7a63ab76d042957d5d33f33827e4297f0f000475318c020240e7bba6389d7d"} err="failed to get container status \"3f7a63ab76d042957d5d33f33827e4297f0f000475318c020240e7bba6389d7d\": rpc error: code = NotFound desc = could not find container \"3f7a63ab76d042957d5d33f33827e4297f0f000475318c020240e7bba6389d7d\": container with ID starting with 3f7a63ab76d042957d5d33f33827e4297f0f000475318c020240e7bba6389d7d not found: ID does not exist"
Jul 28 17:48:49.674676 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.674643    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"]
Jul 28 17:48:49.679740 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:49.679710    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-g8lsh"]
Jul 28 17:48:51.287051 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:48:51.287017    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b7ed013-acd2-4d99-a4f3-f457cf4c6e83" path="/var/lib/kubelet/pods/6b7ed013-acd2-4d99-a4f3-f457cf4c6e83/volumes"
Jul 28 17:50:46.140662 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:50:46.140628    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:50:46.149581 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:50:46.149554    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:55:46.168574 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:55:46.168483    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:55:46.179011 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:55:46.178986    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 17:56:56.835342 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:56.835303    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6"]
Jul 28 17:56:56.835816 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:56.835664    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b7ed013-acd2-4d99-a4f3-f457cf4c6e83" containerName="main"
Jul 28 17:56:56.835816 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:56.835679    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b7ed013-acd2-4d99-a4f3-f457cf4c6e83" containerName="main"
Jul 28 17:56:56.835816 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:56.835730    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b7ed013-acd2-4d99-a4f3-f457cf4c6e83" containerName="main"
Jul 28 17:56:56.838868 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:56.838834    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6"
Jul 28 17:56:56.842139 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:56.842100    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-48639af5\"/\"kube-root-ca.crt\""
Jul 28 17:56:56.843661 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:56.843635    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-48639af5\"/\"openshift-service-ca.crt\""
Jul 28 17:56:56.843824 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:56.843710    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-48639af5\"/\"llmisvc-model-pvc-router-manage-e8706282-epp-sa-dockercfg-84jn5\""
Jul 28 17:56:56.843824 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:56.843795    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-48639af5\"/\"llmisvb19f98874e050eec8ca94d49676113f0-kserve-self-signed-certs\""
Jul 28 17:56:56.843824 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:56.843818    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-48639af5\"/\"default-dockercfg-schcf\""
Jul 28 17:56:56.853961 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:56.853917    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6"]
Jul 28 17:56:56.940484 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:56.940432    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f464d411-d81e-441f-ac1a-05a9d9a28295-tls-certs\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6\" (UID: \"f464d411-d81e-441f-ac1a-05a9d9a28295\") " pod="e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6"
Jul 28 17:56:56.940484 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:56.940485    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wh7v\" (UniqueName: \"kubernetes.io/projected/f464d411-d81e-441f-ac1a-05a9d9a28295-kube-api-access-7wh7v\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6\" (UID: \"f464d411-d81e-441f-ac1a-05a9d9a28295\") " pod="e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6"
Jul 28 17:56:57.041452 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:57.041405    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f464d411-d81e-441f-ac1a-05a9d9a28295-tls-certs\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6\" (UID: \"f464d411-d81e-441f-ac1a-05a9d9a28295\") " pod="e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6"
Jul 28 17:56:57.041452 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:57.041457    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wh7v\" (UniqueName: \"kubernetes.io/projected/f464d411-d81e-441f-ac1a-05a9d9a28295-kube-api-access-7wh7v\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6\" (UID: \"f464d411-d81e-441f-ac1a-05a9d9a28295\") " pod="e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6"
Jul 28 17:56:57.044073 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:57.044034    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f464d411-d81e-441f-ac1a-05a9d9a28295-tls-certs\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6\" (UID: \"f464d411-d81e-441f-ac1a-05a9d9a28295\") " pod="e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6"
Jul 28 17:56:57.051656 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:57.051619    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wh7v\" (UniqueName: \"kubernetes.io/projected/f464d411-d81e-441f-ac1a-05a9d9a28295-kube-api-access-7wh7v\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6\" (UID: \"f464d411-d81e-441f-ac1a-05a9d9a28295\") " pod="e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6"
Jul 28 17:56:57.150709 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:57.150613    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6"
Jul 28 17:56:57.293418 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:57.293383    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6"]
Jul 28 17:56:57.295740 ip-10-0-133-215 kubenswrapper[2567]: W0728 17:56:57.295706    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf464d411_d81e_441f_ac1a_05a9d9a28295.slice/crio-6c4dd41b36dbacf5d4ef34ab850611fbc3f3164b475746c8ef0e759557804fa0 WatchSource:0}: Error finding container 6c4dd41b36dbacf5d4ef34ab850611fbc3f3164b475746c8ef0e759557804fa0: Status 404 returned error can't find the container with id 6c4dd41b36dbacf5d4ef34ab850611fbc3f3164b475746c8ef0e759557804fa0
Jul 28 17:56:58.058332 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:58.058284    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6" event={"ID":"f464d411-d81e-441f-ac1a-05a9d9a28295","Type":"ContainerStarted","Data":"29bbff7e3ff931da4b50203f6f0476b1cb6843817d25d534dd86fa2cc2b24ed8"}
Jul 28 17:56:58.058815 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:58.058347    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6" event={"ID":"f464d411-d81e-441f-ac1a-05a9d9a28295","Type":"ContainerStarted","Data":"6c4dd41b36dbacf5d4ef34ab850611fbc3f3164b475746c8ef0e759557804fa0"}
Jul 28 17:56:58.058815 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:58.058456    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6"
Jul 28 17:56:58.079618 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:56:58.079559    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6" podStartSLOduration=2.07953982 podStartE2EDuration="2.07953982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:56:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:56:58.07816821 +0000 UTC m=+2773.324270416" watchObservedRunningTime="2026-07-28 17:56:58.07953982 +0000 UTC m=+2773.325642026"
Jul 28 17:57:29.064753 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:57:29.064672    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6"
Jul 28 17:59:48.768232 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:59:48.768193    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6"]
Jul 28 17:59:49.064923 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:59:49.064826    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6" podUID="f464d411-d81e-441f-ac1a-05a9d9a28295" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 17:59:59.064657 ip-10-0-133-215 kubenswrapper[2567]: I0728 17:59:59.064615    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6" podUID="f464d411-d81e-441f-ac1a-05a9d9a28295" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 18:00:03.789193 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:03.789129    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6" podUID="f464d411-d81e-441f-ac1a-05a9d9a28295" containerName="main" containerID="cri-o://29bbff7e3ff931da4b50203f6f0476b1cb6843817d25d534dd86fa2cc2b24ed8" gracePeriod=45
Jul 28 18:00:03.960263 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:03.960222    2567 generic.go:356] "Generic (PLEG): container finished" podID="f464d411-d81e-441f-ac1a-05a9d9a28295" containerID="29bbff7e3ff931da4b50203f6f0476b1cb6843817d25d534dd86fa2cc2b24ed8" exitCode=0
Jul 28 18:00:03.960452 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:03.960290    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6" event={"ID":"f464d411-d81e-441f-ac1a-05a9d9a28295","Type":"ContainerDied","Data":"29bbff7e3ff931da4b50203f6f0476b1cb6843817d25d534dd86fa2cc2b24ed8"}
Jul 28 18:00:04.049423 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:04.049349    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6"
Jul 28 18:00:04.110255 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:04.110212    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f464d411-d81e-441f-ac1a-05a9d9a28295-tls-certs\") pod \"f464d411-d81e-441f-ac1a-05a9d9a28295\" (UID: \"f464d411-d81e-441f-ac1a-05a9d9a28295\") "
Jul 28 18:00:04.110255 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:04.110255    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wh7v\" (UniqueName: \"kubernetes.io/projected/f464d411-d81e-441f-ac1a-05a9d9a28295-kube-api-access-7wh7v\") pod \"f464d411-d81e-441f-ac1a-05a9d9a28295\" (UID: \"f464d411-d81e-441f-ac1a-05a9d9a28295\") "
Jul 28 18:00:04.112671 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:04.112639    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f464d411-d81e-441f-ac1a-05a9d9a28295-kube-api-access-7wh7v" (OuterVolumeSpecName: "kube-api-access-7wh7v") pod "f464d411-d81e-441f-ac1a-05a9d9a28295" (UID: "f464d411-d81e-441f-ac1a-05a9d9a28295"). InnerVolumeSpecName "kube-api-access-7wh7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:00:04.112671 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:04.112665    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f464d411-d81e-441f-ac1a-05a9d9a28295-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f464d411-d81e-441f-ac1a-05a9d9a28295" (UID: "f464d411-d81e-441f-ac1a-05a9d9a28295"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:00:04.211096 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:04.211054    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f464d411-d81e-441f-ac1a-05a9d9a28295-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:00:04.211096 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:04.211092    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wh7v\" (UniqueName: \"kubernetes.io/projected/f464d411-d81e-441f-ac1a-05a9d9a28295-kube-api-access-7wh7v\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:00:04.965979 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:04.965949    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6"
Jul 28 18:00:04.966520 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:04.965994    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6" event={"ID":"f464d411-d81e-441f-ac1a-05a9d9a28295","Type":"ContainerDied","Data":"6c4dd41b36dbacf5d4ef34ab850611fbc3f3164b475746c8ef0e759557804fa0"}
Jul 28 18:00:04.966520 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:04.966051    2567 scope.go:117] "RemoveContainer" containerID="29bbff7e3ff931da4b50203f6f0476b1cb6843817d25d534dd86fa2cc2b24ed8"
Jul 28 18:00:04.990277 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:04.990240    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6"]
Jul 28 18:00:04.994019 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:04.993985    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-48639af5/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schehr7h6"]
Jul 28 18:00:05.287663 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:05.287628    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f464d411-d81e-441f-ac1a-05a9d9a28295" path="/var/lib/kubelet/pods/f464d411-d81e-441f-ac1a-05a9d9a28295/volumes"
Jul 28 18:00:46.198346 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:46.198314    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:00:46.209627 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:46.209599    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:00:49.938767 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:49.938728    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"]
Jul 28 18:00:49.939172 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:49.939042    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f464d411-d81e-441f-ac1a-05a9d9a28295" containerName="main"
Jul 28 18:00:49.939172 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:49.939052    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f464d411-d81e-441f-ac1a-05a9d9a28295" containerName="main"
Jul 28 18:00:49.939172 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:49.939100    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f464d411-d81e-441f-ac1a-05a9d9a28295" containerName="main"
Jul 28 18:00:49.943181 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:49.943158    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:00:49.948873 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:49.948845    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-7c3633d1\"/\"kube-root-ca.crt\""
Jul 28 18:00:49.949057 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:49.948924    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-7c3633d1\"/\"openshift-service-ca.crt\""
Jul 28 18:00:49.949057 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:49.948905    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-7c3633d1\"/\"default-dockercfg-9d7kz\""
Jul 28 18:00:49.949057 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:49.948904    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-7c3633d1\"/\"llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs\""
Jul 28 18:00:49.949057 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:49.948989    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-7c3633d1\"/\"llmisvc-model-pvc-router-manage-2577e7-b09f84b1-dockercfg-6lzqk\""
Jul 28 18:00:49.972429 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:49.972390    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"]
Jul 28 18:00:50.099827 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:50.099785    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-home\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:00:50.099998 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:50.099852    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-dshm\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:00:50.099998 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:50.099891    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-model-cache\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:00:50.099998 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:50.099929    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-tls-certs\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:00:50.099998 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:50.099972    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvl89\" (UniqueName: \"kubernetes.io/projected/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-kube-api-access-jvl89\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:00:50.100147 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:50.100054    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-tmp-dir\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:00:50.201195 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:50.201106    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-tmp-dir\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:00:50.201195 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:50.201150    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-home\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:00:50.201195 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:50.201176    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-dshm\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:00:50.201458 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:50.201361    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-model-cache\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:00:50.201458 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:50.201418    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-tls-certs\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:00:50.201589 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:50.201478    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvl89\" (UniqueName: \"kubernetes.io/projected/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-kube-api-access-jvl89\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:00:50.201652 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:50.201635    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-home\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:00:50.201772 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:50.201675    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-tmp-dir\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:00:50.201868 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:50.201841    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-model-cache\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:00:50.203444 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:50.203418    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-dshm\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:00:50.203950 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:50.203927    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-tls-certs\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:00:50.213089 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:00:50.213060    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvl89\" (UniqueName: \"kubernetes.io/projected/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-kube-api-access-jvl89\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:02:52.953669 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:02:52.953614    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1" podUID="9f4d40ff-ce06-48c5-a57d-584aa80e8b49"
Jul 28 18:02:53.792798 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:02:53.792764    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:02:53.961875 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:02:53.961831    2567 secret.go:189] Couldn't get secret e2e-test-llm-inference-service-7c3633d1/llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs: secret "llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs" not found
Jul 28 18:02:53.962353 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:02:53.961956    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-tls-certs podName:9f4d40ff-ce06-48c5-a57d-584aa80e8b49 nodeName:}" failed. No retries permitted until 2026-07-28 18:02:54.461931689 +0000 UTC m=+3129.708033887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-tls-certs") pod "llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1" (UID: "9f4d40ff-ce06-48c5-a57d-584aa80e8b49") : secret "llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs" not found
Jul 28 18:02:54.466597 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:02:54.466555    2567 secret.go:189] Couldn't get secret e2e-test-llm-inference-service-7c3633d1/llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs: secret "llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs" not found
Jul 28 18:02:54.466789 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:02:54.466641    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-tls-certs podName:9f4d40ff-ce06-48c5-a57d-584aa80e8b49 nodeName:}" failed. No retries permitted until 2026-07-28 18:02:55.466625132 +0000 UTC m=+3130.712727316 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-tls-certs") pod "llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1" (UID: "9f4d40ff-ce06-48c5-a57d-584aa80e8b49") : secret "llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs" not found
Jul 28 18:02:55.476801 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:02:55.476759    2567 secret.go:189] Couldn't get secret e2e-test-llm-inference-service-7c3633d1/llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs: secret "llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs" not found
Jul 28 18:02:55.477215 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:02:55.476850    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-tls-certs podName:9f4d40ff-ce06-48c5-a57d-584aa80e8b49 nodeName:}" failed. No retries permitted until 2026-07-28 18:02:57.476830146 +0000 UTC m=+3132.722932331 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-tls-certs") pod "llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1" (UID: "9f4d40ff-ce06-48c5-a57d-584aa80e8b49") : secret "llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs" not found
Jul 28 18:02:57.497153 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:02:57.497117    2567 secret.go:189] Couldn't get secret e2e-test-llm-inference-service-7c3633d1/llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs: secret "llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs" not found
Jul 28 18:02:57.497619 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:02:57.497196    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-tls-certs podName:9f4d40ff-ce06-48c5-a57d-584aa80e8b49 nodeName:}" failed. No retries permitted until 2026-07-28 18:03:01.497180765 +0000 UTC m=+3136.743282950 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-tls-certs") pod "llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1" (UID: "9f4d40ff-ce06-48c5-a57d-584aa80e8b49") : secret "llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs" not found
Jul 28 18:03:00.435119 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:00.435079    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"]
Jul 28 18:03:00.435639 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:03:00.435441    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context canceled" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1" podUID="9f4d40ff-ce06-48c5-a57d-584aa80e8b49"
Jul 28 18:03:00.826470 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:00.826438    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:03:00.834281 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:00.834250    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:03:01.026002 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:01.025964    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-home\") pod \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") "
Jul 28 18:03:01.026203 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:01.026052    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-model-cache\") pod \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") "
Jul 28 18:03:01.026203 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:01.026096    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvl89\" (UniqueName: \"kubernetes.io/projected/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-kube-api-access-jvl89\") pod \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") "
Jul 28 18:03:01.026203 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:01.026146    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-tmp-dir\") pod \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") "
Jul 28 18:03:01.026203 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:01.026169    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-dshm\") pod \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") "
Jul 28 18:03:01.026203 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:01.026198    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-tls-certs\") pod \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\" (UID: \"9f4d40ff-ce06-48c5-a57d-584aa80e8b49\") "
Jul 28 18:03:01.026484 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:01.026165    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-home" (OuterVolumeSpecName: "home") pod "9f4d40ff-ce06-48c5-a57d-584aa80e8b49" (UID: "9f4d40ff-ce06-48c5-a57d-584aa80e8b49"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:03:01.026484 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:01.026279    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-model-cache" (OuterVolumeSpecName: "model-cache") pod "9f4d40ff-ce06-48c5-a57d-584aa80e8b49" (UID: "9f4d40ff-ce06-48c5-a57d-584aa80e8b49"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:03:01.026484 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:01.026366    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9f4d40ff-ce06-48c5-a57d-584aa80e8b49" (UID: "9f4d40ff-ce06-48c5-a57d-584aa80e8b49"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:03:01.026484 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:01.026446    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-model-cache\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:03:01.026484 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:01.026459    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-tmp-dir\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:03:01.026484 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:01.026468    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-home\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:03:01.028484 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:01.028452    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-kube-api-access-jvl89" (OuterVolumeSpecName: "kube-api-access-jvl89") pod "9f4d40ff-ce06-48c5-a57d-584aa80e8b49" (UID: "9f4d40ff-ce06-48c5-a57d-584aa80e8b49"). InnerVolumeSpecName "kube-api-access-jvl89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:03:01.028672 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:01.028648    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-dshm" (OuterVolumeSpecName: "dshm") pod "9f4d40ff-ce06-48c5-a57d-584aa80e8b49" (UID: "9f4d40ff-ce06-48c5-a57d-584aa80e8b49"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:03:01.028778 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:01.028762    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9f4d40ff-ce06-48c5-a57d-584aa80e8b49" (UID: "9f4d40ff-ce06-48c5-a57d-584aa80e8b49"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:03:01.127313 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:01.127214    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvl89\" (UniqueName: \"kubernetes.io/projected/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-kube-api-access-jvl89\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:03:01.127313 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:01.127250    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-dshm\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:03:01.127313 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:01.127259    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f4d40ff-ce06-48c5-a57d-584aa80e8b49-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:03:01.830950 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:01.830916    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 28 18:03:01.867067 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:01.867032    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"]
Jul 28 18:03:01.871170 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:01.871135    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"]
Jul 28 18:03:03.287632 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:03.287591    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f4d40ff-ce06-48c5-a57d-584aa80e8b49" path="/var/lib/kubelet/pods/9f4d40ff-ce06-48c5-a57d-584aa80e8b49/volumes"
Jul 28 18:03:06.285328 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:06.285285    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt"]
Jul 28 18:03:09.498184 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:09.498142    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt" podUID="ae2cd460-12c0-468b-a51d-e4d0a32620cb" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 18:03:19.497849 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:19.497805    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt" podUID="ae2cd460-12c0-468b-a51d-e4d0a32620cb" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 18:03:21.304959 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:21.304892    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt" podUID="ae2cd460-12c0-468b-a51d-e4d0a32620cb" containerName="main" containerID="cri-o://56ad6d1f4e3a030aae17e8c57fd092e1e4e8185400592fd6c052c352de9a0870" gracePeriod=45
Jul 28 18:03:21.567386 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:21.567287    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt"
Jul 28 18:03:21.596584 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:21.596539    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae2cd460-12c0-468b-a51d-e4d0a32620cb-tls-certs\") pod \"ae2cd460-12c0-468b-a51d-e4d0a32620cb\" (UID: \"ae2cd460-12c0-468b-a51d-e4d0a32620cb\") "
Jul 28 18:03:21.596795 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:21.596674    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7gwqs\" (UniqueName: \"kubernetes.io/projected/ae2cd460-12c0-468b-a51d-e4d0a32620cb-kube-api-access-7gwqs\") pod \"ae2cd460-12c0-468b-a51d-e4d0a32620cb\" (UID: \"ae2cd460-12c0-468b-a51d-e4d0a32620cb\") "
Jul 28 18:03:21.598987 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:21.598951    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae2cd460-12c0-468b-a51d-e4d0a32620cb-kube-api-access-7gwqs" (OuterVolumeSpecName: "kube-api-access-7gwqs") pod "ae2cd460-12c0-468b-a51d-e4d0a32620cb" (UID: "ae2cd460-12c0-468b-a51d-e4d0a32620cb"). InnerVolumeSpecName "kube-api-access-7gwqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:03:21.599132 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:21.598993    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae2cd460-12c0-468b-a51d-e4d0a32620cb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ae2cd460-12c0-468b-a51d-e4d0a32620cb" (UID: "ae2cd460-12c0-468b-a51d-e4d0a32620cb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:03:21.697881 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:21.697837    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gwqs\" (UniqueName: \"kubernetes.io/projected/ae2cd460-12c0-468b-a51d-e4d0a32620cb-kube-api-access-7gwqs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:03:21.697881 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:21.697871    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae2cd460-12c0-468b-a51d-e4d0a32620cb-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:03:21.933700 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:21.933593    2567 generic.go:356] "Generic (PLEG): container finished" podID="ae2cd460-12c0-468b-a51d-e4d0a32620cb" containerID="56ad6d1f4e3a030aae17e8c57fd092e1e4e8185400592fd6c052c352de9a0870" exitCode=0
Jul 28 18:03:21.933700 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:21.933638    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt"
Jul 28 18:03:21.933700 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:21.933681    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt" event={"ID":"ae2cd460-12c0-468b-a51d-e4d0a32620cb","Type":"ContainerDied","Data":"56ad6d1f4e3a030aae17e8c57fd092e1e4e8185400592fd6c052c352de9a0870"}
Jul 28 18:03:21.934002 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:21.933734    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt" event={"ID":"ae2cd460-12c0-468b-a51d-e4d0a32620cb","Type":"ContainerDied","Data":"88bee58473953aa045d5fc179b39eacc89131f0e47ea1f55c8b6244ee01c6260"}
Jul 28 18:03:21.934002 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:21.933760    2567 scope.go:117] "RemoveContainer" containerID="56ad6d1f4e3a030aae17e8c57fd092e1e4e8185400592fd6c052c352de9a0870"
Jul 28 18:03:21.943555 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:21.943533    2567 scope.go:117] "RemoveContainer" containerID="56ad6d1f4e3a030aae17e8c57fd092e1e4e8185400592fd6c052c352de9a0870"
Jul 28 18:03:21.943848 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:03:21.943827    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56ad6d1f4e3a030aae17e8c57fd092e1e4e8185400592fd6c052c352de9a0870\": container with ID starting with 56ad6d1f4e3a030aae17e8c57fd092e1e4e8185400592fd6c052c352de9a0870 not found: ID does not exist" containerID="56ad6d1f4e3a030aae17e8c57fd092e1e4e8185400592fd6c052c352de9a0870"
Jul 28 18:03:21.943903 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:21.943858    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56ad6d1f4e3a030aae17e8c57fd092e1e4e8185400592fd6c052c352de9a0870"} err="failed to get container status \"56ad6d1f4e3a030aae17e8c57fd092e1e4e8185400592fd6c052c352de9a0870\": rpc error: code = NotFound desc = could not find container \"56ad6d1f4e3a030aae17e8c57fd092e1e4e8185400592fd6c052c352de9a0870\": container with ID starting with 56ad6d1f4e3a030aae17e8c57fd092e1e4e8185400592fd6c052c352de9a0870 not found: ID does not exist"
Jul 28 18:03:21.956237 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:21.956199    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt"]
Jul 28 18:03:21.960882 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:21.960848    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-router-scheduler-5bbdf87d58-hd4zt"]
Jul 28 18:03:23.289574 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:23.289493    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae2cd460-12c0-468b-a51d-e4d0a32620cb" path="/var/lib/kubelet/pods/ae2cd460-12c0-468b-a51d-e4d0a32620cb/volumes"
Jul 28 18:03:34.367487 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.367447    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25"]
Jul 28 18:03:34.367910 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.367766    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae2cd460-12c0-468b-a51d-e4d0a32620cb" containerName="main"
Jul 28 18:03:34.367910 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.367778    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae2cd460-12c0-468b-a51d-e4d0a32620cb" containerName="main"
Jul 28 18:03:34.367910 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.367842    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae2cd460-12c0-468b-a51d-e4d0a32620cb" containerName="main"
Jul 28 18:03:34.370731 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.370711    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25"
Jul 28 18:03:34.375786 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.375751    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-keda-deployment-b2150d0b\"/\"kube-root-ca.crt\""
Jul 28 18:03:34.375959 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.375776    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-keda-deployment-b2150d0b\"/\"default-dockercfg-gvhzk\""
Jul 28 18:03:34.375959 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.375776    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-keda-deployment-b2150d0b\"/\"autoscale-keda-deploy-epp-sa-dockercfg-lzvgb\""
Jul 28 18:03:34.375959 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.375812    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-keda-deployment-b2150d0b\"/\"openshift-service-ca.crt\""
Jul 28 18:03:34.376146 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.375828    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-keda-deployment-b2150d0b\"/\"autoscale-keda-deploy-kserve-self-signed-certs\""
Jul 28 18:03:34.379447 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.379419    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25"]
Jul 28 18:03:34.500682 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.500646    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e9b20b6-90cb-4c18-98e8-b161c8f299b7-tls-certs\") pod \"autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25\" (UID: \"5e9b20b6-90cb-4c18-98e8-b161c8f299b7\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25"
Jul 28 18:03:34.500865 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.500694    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8jxs\" (UniqueName: \"kubernetes.io/projected/5e9b20b6-90cb-4c18-98e8-b161c8f299b7-kube-api-access-z8jxs\") pod \"autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25\" (UID: \"5e9b20b6-90cb-4c18-98e8-b161c8f299b7\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25"
Jul 28 18:03:34.601957 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.601912    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8jxs\" (UniqueName: \"kubernetes.io/projected/5e9b20b6-90cb-4c18-98e8-b161c8f299b7-kube-api-access-z8jxs\") pod \"autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25\" (UID: \"5e9b20b6-90cb-4c18-98e8-b161c8f299b7\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25"
Jul 28 18:03:34.602178 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.602016    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e9b20b6-90cb-4c18-98e8-b161c8f299b7-tls-certs\") pod \"autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25\" (UID: \"5e9b20b6-90cb-4c18-98e8-b161c8f299b7\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25"
Jul 28 18:03:34.604626 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.604598    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e9b20b6-90cb-4c18-98e8-b161c8f299b7-tls-certs\") pod \"autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25\" (UID: \"5e9b20b6-90cb-4c18-98e8-b161c8f299b7\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25"
Jul 28 18:03:34.611354 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.611316    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8jxs\" (UniqueName: \"kubernetes.io/projected/5e9b20b6-90cb-4c18-98e8-b161c8f299b7-kube-api-access-z8jxs\") pod \"autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25\" (UID: \"5e9b20b6-90cb-4c18-98e8-b161c8f299b7\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25"
Jul 28 18:03:34.681893 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.681786    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25"
Jul 28 18:03:34.831028 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.831000    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25"]
Jul 28 18:03:34.832692 ip-10-0-133-215 kubenswrapper[2567]: W0728 18:03:34.832666    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e9b20b6_90cb_4c18_98e8_b161c8f299b7.slice/crio-bb022e1c7a5ce59e9900a137c3e721a77bd2e59631eedaa681f7f8ea300a3ea1 WatchSource:0}: Error finding container bb022e1c7a5ce59e9900a137c3e721a77bd2e59631eedaa681f7f8ea300a3ea1: Status 404 returned error can't find the container with id bb022e1c7a5ce59e9900a137c3e721a77bd2e59631eedaa681f7f8ea300a3ea1
Jul 28 18:03:34.998102 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.997987    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25" event={"ID":"5e9b20b6-90cb-4c18-98e8-b161c8f299b7","Type":"ContainerStarted","Data":"92d3916516e0d9ec1780b9f8ecef284d9f90255a0857cbcc32a225ea11ce6af5"}
Jul 28 18:03:34.998102 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.998049    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25"
Jul 28 18:03:34.998102 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:34.998067    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25" event={"ID":"5e9b20b6-90cb-4c18-98e8-b161c8f299b7","Type":"ContainerStarted","Data":"bb022e1c7a5ce59e9900a137c3e721a77bd2e59631eedaa681f7f8ea300a3ea1"}
Jul 28 18:03:35.017759 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:03:35.017699    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25" podStartSLOduration=1.017680817 podStartE2EDuration="1.017680817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:03:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:03:35.016675972 +0000 UTC m=+3170.262778179" watchObservedRunningTime="2026-07-28 18:03:35.017680817 +0000 UTC m=+3170.263783023"
Jul 28 18:04:06.004589 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:04:06.004557    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25"
Jul 28 18:05:46.231434 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:05:46.231406    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:05:46.242350 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:05:46.242322    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:06:18.355178 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:06:18.355136    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7"]
Jul 28 18:06:18.359033 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:06:18.359008    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7"
Jul 28 18:06:18.363197 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:06:18.363166    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"default-dockercfg-g9jzg\""
Jul 28 18:06:18.365121 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:06:18.365087    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"openshift-service-ca.crt\""
Jul 28 18:06:18.365121 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:06:18.365113    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-kserve-self-signed-certs\""
Jul 28 18:06:18.365433 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:06:18.365413    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-epp-sa-dockercfg-wwzhq\""
Jul 28 18:06:18.365553 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:06:18.365473    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"kube-root-ca.crt\""
Jul 28 18:06:18.372256 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:06:18.372226    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7"]
Jul 28 18:06:18.442957 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:06:18.442915    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c792f231-37c5-44ff-8c4d-ad89e99908d6-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7\" (UID: \"c792f231-37c5-44ff-8c4d-ad89e99908d6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7"
Jul 28 18:06:18.443154 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:06:18.442980    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99wvt\" (UniqueName: \"kubernetes.io/projected/c792f231-37c5-44ff-8c4d-ad89e99908d6-kube-api-access-99wvt\") pod \"stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7\" (UID: \"c792f231-37c5-44ff-8c4d-ad89e99908d6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7"
Jul 28 18:06:18.547855 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:06:18.547812    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c792f231-37c5-44ff-8c4d-ad89e99908d6-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7\" (UID: \"c792f231-37c5-44ff-8c4d-ad89e99908d6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7"
Jul 28 18:06:18.548078 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:06:18.547946    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99wvt\" (UniqueName: \"kubernetes.io/projected/c792f231-37c5-44ff-8c4d-ad89e99908d6-kube-api-access-99wvt\") pod \"stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7\" (UID: \"c792f231-37c5-44ff-8c4d-ad89e99908d6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7"
Jul 28 18:06:18.550649 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:06:18.550606    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c792f231-37c5-44ff-8c4d-ad89e99908d6-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7\" (UID: \"c792f231-37c5-44ff-8c4d-ad89e99908d6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7"
Jul 28 18:06:18.562273 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:06:18.562237    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99wvt\" (UniqueName: \"kubernetes.io/projected/c792f231-37c5-44ff-8c4d-ad89e99908d6-kube-api-access-99wvt\") pod \"stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7\" (UID: \"c792f231-37c5-44ff-8c4d-ad89e99908d6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7"
Jul 28 18:06:18.671741 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:06:18.671628    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7"
Jul 28 18:06:18.822160 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:06:18.822123    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7"]
Jul 28 18:06:18.824609 ip-10-0-133-215 kubenswrapper[2567]: W0728 18:06:18.824574    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc792f231_37c5_44ff_8c4d_ad89e99908d6.slice/crio-8d5e2eb64375c11b78457228714d5900ed1305b2f81af86fc44855e54c55ea55 WatchSource:0}: Error finding container 8d5e2eb64375c11b78457228714d5900ed1305b2f81af86fc44855e54c55ea55: Status 404 returned error can't find the container with id 8d5e2eb64375c11b78457228714d5900ed1305b2f81af86fc44855e54c55ea55
Jul 28 18:06:19.780083 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:06:19.780047    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7" event={"ID":"c792f231-37c5-44ff-8c4d-ad89e99908d6","Type":"ContainerStarted","Data":"a2c53c2dba120ed74dc71e7c273079f54413995593e569d19b6210c3a3e2a210"}
Jul 28 18:06:19.780541 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:06:19.780111    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7"
Jul 28 18:06:19.780541 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:06:19.780132    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7" event={"ID":"c792f231-37c5-44ff-8c4d-ad89e99908d6","Type":"ContainerStarted","Data":"8d5e2eb64375c11b78457228714d5900ed1305b2f81af86fc44855e54c55ea55"}
Jul 28 18:06:19.803322 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:06:19.803259    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7" podStartSLOduration=1.803238147 podStartE2EDuration="1.803238147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:06:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:06:19.801655639 +0000 UTC m=+3335.047757846" watchObservedRunningTime="2026-07-28 18:06:19.803238147 +0000 UTC m=+3335.049340411"
Jul 28 18:06:50.789368 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:06:50.789327    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7"
Jul 28 18:08:29.801303 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:08:29.801255    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7"]
Jul 28 18:08:30.789959 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:08:30.789915    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7" podUID="c792f231-37c5-44ff-8c4d-ad89e99908d6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 18:08:40.789250 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:08:40.789206    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7" podUID="c792f231-37c5-44ff-8c4d-ad89e99908d6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 18:08:44.821650 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:08:44.821597    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7" podUID="c792f231-37c5-44ff-8c4d-ad89e99908d6" containerName="main" containerID="cri-o://a2c53c2dba120ed74dc71e7c273079f54413995593e569d19b6210c3a3e2a210" gracePeriod=45
Jul 28 18:08:45.082231 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:08:45.082151    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7"
Jul 28 18:08:45.178956 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:08:45.178914    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c792f231-37c5-44ff-8c4d-ad89e99908d6-tls-certs\") pod \"c792f231-37c5-44ff-8c4d-ad89e99908d6\" (UID: \"c792f231-37c5-44ff-8c4d-ad89e99908d6\") "
Jul 28 18:08:45.178956 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:08:45.178956    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99wvt\" (UniqueName: \"kubernetes.io/projected/c792f231-37c5-44ff-8c4d-ad89e99908d6-kube-api-access-99wvt\") pod \"c792f231-37c5-44ff-8c4d-ad89e99908d6\" (UID: \"c792f231-37c5-44ff-8c4d-ad89e99908d6\") "
Jul 28 18:08:45.181394 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:08:45.181354    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c792f231-37c5-44ff-8c4d-ad89e99908d6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c792f231-37c5-44ff-8c4d-ad89e99908d6" (UID: "c792f231-37c5-44ff-8c4d-ad89e99908d6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:08:45.181394 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:08:45.181366    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c792f231-37c5-44ff-8c4d-ad89e99908d6-kube-api-access-99wvt" (OuterVolumeSpecName: "kube-api-access-99wvt") pod "c792f231-37c5-44ff-8c4d-ad89e99908d6" (UID: "c792f231-37c5-44ff-8c4d-ad89e99908d6"). InnerVolumeSpecName "kube-api-access-99wvt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:08:45.280462 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:08:45.280412    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c792f231-37c5-44ff-8c4d-ad89e99908d6-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:08:45.280462 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:08:45.280448    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99wvt\" (UniqueName: \"kubernetes.io/projected/c792f231-37c5-44ff-8c4d-ad89e99908d6-kube-api-access-99wvt\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:08:45.498611 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:08:45.498489    2567 generic.go:356] "Generic (PLEG): container finished" podID="c792f231-37c5-44ff-8c4d-ad89e99908d6" containerID="a2c53c2dba120ed74dc71e7c273079f54413995593e569d19b6210c3a3e2a210" exitCode=0
Jul 28 18:08:45.498611 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:08:45.498550    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7"
Jul 28 18:08:45.498806 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:08:45.498549    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7" event={"ID":"c792f231-37c5-44ff-8c4d-ad89e99908d6","Type":"ContainerDied","Data":"a2c53c2dba120ed74dc71e7c273079f54413995593e569d19b6210c3a3e2a210"}
Jul 28 18:08:45.498806 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:08:45.498668    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7" event={"ID":"c792f231-37c5-44ff-8c4d-ad89e99908d6","Type":"ContainerDied","Data":"8d5e2eb64375c11b78457228714d5900ed1305b2f81af86fc44855e54c55ea55"}
Jul 28 18:08:45.498806 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:08:45.498692    2567 scope.go:117] "RemoveContainer" containerID="a2c53c2dba120ed74dc71e7c273079f54413995593e569d19b6210c3a3e2a210"
Jul 28 18:08:45.507865 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:08:45.507844    2567 scope.go:117] "RemoveContainer" containerID="a2c53c2dba120ed74dc71e7c273079f54413995593e569d19b6210c3a3e2a210"
Jul 28 18:08:45.508214 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:08:45.508193    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2c53c2dba120ed74dc71e7c273079f54413995593e569d19b6210c3a3e2a210\": container with ID starting with a2c53c2dba120ed74dc71e7c273079f54413995593e569d19b6210c3a3e2a210 not found: ID does not exist" containerID="a2c53c2dba120ed74dc71e7c273079f54413995593e569d19b6210c3a3e2a210"
Jul 28 18:08:45.508264 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:08:45.508224    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2c53c2dba120ed74dc71e7c273079f54413995593e569d19b6210c3a3e2a210"} err="failed to get container status \"a2c53c2dba120ed74dc71e7c273079f54413995593e569d19b6210c3a3e2a210\": rpc error: code = NotFound desc = could not find container \"a2c53c2dba120ed74dc71e7c273079f54413995593e569d19b6210c3a3e2a210\": container with ID starting with a2c53c2dba120ed74dc71e7c273079f54413995593e569d19b6210c3a3e2a210 not found: ID does not exist"
Jul 28 18:08:45.517332 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:08:45.517291    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7"]
Jul 28 18:08:45.520957 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:08:45.520919    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-26mt7"]
Jul 28 18:08:47.287851 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:08:47.287815    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c792f231-37c5-44ff-8c4d-ad89e99908d6" path="/var/lib/kubelet/pods/c792f231-37c5-44ff-8c4d-ad89e99908d6/volumes"
Jul 28 18:10:46.264484 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:10:46.264450    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:10:46.274441 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:10:46.274410    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:15:46.295032 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:15:46.295004    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:15:46.304437 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:15:46.304408    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:18:27.264914 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:18:27.264876    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25"]
Jul 28 18:18:36.005196 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:18:36.005156    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25" podUID="5e9b20b6-90cb-4c18-98e8-b161c8f299b7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 18:18:42.286684 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:18:42.286642    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25" podUID="5e9b20b6-90cb-4c18-98e8-b161c8f299b7" containerName="main" containerID="cri-o://92d3916516e0d9ec1780b9f8ecef284d9f90255a0857cbcc32a225ea11ce6af5" gracePeriod=45
Jul 28 18:18:42.434815 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:18:42.434775    2567 generic.go:356] "Generic (PLEG): container finished" podID="5e9b20b6-90cb-4c18-98e8-b161c8f299b7" containerID="92d3916516e0d9ec1780b9f8ecef284d9f90255a0857cbcc32a225ea11ce6af5" exitCode=0
Jul 28 18:18:42.434992 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:18:42.434845    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25" event={"ID":"5e9b20b6-90cb-4c18-98e8-b161c8f299b7","Type":"ContainerDied","Data":"92d3916516e0d9ec1780b9f8ecef284d9f90255a0857cbcc32a225ea11ce6af5"}
Jul 28 18:18:42.531310 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:18:42.531275    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25"
Jul 28 18:18:42.624614 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:18:42.624468    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e9b20b6-90cb-4c18-98e8-b161c8f299b7-tls-certs\") pod \"5e9b20b6-90cb-4c18-98e8-b161c8f299b7\" (UID: \"5e9b20b6-90cb-4c18-98e8-b161c8f299b7\") "
Jul 28 18:18:42.624614 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:18:42.624599    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8jxs\" (UniqueName: \"kubernetes.io/projected/5e9b20b6-90cb-4c18-98e8-b161c8f299b7-kube-api-access-z8jxs\") pod \"5e9b20b6-90cb-4c18-98e8-b161c8f299b7\" (UID: \"5e9b20b6-90cb-4c18-98e8-b161c8f299b7\") "
Jul 28 18:18:42.626905 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:18:42.626866    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e9b20b6-90cb-4c18-98e8-b161c8f299b7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5e9b20b6-90cb-4c18-98e8-b161c8f299b7" (UID: "5e9b20b6-90cb-4c18-98e8-b161c8f299b7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:18:42.627041 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:18:42.626921    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e9b20b6-90cb-4c18-98e8-b161c8f299b7-kube-api-access-z8jxs" (OuterVolumeSpecName: "kube-api-access-z8jxs") pod "5e9b20b6-90cb-4c18-98e8-b161c8f299b7" (UID: "5e9b20b6-90cb-4c18-98e8-b161c8f299b7"). InnerVolumeSpecName "kube-api-access-z8jxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:18:42.725183 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:18:42.725120    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8jxs\" (UniqueName: \"kubernetes.io/projected/5e9b20b6-90cb-4c18-98e8-b161c8f299b7-kube-api-access-z8jxs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:18:42.725183 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:18:42.725173    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e9b20b6-90cb-4c18-98e8-b161c8f299b7-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:18:43.441539 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:18:43.441498    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25"
Jul 28 18:18:43.442035 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:18:43.441491    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25" event={"ID":"5e9b20b6-90cb-4c18-98e8-b161c8f299b7","Type":"ContainerDied","Data":"bb022e1c7a5ce59e9900a137c3e721a77bd2e59631eedaa681f7f8ea300a3ea1"}
Jul 28 18:18:43.442035 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:18:43.441647    2567 scope.go:117] "RemoveContainer" containerID="92d3916516e0d9ec1780b9f8ecef284d9f90255a0857cbcc32a225ea11ce6af5"
Jul 28 18:18:43.460494 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:18:43.460454    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25"]
Jul 28 18:18:43.466762 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:18:43.466710    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-bjf25"]
Jul 28 18:18:45.288576 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:18:45.288538    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e9b20b6-90cb-4c18-98e8-b161c8f299b7" path="/var/lib/kubelet/pods/5e9b20b6-90cb-4c18-98e8-b161c8f299b7/volumes"
Jul 28 18:19:04.737136 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.737099    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"]
Jul 28 18:19:04.737670 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.737556    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e9b20b6-90cb-4c18-98e8-b161c8f299b7" containerName="main"
Jul 28 18:19:04.737670 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.737574    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e9b20b6-90cb-4c18-98e8-b161c8f299b7" containerName="main"
Jul 28 18:19:04.737670 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.737604    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c792f231-37c5-44ff-8c4d-ad89e99908d6" containerName="main"
Jul 28 18:19:04.737670 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.737612    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c792f231-37c5-44ff-8c4d-ad89e99908d6" containerName="main"
Jul 28 18:19:04.737895 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.737680    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c792f231-37c5-44ff-8c4d-ad89e99908d6" containerName="main"
Jul 28 18:19:04.737895 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.737691    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e9b20b6-90cb-4c18-98e8-b161c8f299b7" containerName="main"
Jul 28 18:19:04.740361 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.740339    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:04.745419 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.745389    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2\"/\"autoscale-hpa-lws-kserve-mn-dockercfg-rg2nd\""
Jul 28 18:19:04.745419 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.745407    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2\"/\"openshift-service-ca.crt\""
Jul 28 18:19:04.745682 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.745555    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2\"/\"autoscale-hpa-lws-kserve-self-signed-certs\""
Jul 28 18:19:04.745682 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.745577    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2\"/\"default-dockercfg-nn5fq\""
Jul 28 18:19:04.745682 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.745612    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2\"/\"kube-root-ca.crt\""
Jul 28 18:19:04.752704 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.752664    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"]
Jul 28 18:19:04.805453 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.805407    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/699d2b4a-79e3-495e-a50f-07278ae0c7ba-tls-certs\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:04.805453 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.805458    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-dshm\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:04.805697 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.805484    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-tmp-dir\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:04.805697 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.805543    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-home\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:04.805697 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.805593    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-model-cache\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:04.805697 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.805648    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kkpf\" (UniqueName: \"kubernetes.io/projected/699d2b4a-79e3-495e-a50f-07278ae0c7ba-kube-api-access-6kkpf\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:04.906008 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.905974    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-dshm\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:04.906267 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.906013    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-tmp-dir\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:04.906267 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.906036    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-home\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:04.906267 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.906059    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-model-cache\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:04.906267 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.906107    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kkpf\" (UniqueName: \"kubernetes.io/projected/699d2b4a-79e3-495e-a50f-07278ae0c7ba-kube-api-access-6kkpf\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:04.906552 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.906436    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-model-cache\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:04.906552 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.906467    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/699d2b4a-79e3-495e-a50f-07278ae0c7ba-tls-certs\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:04.907086 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.906696    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-home\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:04.907086 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.906826    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-tmp-dir\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:04.909290 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.909264    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-dshm\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:04.909690 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.909668    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/699d2b4a-79e3-495e-a50f-07278ae0c7ba-tls-certs\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:04.925737 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:04.925700    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kkpf\" (UniqueName: \"kubernetes.io/projected/699d2b4a-79e3-495e-a50f-07278ae0c7ba-kube-api-access-6kkpf\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:05.051186 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:05.051134    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:05.202929 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:05.202641    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"]
Jul 28 18:19:05.553590 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:05.553539    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" event={"ID":"699d2b4a-79e3-495e-a50f-07278ae0c7ba","Type":"ContainerStarted","Data":"0797dc72991934cc79f8e998ffbb43e1bec0250cbd8871a36326ebb34a0486cf"}
Jul 28 18:19:05.553812 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:05.553612    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" event={"ID":"699d2b4a-79e3-495e-a50f-07278ae0c7ba","Type":"ContainerStarted","Data":"7d70278c1a55f7e5e48192e58aa173d49e6d2cfb6e63bf2e197382021df581b3"}
Jul 28 18:19:05.573537 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:05.573430    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" podStartSLOduration=1.5734117589999999 podStartE2EDuration="1.573411759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:19:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:19:05.572187092 +0000 UTC m=+4100.818289313" watchObservedRunningTime="2026-07-28 18:19:05.573411759 +0000 UTC m=+4100.819513966"
Jul 28 18:19:06.051374 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:06.051341    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:15.051371 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:15.051333    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:15.064285 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:15.064255    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:19:15.611457 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:19:15.611422    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:20:46.324784 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:46.324756    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:20:46.334522 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:46.334478    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:20:52.638786 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.638744    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"]
Jul 28 18:20:52.642141 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.642119    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:20:52.645396 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.645363    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"kube-root-ca.crt\""
Jul 28 18:20:52.645592 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.645363    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"openshift-service-ca.crt\""
Jul 28 18:20:52.646924 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.646899    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"default-dockercfg-mg22h\""
Jul 28 18:20:52.647010 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.646962    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"rolling-upgrade-test-kserve-self-signed-certs\""
Jul 28 18:20:52.652785 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.652759    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"]
Jul 28 18:20:52.734727 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.734690    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-dshm\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-lngzs\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:20:52.734913 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.734732    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-model-cache\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-lngzs\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:20:52.734913 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.734800    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-home\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-lngzs\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:20:52.734913 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.734852    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-tls-certs\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-lngzs\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:20:52.734913 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.734897    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlhhp\" (UniqueName: \"kubernetes.io/projected/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-kube-api-access-jlhhp\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-lngzs\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:20:52.735058 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.734933    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-tmp-dir\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-lngzs\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:20:52.835942 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.835907    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-tmp-dir\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-lngzs\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:20:52.836129 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.835951    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-dshm\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-lngzs\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:20:52.836129 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.836074    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-model-cache\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-lngzs\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:20:52.836206 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.836158    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-home\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-lngzs\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:20:52.836206 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.836193    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-tls-certs\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-lngzs\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:20:52.836283 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.836237    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlhhp\" (UniqueName: \"kubernetes.io/projected/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-kube-api-access-jlhhp\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-lngzs\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:20:52.836416 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.836393    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-tmp-dir\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-lngzs\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:20:52.836553 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.836528    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-model-cache\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-lngzs\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:20:52.836630 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.836556    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-home\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-lngzs\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:20:52.838525 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.838460    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-dshm\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-lngzs\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:20:52.838731 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.838715    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-tls-certs\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-lngzs\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:20:52.845761 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.845727    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlhhp\" (UniqueName: \"kubernetes.io/projected/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-kube-api-access-jlhhp\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-lngzs\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:20:52.952712 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:52.952595    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:20:53.101466 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:53.101435    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"]
Jul 28 18:20:53.104199 ip-10-0-133-215 kubenswrapper[2567]: W0728 18:20:53.104166    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57f14e44_04e4_465a_8ca1_8bb4fa2e0d38.slice/crio-d70551e8075717d5b09e6d02e65a2a487027783b149768fc08feb56ce3d61165 WatchSource:0}: Error finding container d70551e8075717d5b09e6d02e65a2a487027783b149768fc08feb56ce3d61165: Status 404 returned error can't find the container with id d70551e8075717d5b09e6d02e65a2a487027783b149768fc08feb56ce3d61165
Jul 28 18:20:54.089679 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:54.089637    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs" event={"ID":"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38","Type":"ContainerStarted","Data":"56d76b61c24de819e43ebb942b4dd5215e688f87c98291634f3819660593fccf"}
Jul 28 18:20:54.090154 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:54.089702    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs" event={"ID":"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38","Type":"ContainerStarted","Data":"d70551e8075717d5b09e6d02e65a2a487027783b149768fc08feb56ce3d61165"}
Jul 28 18:20:54.108179 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:54.108124    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs" podStartSLOduration=2.10810725 podStartE2EDuration="2.10810725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:20:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:20:54.107582496 +0000 UTC m=+4209.353684703" watchObservedRunningTime="2026-07-28 18:20:54.10810725 +0000 UTC m=+4209.354209482"
Jul 28 18:20:54.953094 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:20:54.953016    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:21:02.952809 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:02.952759    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:21:02.965677 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:02.965643    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:21:03.150740 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:03.150698    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:21:17.279152 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.279114    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"]
Jul 28 18:21:17.290630 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:21:17.290590    2567 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 28 18:21:17.290630 ip-10-0-133-215 kubenswrapper[2567]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 18:21:17.290630 ip-10-0-133-215 kubenswrapper[2567]:         , stdout: , stderr: , exit code -1
Jul 28 18:21:17.290630 ip-10-0-133-215 kubenswrapper[2567]:  > containerID="56d76b61c24de819e43ebb942b4dd5215e688f87c98291634f3819660593fccf" cmd=["/bin/sleep","15"]
Jul 28 18:21:17.290911 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.290646    2567 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs" message="" err=<
Jul 28 18:21:17.290911 ip-10-0-133-215 kubenswrapper[2567]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 18:21:17.290911 ip-10-0-133-215 kubenswrapper[2567]:         , stdout: , stderr: , exit code -1
Jul 28 18:21:17.290911 ip-10-0-133-215 kubenswrapper[2567]:  >
Jul 28 18:21:17.290911 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:21:17.290661    2567 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 28 18:21:17.290911 ip-10-0-133-215 kubenswrapper[2567]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 18:21:17.290911 ip-10-0-133-215 kubenswrapper[2567]:         , stdout: , stderr: , exit code -1
Jul 28 18:21:17.290911 ip-10-0-133-215 kubenswrapper[2567]:  > pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs" podUID="57f14e44-04e4-465a-8ca1-8bb4fa2e0d38" containerName="main" containerID="cri-o://56d76b61c24de819e43ebb942b4dd5215e688f87c98291634f3819660593fccf"
Jul 28 18:21:17.290911 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.290704    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs" podUID="57f14e44-04e4-465a-8ca1-8bb4fa2e0d38" containerName="main" containerID="cri-o://56d76b61c24de819e43ebb942b4dd5215e688f87c98291634f3819660593fccf" gracePeriod=60
Jul 28 18:21:17.657133 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.657099    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:21:17.752807 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.752768    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-home\") pod \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") "
Jul 28 18:21:17.752807 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.752811    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-tls-certs\") pod \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") "
Jul 28 18:21:17.753063 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.752846    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-tmp-dir\") pod \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") "
Jul 28 18:21:17.753063 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.752877    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlhhp\" (UniqueName: \"kubernetes.io/projected/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-kube-api-access-jlhhp\") pod \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") "
Jul 28 18:21:17.753063 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.752893    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-dshm\") pod \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") "
Jul 28 18:21:17.753063 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.752909    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-model-cache\") pod \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\" (UID: \"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38\") "
Jul 28 18:21:17.753281 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.753081    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-home" (OuterVolumeSpecName: "home") pod "57f14e44-04e4-465a-8ca1-8bb4fa2e0d38" (UID: "57f14e44-04e4-465a-8ca1-8bb4fa2e0d38"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:21:17.753281 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.753154    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-home\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:21:17.753281 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.753216    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "57f14e44-04e4-465a-8ca1-8bb4fa2e0d38" (UID: "57f14e44-04e4-465a-8ca1-8bb4fa2e0d38"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:21:17.753414 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.753281    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-model-cache" (OuterVolumeSpecName: "model-cache") pod "57f14e44-04e4-465a-8ca1-8bb4fa2e0d38" (UID: "57f14e44-04e4-465a-8ca1-8bb4fa2e0d38"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:21:17.755297 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.755256    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "57f14e44-04e4-465a-8ca1-8bb4fa2e0d38" (UID: "57f14e44-04e4-465a-8ca1-8bb4fa2e0d38"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:21:17.755297 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.755261    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-dshm" (OuterVolumeSpecName: "dshm") pod "57f14e44-04e4-465a-8ca1-8bb4fa2e0d38" (UID: "57f14e44-04e4-465a-8ca1-8bb4fa2e0d38"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:21:17.755482 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.755258    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-kube-api-access-jlhhp" (OuterVolumeSpecName: "kube-api-access-jlhhp") pod "57f14e44-04e4-465a-8ca1-8bb4fa2e0d38" (UID: "57f14e44-04e4-465a-8ca1-8bb4fa2e0d38"). InnerVolumeSpecName "kube-api-access-jlhhp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:21:17.853981 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.853879    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:21:17.853981 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.853940    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-tmp-dir\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:21:17.853981 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.853952    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlhhp\" (UniqueName: \"kubernetes.io/projected/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-kube-api-access-jlhhp\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:21:17.853981 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.853967    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-dshm\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:21:17.853981 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:17.853977    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38-model-cache\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:21:18.227848 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:18.227738    2567 generic.go:356] "Generic (PLEG): container finished" podID="57f14e44-04e4-465a-8ca1-8bb4fa2e0d38" containerID="56d76b61c24de819e43ebb942b4dd5215e688f87c98291634f3819660593fccf" exitCode=0
Jul 28 18:21:18.227848 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:18.227782    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"
Jul 28 18:21:18.227848 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:18.227824    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs" event={"ID":"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38","Type":"ContainerDied","Data":"56d76b61c24de819e43ebb942b4dd5215e688f87c98291634f3819660593fccf"}
Jul 28 18:21:18.228113 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:18.227876    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs" event={"ID":"57f14e44-04e4-465a-8ca1-8bb4fa2e0d38","Type":"ContainerDied","Data":"d70551e8075717d5b09e6d02e65a2a487027783b149768fc08feb56ce3d61165"}
Jul 28 18:21:18.228113 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:18.227900    2567 scope.go:117] "RemoveContainer" containerID="56d76b61c24de819e43ebb942b4dd5215e688f87c98291634f3819660593fccf"
Jul 28 18:21:18.240417 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:18.240320    2567 scope.go:117] "RemoveContainer" containerID="56d76b61c24de819e43ebb942b4dd5215e688f87c98291634f3819660593fccf"
Jul 28 18:21:18.240924 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:21:18.240902    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56d76b61c24de819e43ebb942b4dd5215e688f87c98291634f3819660593fccf\": container with ID starting with 56d76b61c24de819e43ebb942b4dd5215e688f87c98291634f3819660593fccf not found: ID does not exist" containerID="56d76b61c24de819e43ebb942b4dd5215e688f87c98291634f3819660593fccf"
Jul 28 18:21:18.241034 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:18.240931    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56d76b61c24de819e43ebb942b4dd5215e688f87c98291634f3819660593fccf"} err="failed to get container status \"56d76b61c24de819e43ebb942b4dd5215e688f87c98291634f3819660593fccf\": rpc error: code = NotFound desc = could not find container \"56d76b61c24de819e43ebb942b4dd5215e688f87c98291634f3819660593fccf\": container with ID starting with 56d76b61c24de819e43ebb942b4dd5215e688f87c98291634f3819660593fccf not found: ID does not exist"
Jul 28 18:21:18.256158 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:18.256122    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"]
Jul 28 18:21:18.262286 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:18.262246    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-lngzs"]
Jul 28 18:21:19.289374 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:21:19.289322    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57f14e44-04e4-465a-8ca1-8bb4fa2e0d38" path="/var/lib/kubelet/pods/57f14e44-04e4-465a-8ca1-8bb4fa2e0d38/volumes"
Jul 28 18:22:12.729540 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:22:12.729486    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/llmisvc-controller-manager-545d98d4c7-4jhnx"]
Jul 28 18:22:12.730030 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:22:12.729715    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/llmisvc-controller-manager-545d98d4c7-4jhnx" podUID="286a752c-1b32-4e69-9d52-42daa1ba014d" containerName="manager" containerID="cri-o://a3a39aea37518d76c6b5e01a3fcb6f4d9a141fe075a55dec959f758197bcbabb" gracePeriod=30
Jul 28 18:22:12.989212 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:22:12.989139    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-545d98d4c7-4jhnx"
Jul 28 18:22:13.113677 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:22:13.113634    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nk47\" (UniqueName: \"kubernetes.io/projected/286a752c-1b32-4e69-9d52-42daa1ba014d-kube-api-access-9nk47\") pod \"286a752c-1b32-4e69-9d52-42daa1ba014d\" (UID: \"286a752c-1b32-4e69-9d52-42daa1ba014d\") "
Jul 28 18:22:13.113677 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:22:13.113675    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/286a752c-1b32-4e69-9d52-42daa1ba014d-cert\") pod \"286a752c-1b32-4e69-9d52-42daa1ba014d\" (UID: \"286a752c-1b32-4e69-9d52-42daa1ba014d\") "
Jul 28 18:22:13.116016 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:22:13.115982    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/286a752c-1b32-4e69-9d52-42daa1ba014d-kube-api-access-9nk47" (OuterVolumeSpecName: "kube-api-access-9nk47") pod "286a752c-1b32-4e69-9d52-42daa1ba014d" (UID: "286a752c-1b32-4e69-9d52-42daa1ba014d"). InnerVolumeSpecName "kube-api-access-9nk47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:22:13.116155 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:22:13.116026    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/286a752c-1b32-4e69-9d52-42daa1ba014d-cert" (OuterVolumeSpecName: "cert") pod "286a752c-1b32-4e69-9d52-42daa1ba014d" (UID: "286a752c-1b32-4e69-9d52-42daa1ba014d"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:22:13.214374 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:22:13.214320    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nk47\" (UniqueName: \"kubernetes.io/projected/286a752c-1b32-4e69-9d52-42daa1ba014d-kube-api-access-9nk47\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:22:13.214374 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:22:13.214366    2567 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/286a752c-1b32-4e69-9d52-42daa1ba014d-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:22:13.486925 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:22:13.486885    2567 generic.go:356] "Generic (PLEG): container finished" podID="286a752c-1b32-4e69-9d52-42daa1ba014d" containerID="a3a39aea37518d76c6b5e01a3fcb6f4d9a141fe075a55dec959f758197bcbabb" exitCode=0
Jul 28 18:22:13.487134 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:22:13.486932    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-545d98d4c7-4jhnx"
Jul 28 18:22:13.487134 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:22:13.486967    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-545d98d4c7-4jhnx" event={"ID":"286a752c-1b32-4e69-9d52-42daa1ba014d","Type":"ContainerDied","Data":"a3a39aea37518d76c6b5e01a3fcb6f4d9a141fe075a55dec959f758197bcbabb"}
Jul 28 18:22:13.487134 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:22:13.487031    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-545d98d4c7-4jhnx" event={"ID":"286a752c-1b32-4e69-9d52-42daa1ba014d","Type":"ContainerDied","Data":"a337982ede6f79c83cc76b3c0e6ad947ffb8af3c248347db2003168f4c3a1426"}
Jul 28 18:22:13.487134 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:22:13.487061    2567 scope.go:117] "RemoveContainer" containerID="a3a39aea37518d76c6b5e01a3fcb6f4d9a141fe075a55dec959f758197bcbabb"
Jul 28 18:22:13.498928 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:22:13.498908    2567 scope.go:117] "RemoveContainer" containerID="a3a39aea37518d76c6b5e01a3fcb6f4d9a141fe075a55dec959f758197bcbabb"
Jul 28 18:22:13.499205 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:22:13.499184    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3a39aea37518d76c6b5e01a3fcb6f4d9a141fe075a55dec959f758197bcbabb\": container with ID starting with a3a39aea37518d76c6b5e01a3fcb6f4d9a141fe075a55dec959f758197bcbabb not found: ID does not exist" containerID="a3a39aea37518d76c6b5e01a3fcb6f4d9a141fe075a55dec959f758197bcbabb"
Jul 28 18:22:13.499251 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:22:13.499216    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3a39aea37518d76c6b5e01a3fcb6f4d9a141fe075a55dec959f758197bcbabb"} err="failed to get container status \"a3a39aea37518d76c6b5e01a3fcb6f4d9a141fe075a55dec959f758197bcbabb\": rpc error: code = NotFound desc = could not find container \"a3a39aea37518d76c6b5e01a3fcb6f4d9a141fe075a55dec959f758197bcbabb\": container with ID starting with a3a39aea37518d76c6b5e01a3fcb6f4d9a141fe075a55dec959f758197bcbabb not found: ID does not exist"
Jul 28 18:22:13.506837 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:22:13.506807    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/llmisvc-controller-manager-545d98d4c7-4jhnx"]
Jul 28 18:22:13.510167 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:22:13.510136    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/llmisvc-controller-manager-545d98d4c7-4jhnx"]
Jul 28 18:22:15.287317 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:22:15.287268    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="286a752c-1b32-4e69-9d52-42daa1ba014d" path="/var/lib/kubelet/pods/286a752c-1b32-4e69-9d52-42daa1ba014d/volumes"
Jul 28 18:23:34.906779 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:34.906736    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"]
Jul 28 18:23:34.907219 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:34.907049    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="286a752c-1b32-4e69-9d52-42daa1ba014d" containerName="manager"
Jul 28 18:23:34.907219 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:34.907060    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="286a752c-1b32-4e69-9d52-42daa1ba014d" containerName="manager"
Jul 28 18:23:34.907219 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:34.907072    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57f14e44-04e4-465a-8ca1-8bb4fa2e0d38" containerName="main"
Jul 28 18:23:34.907219 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:34.907078    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="57f14e44-04e4-465a-8ca1-8bb4fa2e0d38" containerName="main"
Jul 28 18:23:34.907219 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:34.907144    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="286a752c-1b32-4e69-9d52-42daa1ba014d" containerName="manager"
Jul 28 18:23:34.907219 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:34.907151    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="57f14e44-04e4-465a-8ca1-8bb4fa2e0d38" containerName="main"
Jul 28 18:23:34.909982 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:34.909963    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:34.913228 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:34.913203    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-three-member-group-c33d828b\"/\"openshift-service-ca.crt\""
Jul 28 18:23:34.914849 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:34.914824    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-three-member-group-c33d828b\"/\"tri-v1-kserve-self-signed-certs\""
Jul 28 18:23:34.914982 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:34.914831    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-three-member-group-c33d828b\"/\"kube-root-ca.crt\""
Jul 28 18:23:34.914982 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:34.914972    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-three-member-group-c33d828b\"/\"default-dockercfg-bwqbj\""
Jul 28 18:23:34.919735 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:34.919707    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"]
Jul 28 18:23:35.026662 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.026615    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-tmp-dir\") pod \"tri-v1-kserve-79f8f98bc5-crkvj\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:35.026887 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.026670    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-model-cache\") pod \"tri-v1-kserve-79f8f98bc5-crkvj\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:35.026887 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.026740    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-dshm\") pod \"tri-v1-kserve-79f8f98bc5-crkvj\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:35.026887 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.026776    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-home\") pod \"tri-v1-kserve-79f8f98bc5-crkvj\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:35.026887 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.026807    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shbh2\" (UniqueName: \"kubernetes.io/projected/7bdd820b-4baf-4eb9-b146-59d90a782624-kube-api-access-shbh2\") pod \"tri-v1-kserve-79f8f98bc5-crkvj\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:35.027065 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.026879    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bdd820b-4baf-4eb9-b146-59d90a782624-tls-certs\") pod \"tri-v1-kserve-79f8f98bc5-crkvj\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:35.127811 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.127771    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shbh2\" (UniqueName: \"kubernetes.io/projected/7bdd820b-4baf-4eb9-b146-59d90a782624-kube-api-access-shbh2\") pod \"tri-v1-kserve-79f8f98bc5-crkvj\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:35.127811 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.127818    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bdd820b-4baf-4eb9-b146-59d90a782624-tls-certs\") pod \"tri-v1-kserve-79f8f98bc5-crkvj\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:35.128105 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.127858    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-tmp-dir\") pod \"tri-v1-kserve-79f8f98bc5-crkvj\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:35.128105 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.127879    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-model-cache\") pod \"tri-v1-kserve-79f8f98bc5-crkvj\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:35.128105 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.127909    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-dshm\") pod \"tri-v1-kserve-79f8f98bc5-crkvj\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:35.128105 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.127928    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-home\") pod \"tri-v1-kserve-79f8f98bc5-crkvj\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:35.128342 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.128297    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-tmp-dir\") pod \"tri-v1-kserve-79f8f98bc5-crkvj\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:35.128342 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.128319    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-home\") pod \"tri-v1-kserve-79f8f98bc5-crkvj\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:35.128441 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.128372    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-model-cache\") pod \"tri-v1-kserve-79f8f98bc5-crkvj\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:35.130367 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.130333    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-dshm\") pod \"tri-v1-kserve-79f8f98bc5-crkvj\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:35.130568 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.130550    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bdd820b-4baf-4eb9-b146-59d90a782624-tls-certs\") pod \"tri-v1-kserve-79f8f98bc5-crkvj\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:35.137178 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.137152    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shbh2\" (UniqueName: \"kubernetes.io/projected/7bdd820b-4baf-4eb9-b146-59d90a782624-kube-api-access-shbh2\") pod \"tri-v1-kserve-79f8f98bc5-crkvj\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:35.221315 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.221221    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:35.384210 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.384178    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"]
Jul 28 18:23:35.387176 ip-10-0-133-215 kubenswrapper[2567]: W0728 18:23:35.387140    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bdd820b_4baf_4eb9_b146_59d90a782624.slice/crio-63eb8e57cfb158a6e3aa84a39cedd76976e9c47ff39534a125c461d44be53507 WatchSource:0}: Error finding container 63eb8e57cfb158a6e3aa84a39cedd76976e9c47ff39534a125c461d44be53507: Status 404 returned error can't find the container with id 63eb8e57cfb158a6e3aa84a39cedd76976e9c47ff39534a125c461d44be53507
Jul 28 18:23:35.389870 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.389843    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 18:23:35.890275 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:35.890224    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj" event={"ID":"7bdd820b-4baf-4eb9-b146-59d90a782624","Type":"ContainerStarted","Data":"63eb8e57cfb158a6e3aa84a39cedd76976e9c47ff39534a125c461d44be53507"}
Jul 28 18:23:36.897710 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:36.897664    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj" event={"ID":"7bdd820b-4baf-4eb9-b146-59d90a782624","Type":"ContainerStarted","Data":"bfdbb632bfda7f94bd39277655563c968a37179016e392faee739c04ec516782"}
Jul 28 18:23:36.921706 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:36.921646    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj" podStartSLOduration=1.897344998 podStartE2EDuration="2.921629526s" totalImagesPullingTime="1.024284528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:23:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:23:36.92114141 +0000 UTC m=+4372.167243621" watchObservedRunningTime="2026-07-28 18:23:36.921629526 +0000 UTC m=+4372.167731732"
Jul 28 18:23:37.221878 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:37.221777    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:45.222436 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:45.222385    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:45.241349 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:45.241307    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:23:45.955272 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:23:45.955213    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:25:46.358528 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:25:46.358418    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:25:46.372013 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:25:46.371976    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:25:51.754627 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:25:51.754579    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"]
Jul 28 18:26:06.773979 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:06.773931    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj" podUID="7bdd820b-4baf-4eb9-b146-59d90a782624" containerName="main" containerID="cri-o://bfdbb632bfda7f94bd39277655563c968a37179016e392faee739c04ec516782" gracePeriod=45
Jul 28 18:26:06.944659 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:06.944611    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj" podUID="7bdd820b-4baf-4eb9-b146-59d90a782624" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/health\": dial tcp 10.134.0.50:8000: connect: connection refused"
Jul 28 18:26:07.031048 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.030953    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:26:07.108558 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.108494    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shbh2\" (UniqueName: \"kubernetes.io/projected/7bdd820b-4baf-4eb9-b146-59d90a782624-kube-api-access-shbh2\") pod \"7bdd820b-4baf-4eb9-b146-59d90a782624\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") "
Jul 28 18:26:07.108774 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.108594    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bdd820b-4baf-4eb9-b146-59d90a782624-tls-certs\") pod \"7bdd820b-4baf-4eb9-b146-59d90a782624\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") "
Jul 28 18:26:07.108774 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.108695    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-dshm\") pod \"7bdd820b-4baf-4eb9-b146-59d90a782624\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") "
Jul 28 18:26:07.108774 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.108734    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-model-cache\") pod \"7bdd820b-4baf-4eb9-b146-59d90a782624\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") "
Jul 28 18:26:07.108945 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.108799    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-tmp-dir\") pod \"7bdd820b-4baf-4eb9-b146-59d90a782624\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") "
Jul 28 18:26:07.108945 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.108840    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-home\") pod \"7bdd820b-4baf-4eb9-b146-59d90a782624\" (UID: \"7bdd820b-4baf-4eb9-b146-59d90a782624\") "
Jul 28 18:26:07.109165 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.109135    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-model-cache" (OuterVolumeSpecName: "model-cache") pod "7bdd820b-4baf-4eb9-b146-59d90a782624" (UID: "7bdd820b-4baf-4eb9-b146-59d90a782624"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:26:07.109398 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.109358    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-home" (OuterVolumeSpecName: "home") pod "7bdd820b-4baf-4eb9-b146-59d90a782624" (UID: "7bdd820b-4baf-4eb9-b146-59d90a782624"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:26:07.109497 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.109459    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7bdd820b-4baf-4eb9-b146-59d90a782624" (UID: "7bdd820b-4baf-4eb9-b146-59d90a782624"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:26:07.111544 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.111487    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bdd820b-4baf-4eb9-b146-59d90a782624-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7bdd820b-4baf-4eb9-b146-59d90a782624" (UID: "7bdd820b-4baf-4eb9-b146-59d90a782624"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:26:07.111544 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.111486    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-dshm" (OuterVolumeSpecName: "dshm") pod "7bdd820b-4baf-4eb9-b146-59d90a782624" (UID: "7bdd820b-4baf-4eb9-b146-59d90a782624"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:26:07.111793 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.111706    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bdd820b-4baf-4eb9-b146-59d90a782624-kube-api-access-shbh2" (OuterVolumeSpecName: "kube-api-access-shbh2") pod "7bdd820b-4baf-4eb9-b146-59d90a782624" (UID: "7bdd820b-4baf-4eb9-b146-59d90a782624"). InnerVolumeSpecName "kube-api-access-shbh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:26:07.210323 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.210261    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-dshm\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:26:07.210323 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.210314    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-model-cache\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:26:07.210323 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.210326    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-tmp-dir\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:26:07.210323 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.210333    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bdd820b-4baf-4eb9-b146-59d90a782624-home\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:26:07.210323 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.210343    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shbh2\" (UniqueName: \"kubernetes.io/projected/7bdd820b-4baf-4eb9-b146-59d90a782624-kube-api-access-shbh2\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:26:07.210687 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.210353    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bdd820b-4baf-4eb9-b146-59d90a782624-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:26:07.681305 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.681171    2567 generic.go:356] "Generic (PLEG): container finished" podID="7bdd820b-4baf-4eb9-b146-59d90a782624" containerID="bfdbb632bfda7f94bd39277655563c968a37179016e392faee739c04ec516782" exitCode=0
Jul 28 18:26:07.681305 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.681231    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj" event={"ID":"7bdd820b-4baf-4eb9-b146-59d90a782624","Type":"ContainerDied","Data":"bfdbb632bfda7f94bd39277655563c968a37179016e392faee739c04ec516782"}
Jul 28 18:26:07.681305 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.681256    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"
Jul 28 18:26:07.681305 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.681300    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj" event={"ID":"7bdd820b-4baf-4eb9-b146-59d90a782624","Type":"ContainerDied","Data":"63eb8e57cfb158a6e3aa84a39cedd76976e9c47ff39534a125c461d44be53507"}
Jul 28 18:26:07.681716 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.681336    2567 scope.go:117] "RemoveContainer" containerID="bfdbb632bfda7f94bd39277655563c968a37179016e392faee739c04ec516782"
Jul 28 18:26:07.697836 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.697801    2567 scope.go:117] "RemoveContainer" containerID="bfdbb632bfda7f94bd39277655563c968a37179016e392faee739c04ec516782"
Jul 28 18:26:07.698577 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:26:07.698523    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfdbb632bfda7f94bd39277655563c968a37179016e392faee739c04ec516782\": container with ID starting with bfdbb632bfda7f94bd39277655563c968a37179016e392faee739c04ec516782 not found: ID does not exist" containerID="bfdbb632bfda7f94bd39277655563c968a37179016e392faee739c04ec516782"
Jul 28 18:26:07.698727 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.698576    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfdbb632bfda7f94bd39277655563c968a37179016e392faee739c04ec516782"} err="failed to get container status \"bfdbb632bfda7f94bd39277655563c968a37179016e392faee739c04ec516782\": rpc error: code = NotFound desc = could not find container \"bfdbb632bfda7f94bd39277655563c968a37179016e392faee739c04ec516782\": container with ID starting with bfdbb632bfda7f94bd39277655563c968a37179016e392faee739c04ec516782 not found: ID does not exist"
Jul 28 18:26:07.704592 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.704536    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"]
Jul 28 18:26:07.707823 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:07.707769    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-crkvj"]
Jul 28 18:26:09.287186 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:26:09.287151    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bdd820b-4baf-4eb9-b146-59d90a782624" path="/var/lib/kubelet/pods/7bdd820b-4baf-4eb9-b146-59d90a782624/volumes"
Jul 28 18:27:26.803464 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:26.803417    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"]
Jul 28 18:27:26.804007 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:26.803750    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bdd820b-4baf-4eb9-b146-59d90a782624" containerName="main"
Jul 28 18:27:26.804007 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:26.803760    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bdd820b-4baf-4eb9-b146-59d90a782624" containerName="main"
Jul 28 18:27:26.804007 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:26.803820    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bdd820b-4baf-4eb9-b146-59d90a782624" containerName="main"
Jul 28 18:27:26.806683 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:26.806662    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:26.811584 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:26.811549    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-delete-at-nonzero-weight-6b4017d2\"/\"default-dockercfg-7pct8\""
Jul 28 18:27:26.811584 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:26.811584    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-delete-at-nonzero-weight-6b4017d2\"/\"kube-root-ca.crt\""
Jul 28 18:27:26.811584 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:26.811593    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-delete-at-nonzero-weight-6b4017d2\"/\"delw-v1-kserve-self-signed-certs\""
Jul 28 18:27:26.811861 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:26.811609    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-delete-at-nonzero-weight-6b4017d2\"/\"openshift-service-ca.crt\""
Jul 28 18:27:26.817614 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:26.817584    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"]
Jul 28 18:27:26.990739 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:26.990690    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-home\") pod \"delw-v1-kserve-6cf8cbb847-tgp24\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:26.990739 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:26.990735    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-tmp-dir\") pod \"delw-v1-kserve-6cf8cbb847-tgp24\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:26.990985 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:26.990758    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d1df41e-3606-45eb-a66e-2e79977cfdbe-tls-certs\") pod \"delw-v1-kserve-6cf8cbb847-tgp24\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:26.990985 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:26.990818    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr2vv\" (UniqueName: \"kubernetes.io/projected/5d1df41e-3606-45eb-a66e-2e79977cfdbe-kube-api-access-gr2vv\") pod \"delw-v1-kserve-6cf8cbb847-tgp24\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:26.990985 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:26.990861    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-model-cache\") pod \"delw-v1-kserve-6cf8cbb847-tgp24\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:26.990985 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:26.990922    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-dshm\") pod \"delw-v1-kserve-6cf8cbb847-tgp24\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:27.091428 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:27.091325    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-home\") pod \"delw-v1-kserve-6cf8cbb847-tgp24\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:27.091428 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:27.091361    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-tmp-dir\") pod \"delw-v1-kserve-6cf8cbb847-tgp24\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:27.091428 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:27.091386    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d1df41e-3606-45eb-a66e-2e79977cfdbe-tls-certs\") pod \"delw-v1-kserve-6cf8cbb847-tgp24\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:27.091428 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:27.091412    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr2vv\" (UniqueName: \"kubernetes.io/projected/5d1df41e-3606-45eb-a66e-2e79977cfdbe-kube-api-access-gr2vv\") pod \"delw-v1-kserve-6cf8cbb847-tgp24\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:27.091813 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:27.091442    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-model-cache\") pod \"delw-v1-kserve-6cf8cbb847-tgp24\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:27.091813 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:27.091573    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-dshm\") pod \"delw-v1-kserve-6cf8cbb847-tgp24\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:27.091933 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:27.091900    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-model-cache\") pod \"delw-v1-kserve-6cf8cbb847-tgp24\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:27.091988 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:27.091941    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-tmp-dir\") pod \"delw-v1-kserve-6cf8cbb847-tgp24\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:27.092211 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:27.092190    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-home\") pod \"delw-v1-kserve-6cf8cbb847-tgp24\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:27.094173 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:27.094147    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-dshm\") pod \"delw-v1-kserve-6cf8cbb847-tgp24\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:27.094445 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:27.094421    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d1df41e-3606-45eb-a66e-2e79977cfdbe-tls-certs\") pod \"delw-v1-kserve-6cf8cbb847-tgp24\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:27.101275 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:27.101240    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr2vv\" (UniqueName: \"kubernetes.io/projected/5d1df41e-3606-45eb-a66e-2e79977cfdbe-kube-api-access-gr2vv\") pod \"delw-v1-kserve-6cf8cbb847-tgp24\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:27.117925 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:27.117888    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:27.268259 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:27.268226    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"]
Jul 28 18:27:27.270833 ip-10-0-133-215 kubenswrapper[2567]: W0728 18:27:27.270803    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d1df41e_3606_45eb_a66e_2e79977cfdbe.slice/crio-806796c47958a2be5ff46ccdd146fe6e8a6e2f999bec867ed359371e084e1500 WatchSource:0}: Error finding container 806796c47958a2be5ff46ccdd146fe6e8a6e2f999bec867ed359371e084e1500: Status 404 returned error can't find the container with id 806796c47958a2be5ff46ccdd146fe6e8a6e2f999bec867ed359371e084e1500
Jul 28 18:27:28.087373 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:28.087333    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24" event={"ID":"5d1df41e-3606-45eb-a66e-2e79977cfdbe","Type":"ContainerStarted","Data":"489f3844eaa15b65ed045437f4a38b3db3f1581316865df8cfaa7403afe09f06"}
Jul 28 18:27:28.087842 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:28.087392    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24" event={"ID":"5d1df41e-3606-45eb-a66e-2e79977cfdbe","Type":"ContainerStarted","Data":"806796c47958a2be5ff46ccdd146fe6e8a6e2f999bec867ed359371e084e1500"}
Jul 28 18:27:28.108866 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:28.108797    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24" podStartSLOduration=2.108775969 podStartE2EDuration="2.108775969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:27:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:27:28.10734743 +0000 UTC m=+4603.353449636" watchObservedRunningTime="2026-07-28 18:27:28.108775969 +0000 UTC m=+4603.354878176"
Jul 28 18:27:28.118408 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:28.118364    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:37.118547 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:37.118475    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:37.131417 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:37.131384    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:27:37.143433 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:27:37.143403    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:28:34.262608 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:34.262567    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"]
Jul 28 18:28:49.282632 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.282501    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24" podUID="5d1df41e-3606-45eb-a66e-2e79977cfdbe" containerName="main" containerID="cri-o://489f3844eaa15b65ed045437f4a38b3db3f1581316865df8cfaa7403afe09f06" gracePeriod=45
Jul 28 18:28:49.493642 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.493605    2567 generic.go:356] "Generic (PLEG): container finished" podID="5d1df41e-3606-45eb-a66e-2e79977cfdbe" containerID="489f3844eaa15b65ed045437f4a38b3db3f1581316865df8cfaa7403afe09f06" exitCode=0
Jul 28 18:28:49.493859 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.493651    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24" event={"ID":"5d1df41e-3606-45eb-a66e-2e79977cfdbe","Type":"ContainerDied","Data":"489f3844eaa15b65ed045437f4a38b3db3f1581316865df8cfaa7403afe09f06"}
Jul 28 18:28:49.542236 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.542160    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:28:49.598561 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.598497    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-dshm\") pod \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") "
Jul 28 18:28:49.598767 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.598631    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-tmp-dir\") pod \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") "
Jul 28 18:28:49.598767 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.598665    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gr2vv\" (UniqueName: \"kubernetes.io/projected/5d1df41e-3606-45eb-a66e-2e79977cfdbe-kube-api-access-gr2vv\") pod \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") "
Jul 28 18:28:49.598767 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.598683    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-model-cache\") pod \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") "
Jul 28 18:28:49.598767 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.598701    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-home\") pod \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") "
Jul 28 18:28:49.598767 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.598726    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d1df41e-3606-45eb-a66e-2e79977cfdbe-tls-certs\") pod \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\" (UID: \"5d1df41e-3606-45eb-a66e-2e79977cfdbe\") "
Jul 28 18:28:49.599040 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.598958    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-home" (OuterVolumeSpecName: "home") pod "5d1df41e-3606-45eb-a66e-2e79977cfdbe" (UID: "5d1df41e-3606-45eb-a66e-2e79977cfdbe"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:28:49.599040 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.598970    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5d1df41e-3606-45eb-a66e-2e79977cfdbe" (UID: "5d1df41e-3606-45eb-a66e-2e79977cfdbe"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:28:49.599040 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.598986    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-model-cache" (OuterVolumeSpecName: "model-cache") pod "5d1df41e-3606-45eb-a66e-2e79977cfdbe" (UID: "5d1df41e-3606-45eb-a66e-2e79977cfdbe"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:28:49.600874 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.600841    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-dshm" (OuterVolumeSpecName: "dshm") pod "5d1df41e-3606-45eb-a66e-2e79977cfdbe" (UID: "5d1df41e-3606-45eb-a66e-2e79977cfdbe"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:28:49.601012 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.600933    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d1df41e-3606-45eb-a66e-2e79977cfdbe-kube-api-access-gr2vv" (OuterVolumeSpecName: "kube-api-access-gr2vv") pod "5d1df41e-3606-45eb-a66e-2e79977cfdbe" (UID: "5d1df41e-3606-45eb-a66e-2e79977cfdbe"). InnerVolumeSpecName "kube-api-access-gr2vv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:28:49.601069 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.601034    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d1df41e-3606-45eb-a66e-2e79977cfdbe-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5d1df41e-3606-45eb-a66e-2e79977cfdbe" (UID: "5d1df41e-3606-45eb-a66e-2e79977cfdbe"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:28:49.699746 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.699696    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gr2vv\" (UniqueName: \"kubernetes.io/projected/5d1df41e-3606-45eb-a66e-2e79977cfdbe-kube-api-access-gr2vv\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:28:49.699746 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.699735    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-model-cache\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:28:49.699746 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.699744    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-home\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:28:49.699746 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.699752    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d1df41e-3606-45eb-a66e-2e79977cfdbe-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:28:49.699746 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.699760    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-dshm\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:28:49.700067 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:49.699768    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d1df41e-3606-45eb-a66e-2e79977cfdbe-tmp-dir\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:28:50.499823 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:50.499791    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"
Jul 28 18:28:50.500323 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:50.499788    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24" event={"ID":"5d1df41e-3606-45eb-a66e-2e79977cfdbe","Type":"ContainerDied","Data":"806796c47958a2be5ff46ccdd146fe6e8a6e2f999bec867ed359371e084e1500"}
Jul 28 18:28:50.500323 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:50.499933    2567 scope.go:117] "RemoveContainer" containerID="489f3844eaa15b65ed045437f4a38b3db3f1581316865df8cfaa7403afe09f06"
Jul 28 18:28:50.522992 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:50.522953    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"]
Jul 28 18:28:50.526903 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:50.526871    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v1-kserve-6cf8cbb847-tgp24"]
Jul 28 18:28:51.288049 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:28:51.288007    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d1df41e-3606-45eb-a66e-2e79977cfdbe" path="/var/lib/kubelet/pods/5d1df41e-3606-45eb-a66e-2e79977cfdbe/volumes"
Jul 28 18:30:21.108276 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.108237    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"]
Jul 28 18:30:21.108845 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.108568    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d1df41e-3606-45eb-a66e-2e79977cfdbe" containerName="main"
Jul 28 18:30:21.108845 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.108580    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d1df41e-3606-45eb-a66e-2e79977cfdbe" containerName="main"
Jul 28 18:30:21.108845 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.108641    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d1df41e-3606-45eb-a66e-2e79977cfdbe" containerName="main"
Jul 28 18:30:21.111738 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.111714    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:21.114988 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.114959    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-force-stop-route-owner-5f630121\"/\"openshift-service-ca.crt\""
Jul 28 18:30:21.116569 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.116544    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-force-stop-route-owner-5f630121\"/\"owner-v2-kserve-self-signed-certs\""
Jul 28 18:30:21.116569 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.116559    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-force-stop-route-owner-5f630121\"/\"default-dockercfg-v29pc\""
Jul 28 18:30:21.116811 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.116551    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-force-stop-route-owner-5f630121\"/\"kube-root-ca.crt\""
Jul 28 18:30:21.124423 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.124384    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"]
Jul 28 18:30:21.193837 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.193796    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18985600-9d98-4571-83ed-db613f8350f6-tls-certs\") pod \"owner-v2-kserve-5cf8959f95-w4kpq\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:21.194048 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.193843    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmtpj\" (UniqueName: \"kubernetes.io/projected/18985600-9d98-4571-83ed-db613f8350f6-kube-api-access-fmtpj\") pod \"owner-v2-kserve-5cf8959f95-w4kpq\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:21.194048 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.193909    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-tmp-dir\") pod \"owner-v2-kserve-5cf8959f95-w4kpq\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:21.194048 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.193980    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-model-cache\") pod \"owner-v2-kserve-5cf8959f95-w4kpq\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:21.194048 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.194027    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-dshm\") pod \"owner-v2-kserve-5cf8959f95-w4kpq\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:21.194203 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.194068    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-home\") pod \"owner-v2-kserve-5cf8959f95-w4kpq\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:21.295474 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.295418    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18985600-9d98-4571-83ed-db613f8350f6-tls-certs\") pod \"owner-v2-kserve-5cf8959f95-w4kpq\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:21.295723 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.295487    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmtpj\" (UniqueName: \"kubernetes.io/projected/18985600-9d98-4571-83ed-db613f8350f6-kube-api-access-fmtpj\") pod \"owner-v2-kserve-5cf8959f95-w4kpq\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:21.295723 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.295567    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-tmp-dir\") pod \"owner-v2-kserve-5cf8959f95-w4kpq\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:21.295723 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.295606    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-model-cache\") pod \"owner-v2-kserve-5cf8959f95-w4kpq\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:21.295723 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.295650    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-dshm\") pod \"owner-v2-kserve-5cf8959f95-w4kpq\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:21.295723 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.295703    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-home\") pod \"owner-v2-kserve-5cf8959f95-w4kpq\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:21.296134 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.296106    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-model-cache\") pod \"owner-v2-kserve-5cf8959f95-w4kpq\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:21.296233 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.296174    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-home\") pod \"owner-v2-kserve-5cf8959f95-w4kpq\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:21.296395 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.296367    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-tmp-dir\") pod \"owner-v2-kserve-5cf8959f95-w4kpq\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:21.298329 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.298294    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-dshm\") pod \"owner-v2-kserve-5cf8959f95-w4kpq\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:21.298949 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.298923    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18985600-9d98-4571-83ed-db613f8350f6-tls-certs\") pod \"owner-v2-kserve-5cf8959f95-w4kpq\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:21.306665 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.306631    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmtpj\" (UniqueName: \"kubernetes.io/projected/18985600-9d98-4571-83ed-db613f8350f6-kube-api-access-fmtpj\") pod \"owner-v2-kserve-5cf8959f95-w4kpq\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:21.425633 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.425524    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:21.576985 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.576953    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"]
Jul 28 18:30:21.579225 ip-10-0-133-215 kubenswrapper[2567]: W0728 18:30:21.579195    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18985600_9d98_4571_83ed_db613f8350f6.slice/crio-81dbebe7fe75bc76c20888e6c6c95ae892c27bdad3de134b2d453ca3ebbf9ff1 WatchSource:0}: Error finding container 81dbebe7fe75bc76c20888e6c6c95ae892c27bdad3de134b2d453ca3ebbf9ff1: Status 404 returned error can't find the container with id 81dbebe7fe75bc76c20888e6c6c95ae892c27bdad3de134b2d453ca3ebbf9ff1
Jul 28 18:30:21.957953 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.957909    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq" event={"ID":"18985600-9d98-4571-83ed-db613f8350f6","Type":"ContainerStarted","Data":"59573cb37ad19860595ef7b5193ca5d40caa3594e2a61a109153db0283b17f8e"}
Jul 28 18:30:21.958147 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.957977    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq" event={"ID":"18985600-9d98-4571-83ed-db613f8350f6","Type":"ContainerStarted","Data":"81dbebe7fe75bc76c20888e6c6c95ae892c27bdad3de134b2d453ca3ebbf9ff1"}
Jul 28 18:30:21.979580 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:21.979519    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq" podStartSLOduration=0.979486868 podStartE2EDuration="979.486868ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:30:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:30:21.97719019 +0000 UTC m=+4777.223292397" watchObservedRunningTime="2026-07-28 18:30:21.979486868 +0000 UTC m=+4777.225589077"
Jul 28 18:30:22.426317 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:22.426260    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:31.426585 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:31.426538    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:31.439082 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:31.439047    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:32.021334 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:32.021295    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:30:46.390422 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:46.390390    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:30:46.404575 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:30:46.404548    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:31:35.129312 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.129278    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"]
Jul 28 18:31:35.132377 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.132356    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.135720 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.135694    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"kube-root-ca.crt\""
Jul 28 18:31:35.135969 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.135951    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"router-gateway-1-openshift-default-dockercfg-bz5wt\""
Jul 28 18:31:35.136055 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.136033    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"openshift-service-ca.crt\""
Jul 28 18:31:35.136227 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.136209    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"openshift-gw-ca-root-cert\""
Jul 28 18:31:35.146213 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.146186    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"]
Jul 28 18:31:35.222492 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.222453    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-workload-socket\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.222706 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.222499    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-token\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.222706 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.222553    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.222706 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.222646    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-podinfo\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.222706 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.222692    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-credential-socket\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.222900 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.222711    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-workload-certs\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.222900 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.222793    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdc4j\" (UniqueName: \"kubernetes.io/projected/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-kube-api-access-wdc4j\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.222900 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.222863    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-envoy\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.222900 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.222891    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-data\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.324044 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.324008    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdc4j\" (UniqueName: \"kubernetes.io/projected/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-kube-api-access-wdc4j\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.324255 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.324063    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-envoy\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.324255 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.324088    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-data\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.324255 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.324134    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-workload-socket\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.324255 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.324165    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-token\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.324255 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.324189    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.324491 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.324427    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-podinfo\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.324588 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.324493    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-credential-socket\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.324588 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.324548    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-workload-certs\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.324696 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.324616    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-workload-socket\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.324696 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.324646    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-data\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.324918 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.324890    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-credential-socket\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.324990 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.324899    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-workload-certs\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.324990 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.324951    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.326651 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.326620    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-envoy\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.327027 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.327005    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-podinfo\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.337242 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.337200    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-token\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.337428 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.337321    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdc4j\" (UniqueName: \"kubernetes.io/projected/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-kube-api-access-wdc4j\") pod \"router-gateway-1-openshift-default-79b5f47cdc-vhdsh\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.444381 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.444294    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:35.586484 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.586446    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"]
Jul 28 18:31:35.589742 ip-10-0-133-215 kubenswrapper[2567]: W0728 18:31:35.589709    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc211b14e_7d7e_4d80_a316_a7d0eef98d9e.slice/crio-3936650cdc380675075d34e1e7808f89aaa80766bb189dd66dac2ed96c9f3fb7 WatchSource:0}: Error finding container 3936650cdc380675075d34e1e7808f89aaa80766bb189dd66dac2ed96c9f3fb7: Status 404 returned error can't find the container with id 3936650cdc380675075d34e1e7808f89aaa80766bb189dd66dac2ed96c9f3fb7
Jul 28 18:31:35.591811 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.591772    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 18:31:35.591935 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.591863    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 18:31:35.591935 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:35.591911    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 18:31:36.328354 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:36.328317    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh" event={"ID":"c211b14e-7d7e-4d80-a316-a7d0eef98d9e","Type":"ContainerStarted","Data":"d757002975d5f237b3dbc5dedd389d0e402cd49f2963633aab5153ea5fb16c47"}
Jul 28 18:31:36.328802 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:36.328378    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh" event={"ID":"c211b14e-7d7e-4d80-a316-a7d0eef98d9e","Type":"ContainerStarted","Data":"3936650cdc380675075d34e1e7808f89aaa80766bb189dd66dac2ed96c9f3fb7"}
Jul 28 18:31:36.352307 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:36.352255    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh" podStartSLOduration=1.352237084 podStartE2EDuration="1.352237084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:31:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:31:36.350237397 +0000 UTC m=+4851.596339606" watchObservedRunningTime="2026-07-28 18:31:36.352237084 +0000 UTC m=+4851.598339290"
Jul 28 18:31:36.445449 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:36.445412    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:36.450667 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:36.450642    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:37.336393 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:37.336354    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:37.337800 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:37.337773    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:31:39.881258 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:39.881210    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"]
Jul 28 18:31:54.902167 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:54.902120    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq" podUID="18985600-9d98-4571-83ed-db613f8350f6" containerName="main" containerID="cri-o://59573cb37ad19860595ef7b5193ca5d40caa3594e2a61a109153db0283b17f8e" gracePeriod=45
Jul 28 18:31:55.009817 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.009776    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq" podUID="18985600-9d98-4571-83ed-db613f8350f6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/health\": dial tcp 10.134.0.52:8000: connect: connection refused"
Jul 28 18:31:55.181945 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.181910    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:31:55.303947 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.303906    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmtpj\" (UniqueName: \"kubernetes.io/projected/18985600-9d98-4571-83ed-db613f8350f6-kube-api-access-fmtpj\") pod \"18985600-9d98-4571-83ed-db613f8350f6\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") "
Jul 28 18:31:55.304155 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.303998    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-tmp-dir\") pod \"18985600-9d98-4571-83ed-db613f8350f6\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") "
Jul 28 18:31:55.304155 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.304031    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-model-cache\") pod \"18985600-9d98-4571-83ed-db613f8350f6\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") "
Jul 28 18:31:55.304155 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.304075    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18985600-9d98-4571-83ed-db613f8350f6-tls-certs\") pod \"18985600-9d98-4571-83ed-db613f8350f6\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") "
Jul 28 18:31:55.304155 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.304102    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-home\") pod \"18985600-9d98-4571-83ed-db613f8350f6\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") "
Jul 28 18:31:55.304155 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.304133    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-dshm\") pod \"18985600-9d98-4571-83ed-db613f8350f6\" (UID: \"18985600-9d98-4571-83ed-db613f8350f6\") "
Jul 28 18:31:55.304577 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.304345    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-model-cache" (OuterVolumeSpecName: "model-cache") pod "18985600-9d98-4571-83ed-db613f8350f6" (UID: "18985600-9d98-4571-83ed-db613f8350f6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:31:55.304577 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.304351    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "18985600-9d98-4571-83ed-db613f8350f6" (UID: "18985600-9d98-4571-83ed-db613f8350f6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:31:55.304577 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.304398    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-home" (OuterVolumeSpecName: "home") pod "18985600-9d98-4571-83ed-db613f8350f6" (UID: "18985600-9d98-4571-83ed-db613f8350f6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:31:55.306750 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.306707    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18985600-9d98-4571-83ed-db613f8350f6-kube-api-access-fmtpj" (OuterVolumeSpecName: "kube-api-access-fmtpj") pod "18985600-9d98-4571-83ed-db613f8350f6" (UID: "18985600-9d98-4571-83ed-db613f8350f6"). InnerVolumeSpecName "kube-api-access-fmtpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:31:55.306950 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.306785    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-dshm" (OuterVolumeSpecName: "dshm") pod "18985600-9d98-4571-83ed-db613f8350f6" (UID: "18985600-9d98-4571-83ed-db613f8350f6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:31:55.306950 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.306789    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18985600-9d98-4571-83ed-db613f8350f6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "18985600-9d98-4571-83ed-db613f8350f6" (UID: "18985600-9d98-4571-83ed-db613f8350f6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:31:55.405094 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.405050    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18985600-9d98-4571-83ed-db613f8350f6-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:31:55.405094 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.405087    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-home\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:31:55.405094 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.405098    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-dshm\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:31:55.405358 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.405109    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmtpj\" (UniqueName: \"kubernetes.io/projected/18985600-9d98-4571-83ed-db613f8350f6-kube-api-access-fmtpj\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:31:55.405358 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.405148    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-tmp-dir\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:31:55.405358 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.405163    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18985600-9d98-4571-83ed-db613f8350f6-model-cache\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:31:55.436377 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.436265    2567 generic.go:356] "Generic (PLEG): container finished" podID="18985600-9d98-4571-83ed-db613f8350f6" containerID="59573cb37ad19860595ef7b5193ca5d40caa3594e2a61a109153db0283b17f8e" exitCode=0
Jul 28 18:31:55.436377 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.436326    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"
Jul 28 18:31:55.436640 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.436431    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq" event={"ID":"18985600-9d98-4571-83ed-db613f8350f6","Type":"ContainerDied","Data":"59573cb37ad19860595ef7b5193ca5d40caa3594e2a61a109153db0283b17f8e"}
Jul 28 18:31:55.436640 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.436479    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq" event={"ID":"18985600-9d98-4571-83ed-db613f8350f6","Type":"ContainerDied","Data":"81dbebe7fe75bc76c20888e6c6c95ae892c27bdad3de134b2d453ca3ebbf9ff1"}
Jul 28 18:31:55.436640 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.436524    2567 scope.go:117] "RemoveContainer" containerID="59573cb37ad19860595ef7b5193ca5d40caa3594e2a61a109153db0283b17f8e"
Jul 28 18:31:55.451542 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.451493    2567 scope.go:117] "RemoveContainer" containerID="59573cb37ad19860595ef7b5193ca5d40caa3594e2a61a109153db0283b17f8e"
Jul 28 18:31:55.451913 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:31:55.451889    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59573cb37ad19860595ef7b5193ca5d40caa3594e2a61a109153db0283b17f8e\": container with ID starting with 59573cb37ad19860595ef7b5193ca5d40caa3594e2a61a109153db0283b17f8e not found: ID does not exist" containerID="59573cb37ad19860595ef7b5193ca5d40caa3594e2a61a109153db0283b17f8e"
Jul 28 18:31:55.451992 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.451920    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59573cb37ad19860595ef7b5193ca5d40caa3594e2a61a109153db0283b17f8e"} err="failed to get container status \"59573cb37ad19860595ef7b5193ca5d40caa3594e2a61a109153db0283b17f8e\": rpc error: code = NotFound desc = could not find container \"59573cb37ad19860595ef7b5193ca5d40caa3594e2a61a109153db0283b17f8e\": container with ID starting with 59573cb37ad19860595ef7b5193ca5d40caa3594e2a61a109153db0283b17f8e not found: ID does not exist"
Jul 28 18:31:55.464914 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.464879    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"]
Jul 28 18:31:55.466349 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:31:55.466317    2567 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18985600_9d98_4571_83ed_db613f8350f6.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18985600_9d98_4571_83ed_db613f8350f6.slice/crio-81dbebe7fe75bc76c20888e6c6c95ae892c27bdad3de134b2d453ca3ebbf9ff1\": RecentStats: unable to find data in memory cache]"
Jul 28 18:31:55.471007 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:55.470970    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-w4kpq"]
Jul 28 18:31:57.289308 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:57.289260    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18985600-9d98-4571-83ed-db613f8350f6" path="/var/lib/kubelet/pods/18985600-9d98-4571-83ed-db613f8350f6/volumes"
Jul 28 18:31:58.464197 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:58.464086    2567 generic.go:356] "Generic (PLEG): container finished" podID="c211b14e-7d7e-4d80-a316-a7d0eef98d9e" containerID="d757002975d5f237b3dbc5dedd389d0e402cd49f2963633aab5153ea5fb16c47" exitCode=0
Jul 28 18:31:58.464197 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:58.464127    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh" event={"ID":"c211b14e-7d7e-4d80-a316-a7d0eef98d9e","Type":"ContainerDied","Data":"d757002975d5f237b3dbc5dedd389d0e402cd49f2963633aab5153ea5fb16c47"}
Jul 28 18:31:58.464756 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:58.464616    2567 scope.go:117] "RemoveContainer" containerID="d757002975d5f237b3dbc5dedd389d0e402cd49f2963633aab5153ea5fb16c47"
Jul 28 18:31:58.465712 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:58.465677    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 18:31:58.465819 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:58.465761    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 18:31:58.465819 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:58.465805    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 18:31:59.470748 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:31:59.470709    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh" event={"ID":"c211b14e-7d7e-4d80-a316-a7d0eef98d9e","Type":"ContainerStarted","Data":"4ee084adee8c73796fd5294b155277191f45d03c8ce5ce195a8d3f37b9ebfdce"}
Jul 28 18:32:00.445549 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:32:00.445491    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:32:00.445839 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:32:00.445807    2567 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh" podUID="c211b14e-7d7e-4d80-a316-a7d0eef98d9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 28 18:32:01.445142 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:32:01.445095    2567 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh" podUID="c211b14e-7d7e-4d80-a316-a7d0eef98d9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 28 18:32:02.445434 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:32:02.445391    2567 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh" podUID="c211b14e-7d7e-4d80-a316-a7d0eef98d9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 28 18:32:03.444981 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:32:03.444933    2567 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh" podUID="c211b14e-7d7e-4d80-a316-a7d0eef98d9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 28 18:32:04.445196 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:32:04.445147    2567 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh" podUID="c211b14e-7d7e-4d80-a316-a7d0eef98d9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 28 18:32:05.449331 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:32:05.449287    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:32:05.449834 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:32:05.449815    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:32:05.450516 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:32:05.450480    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:33:43.214276 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:43.214227    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"]
Jul 28 18:33:43.214879 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:43.214481    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh" podUID="c211b14e-7d7e-4d80-a316-a7d0eef98d9e" containerName="istio-proxy" containerID="cri-o://4ee084adee8c73796fd5294b155277191f45d03c8ce5ce195a8d3f37b9ebfdce" gracePeriod=30
Jul 28 18:33:48.459649 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.459617    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:33:48.554671 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.554640    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istiod-ca-cert\") pod \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") "
Jul 28 18:33:48.554824 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.554686    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-credential-socket\") pod \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") "
Jul 28 18:33:48.554824 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.554733    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-data\") pod \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") "
Jul 28 18:33:48.554954 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.554834    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-workload-socket\") pod \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") "
Jul 28 18:33:48.554954 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.554912    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-workload-certs\") pod \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") "
Jul 28 18:33:48.554954 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.554939    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-envoy\") pod \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") "
Jul 28 18:33:48.555112 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.554983    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-podinfo\") pod \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") "
Jul 28 18:33:48.555112 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.554998    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-data" (OuterVolumeSpecName: "istio-data") pod "c211b14e-7d7e-4d80-a316-a7d0eef98d9e" (UID: "c211b14e-7d7e-4d80-a316-a7d0eef98d9e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:33:48.555112 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.555011    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-token\") pod \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") "
Jul 28 18:33:48.555112 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.555037    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdc4j\" (UniqueName: \"kubernetes.io/projected/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-kube-api-access-wdc4j\") pod \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\" (UID: \"c211b14e-7d7e-4d80-a316-a7d0eef98d9e\") "
Jul 28 18:33:48.555112 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.555021    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c211b14e-7d7e-4d80-a316-a7d0eef98d9e" (UID: "c211b14e-7d7e-4d80-a316-a7d0eef98d9e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:33:48.555367 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.555100    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c211b14e-7d7e-4d80-a316-a7d0eef98d9e" (UID: "c211b14e-7d7e-4d80-a316-a7d0eef98d9e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:33:48.555367 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.555190    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c211b14e-7d7e-4d80-a316-a7d0eef98d9e" (UID: "c211b14e-7d7e-4d80-a316-a7d0eef98d9e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:33:48.555367 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.555101    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c211b14e-7d7e-4d80-a316-a7d0eef98d9e" (UID: "c211b14e-7d7e-4d80-a316-a7d0eef98d9e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:33:48.555367 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.555348    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istiod-ca-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:33:48.555567 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.555373    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-credential-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:33:48.555567 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.555393    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-data\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:33:48.555567 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.555406    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-workload-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:33:48.555567 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.555417    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-workload-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:33:48.557223 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.557201    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-token" (OuterVolumeSpecName: "istio-token") pod "c211b14e-7d7e-4d80-a316-a7d0eef98d9e" (UID: "c211b14e-7d7e-4d80-a316-a7d0eef98d9e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:33:48.557325 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.557240    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-kube-api-access-wdc4j" (OuterVolumeSpecName: "kube-api-access-wdc4j") pod "c211b14e-7d7e-4d80-a316-a7d0eef98d9e" (UID: "c211b14e-7d7e-4d80-a316-a7d0eef98d9e"). InnerVolumeSpecName "kube-api-access-wdc4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:33:48.557473 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.557445    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c211b14e-7d7e-4d80-a316-a7d0eef98d9e" (UID: "c211b14e-7d7e-4d80-a316-a7d0eef98d9e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:33:48.557552 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.557447    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c211b14e-7d7e-4d80-a316-a7d0eef98d9e" (UID: "c211b14e-7d7e-4d80-a316-a7d0eef98d9e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 18:33:48.656125 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.656081    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-envoy\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:33:48.656125 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.656116    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-podinfo\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:33:48.656125 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.656129    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-istio-token\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:33:48.656376 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:48.656142    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdc4j\" (UniqueName: \"kubernetes.io/projected/c211b14e-7d7e-4d80-a316-a7d0eef98d9e-kube-api-access-wdc4j\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:33:49.031005 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:49.030966    2567 generic.go:356] "Generic (PLEG): container finished" podID="c211b14e-7d7e-4d80-a316-a7d0eef98d9e" containerID="4ee084adee8c73796fd5294b155277191f45d03c8ce5ce195a8d3f37b9ebfdce" exitCode=0
Jul 28 18:33:49.031171 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:49.031022    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"
Jul 28 18:33:49.031171 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:49.031049    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh" event={"ID":"c211b14e-7d7e-4d80-a316-a7d0eef98d9e","Type":"ContainerDied","Data":"4ee084adee8c73796fd5294b155277191f45d03c8ce5ce195a8d3f37b9ebfdce"}
Jul 28 18:33:49.031171 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:49.031103    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh" event={"ID":"c211b14e-7d7e-4d80-a316-a7d0eef98d9e","Type":"ContainerDied","Data":"3936650cdc380675075d34e1e7808f89aaa80766bb189dd66dac2ed96c9f3fb7"}
Jul 28 18:33:49.031171 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:49.031125    2567 scope.go:117] "RemoveContainer" containerID="4ee084adee8c73796fd5294b155277191f45d03c8ce5ce195a8d3f37b9ebfdce"
Jul 28 18:33:49.040610 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:49.040589    2567 scope.go:117] "RemoveContainer" containerID="d757002975d5f237b3dbc5dedd389d0e402cd49f2963633aab5153ea5fb16c47"
Jul 28 18:33:49.049378 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:49.049360    2567 scope.go:117] "RemoveContainer" containerID="4ee084adee8c73796fd5294b155277191f45d03c8ce5ce195a8d3f37b9ebfdce"
Jul 28 18:33:49.049694 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:33:49.049675    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ee084adee8c73796fd5294b155277191f45d03c8ce5ce195a8d3f37b9ebfdce\": container with ID starting with 4ee084adee8c73796fd5294b155277191f45d03c8ce5ce195a8d3f37b9ebfdce not found: ID does not exist" containerID="4ee084adee8c73796fd5294b155277191f45d03c8ce5ce195a8d3f37b9ebfdce"
Jul 28 18:33:49.049749 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:49.049703    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ee084adee8c73796fd5294b155277191f45d03c8ce5ce195a8d3f37b9ebfdce"} err="failed to get container status \"4ee084adee8c73796fd5294b155277191f45d03c8ce5ce195a8d3f37b9ebfdce\": rpc error: code = NotFound desc = could not find container \"4ee084adee8c73796fd5294b155277191f45d03c8ce5ce195a8d3f37b9ebfdce\": container with ID starting with 4ee084adee8c73796fd5294b155277191f45d03c8ce5ce195a8d3f37b9ebfdce not found: ID does not exist"
Jul 28 18:33:49.049749 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:49.049722    2567 scope.go:117] "RemoveContainer" containerID="d757002975d5f237b3dbc5dedd389d0e402cd49f2963633aab5153ea5fb16c47"
Jul 28 18:33:49.049965 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:33:49.049945    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d757002975d5f237b3dbc5dedd389d0e402cd49f2963633aab5153ea5fb16c47\": container with ID starting with d757002975d5f237b3dbc5dedd389d0e402cd49f2963633aab5153ea5fb16c47 not found: ID does not exist" containerID="d757002975d5f237b3dbc5dedd389d0e402cd49f2963633aab5153ea5fb16c47"
Jul 28 18:33:49.050008 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:49.049973    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d757002975d5f237b3dbc5dedd389d0e402cd49f2963633aab5153ea5fb16c47"} err="failed to get container status \"d757002975d5f237b3dbc5dedd389d0e402cd49f2963633aab5153ea5fb16c47\": rpc error: code = NotFound desc = could not find container \"d757002975d5f237b3dbc5dedd389d0e402cd49f2963633aab5153ea5fb16c47\": container with ID starting with d757002975d5f237b3dbc5dedd389d0e402cd49f2963633aab5153ea5fb16c47 not found: ID does not exist"
Jul 28 18:33:49.055748 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:49.055717    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"]
Jul 28 18:33:49.059517 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:49.059477    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-79b5f47cdc-vhdsh"]
Jul 28 18:33:49.286643 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:49.286552    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c211b14e-7d7e-4d80-a316-a7d0eef98d9e" path="/var/lib/kubelet/pods/c211b14e-7d7e-4d80-a316-a7d0eef98d9e/volumes"
Jul 28 18:33:50.190764 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.190714    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx"]
Jul 28 18:33:50.191203 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.191185    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c211b14e-7d7e-4d80-a316-a7d0eef98d9e" containerName="istio-proxy"
Jul 28 18:33:50.191281 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.191205    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c211b14e-7d7e-4d80-a316-a7d0eef98d9e" containerName="istio-proxy"
Jul 28 18:33:50.191281 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.191234    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18985600-9d98-4571-83ed-db613f8350f6" containerName="main"
Jul 28 18:33:50.191281 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.191243    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="18985600-9d98-4571-83ed-db613f8350f6" containerName="main"
Jul 28 18:33:50.191422 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.191321    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c211b14e-7d7e-4d80-a316-a7d0eef98d9e" containerName="istio-proxy"
Jul 28 18:33:50.191422 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.191335    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="18985600-9d98-4571-83ed-db613f8350f6" containerName="main"
Jul 28 18:33:50.195044 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.195019    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx"
Jul 28 18:33:50.198009 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.197977    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"openshift-service-ca.crt\""
Jul 28 18:33:50.200193 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.199347    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"default-dockercfg-k2crv\""
Jul 28 18:33:50.200193 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.199640    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"kube-root-ca.crt\""
Jul 28 18:33:50.200193 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.199908    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"llmisvc-model-fb-opt-125m-route-87882a8e-epp-sa-dockercfg-pdxmh\""
Jul 28 18:33:50.200193 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.200120    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs\""
Jul 28 18:33:50.203780 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.203756    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx"]
Jul 28 18:33:50.268852 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.268810    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4hqt\" (UniqueName: \"kubernetes.io/projected/09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed-kube-api-access-v4hqt\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx\" (UID: \"09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx"
Jul 28 18:33:50.268852 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.268857    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx\" (UID: \"09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx"
Jul 28 18:33:50.369296 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.369252    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4hqt\" (UniqueName: \"kubernetes.io/projected/09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed-kube-api-access-v4hqt\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx\" (UID: \"09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx"
Jul 28 18:33:50.369296 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.369295    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx\" (UID: \"09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx"
Jul 28 18:33:50.371834 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.371798    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx\" (UID: \"09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx"
Jul 28 18:33:50.377753 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.377727    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4hqt\" (UniqueName: \"kubernetes.io/projected/09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed-kube-api-access-v4hqt\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx\" (UID: \"09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx"
Jul 28 18:33:50.510927 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.510882    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx"
Jul 28 18:33:50.657409 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:50.657374    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx"]
Jul 28 18:33:50.661346 ip-10-0-133-215 kubenswrapper[2567]: W0728 18:33:50.661316    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09fcfe9b_d5d7_4698_92ed_1c6f8bb195ed.slice/crio-8f12f0f40578a67091b10cf81861082ee6e1dcf8d834c8dc0f3faf69e53e3565 WatchSource:0}: Error finding container 8f12f0f40578a67091b10cf81861082ee6e1dcf8d834c8dc0f3faf69e53e3565: Status 404 returned error can't find the container with id 8f12f0f40578a67091b10cf81861082ee6e1dcf8d834c8dc0f3faf69e53e3565
Jul 28 18:33:51.045241 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:51.045204    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx" event={"ID":"09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed","Type":"ContainerStarted","Data":"f65c1502fdbcb98d0ece1cbd66f0c01a5e6e90825806727edb0ce46a7870f8b6"}
Jul 28 18:33:51.045429 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:51.045264    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx" event={"ID":"09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed","Type":"ContainerStarted","Data":"8f12f0f40578a67091b10cf81861082ee6e1dcf8d834c8dc0f3faf69e53e3565"}
Jul 28 18:33:51.045429 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:51.045288    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx"
Jul 28 18:33:51.064770 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:51.064706    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx" podStartSLOduration=1.064687909 podStartE2EDuration="1.064687909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:33:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:33:51.062251869 +0000 UTC m=+4986.308354077" watchObservedRunningTime="2026-07-28 18:33:51.064687909 +0000 UTC m=+4986.310790115"
Jul 28 18:33:55.681367 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:55.681332    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"]
Jul 28 18:33:55.690625 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:33:55.690586    2567 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 28 18:33:55.690625 ip-10-0-133-215 kubenswrapper[2567]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 18:33:55.690625 ip-10-0-133-215 kubenswrapper[2567]:         , stdout: , stderr: , exit code -1
Jul 28 18:33:55.690625 ip-10-0-133-215 kubenswrapper[2567]:  > containerID="0797dc72991934cc79f8e998ffbb43e1bec0250cbd8871a36326ebb34a0486cf" cmd=["/bin/sleep","15"]
Jul 28 18:33:55.690938 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:55.690642    2567 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" message="" err=<
Jul 28 18:33:55.690938 ip-10-0-133-215 kubenswrapper[2567]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 18:33:55.690938 ip-10-0-133-215 kubenswrapper[2567]:         , stdout: , stderr: , exit code -1
Jul 28 18:33:55.690938 ip-10-0-133-215 kubenswrapper[2567]:  >
Jul 28 18:33:55.690938 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:33:55.690653    2567 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 28 18:33:55.690938 ip-10-0-133-215 kubenswrapper[2567]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 18:33:55.690938 ip-10-0-133-215 kubenswrapper[2567]:         , stdout: , stderr: , exit code -1
Jul 28 18:33:55.690938 ip-10-0-133-215 kubenswrapper[2567]:  > pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" podUID="699d2b4a-79e3-495e-a50f-07278ae0c7ba" containerName="main" containerID="cri-o://0797dc72991934cc79f8e998ffbb43e1bec0250cbd8871a36326ebb34a0486cf"
Jul 28 18:33:55.690938 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:55.690692    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" podUID="699d2b4a-79e3-495e-a50f-07278ae0c7ba" containerName="main" containerID="cri-o://0797dc72991934cc79f8e998ffbb43e1bec0250cbd8871a36326ebb34a0486cf" gracePeriod=60
Jul 28 18:33:56.045092 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.045060    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:33:56.070334 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.070292    2567 generic.go:356] "Generic (PLEG): container finished" podID="699d2b4a-79e3-495e-a50f-07278ae0c7ba" containerID="0797dc72991934cc79f8e998ffbb43e1bec0250cbd8871a36326ebb34a0486cf" exitCode=0
Jul 28 18:33:56.070334 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.070333    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 18:33:56.070617 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.070393    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" event={"ID":"699d2b4a-79e3-495e-a50f-07278ae0c7ba","Type":"ContainerDied","Data":"0797dc72991934cc79f8e998ffbb43e1bec0250cbd8871a36326ebb34a0486cf"}
Jul 28 18:33:56.070617 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.070463    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" event={"ID":"699d2b4a-79e3-495e-a50f-07278ae0c7ba","Type":"ContainerDied","Data":"7d70278c1a55f7e5e48192e58aa173d49e6d2cfb6e63bf2e197382021df581b3"}
Jul 28 18:33:56.070617 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.070494    2567 scope.go:117] "RemoveContainer" containerID="0797dc72991934cc79f8e998ffbb43e1bec0250cbd8871a36326ebb34a0486cf"
Jul 28 18:33:56.081040 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.081012    2567 scope.go:117] "RemoveContainer" containerID="0797dc72991934cc79f8e998ffbb43e1bec0250cbd8871a36326ebb34a0486cf"
Jul 28 18:33:56.081430 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:33:56.081404    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0797dc72991934cc79f8e998ffbb43e1bec0250cbd8871a36326ebb34a0486cf\": container with ID starting with 0797dc72991934cc79f8e998ffbb43e1bec0250cbd8871a36326ebb34a0486cf not found: ID does not exist" containerID="0797dc72991934cc79f8e998ffbb43e1bec0250cbd8871a36326ebb34a0486cf"
Jul 28 18:33:56.081525 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.081442    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0797dc72991934cc79f8e998ffbb43e1bec0250cbd8871a36326ebb34a0486cf"} err="failed to get container status \"0797dc72991934cc79f8e998ffbb43e1bec0250cbd8871a36326ebb34a0486cf\": rpc error: code = NotFound desc = could not find container \"0797dc72991934cc79f8e998ffbb43e1bec0250cbd8871a36326ebb34a0486cf\": container with ID starting with 0797dc72991934cc79f8e998ffbb43e1bec0250cbd8871a36326ebb34a0486cf not found: ID does not exist"
Jul 28 18:33:56.116968 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.116928    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kkpf\" (UniqueName: \"kubernetes.io/projected/699d2b4a-79e3-495e-a50f-07278ae0c7ba-kube-api-access-6kkpf\") pod \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") "
Jul 28 18:33:56.117167 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.116995    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-home\") pod \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") "
Jul 28 18:33:56.117167 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.117019    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-model-cache\") pod \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") "
Jul 28 18:33:56.117167 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.117050    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/699d2b4a-79e3-495e-a50f-07278ae0c7ba-tls-certs\") pod \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") "
Jul 28 18:33:56.117167 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.117111    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-dshm\") pod \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") "
Jul 28 18:33:56.117167 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.117139    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-tmp-dir\") pod \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\" (UID: \"699d2b4a-79e3-495e-a50f-07278ae0c7ba\") "
Jul 28 18:33:56.117530 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.117304    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-home" (OuterVolumeSpecName: "home") pod "699d2b4a-79e3-495e-a50f-07278ae0c7ba" (UID: "699d2b4a-79e3-495e-a50f-07278ae0c7ba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:33:56.117530 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.117419    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-model-cache" (OuterVolumeSpecName: "model-cache") pod "699d2b4a-79e3-495e-a50f-07278ae0c7ba" (UID: "699d2b4a-79e3-495e-a50f-07278ae0c7ba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:33:56.117530 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.117437    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-home\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:33:56.117704 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.117538    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "699d2b4a-79e3-495e-a50f-07278ae0c7ba" (UID: "699d2b4a-79e3-495e-a50f-07278ae0c7ba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:33:56.119500 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.119469    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-dshm" (OuterVolumeSpecName: "dshm") pod "699d2b4a-79e3-495e-a50f-07278ae0c7ba" (UID: "699d2b4a-79e3-495e-a50f-07278ae0c7ba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:33:56.119653 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.119536    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/699d2b4a-79e3-495e-a50f-07278ae0c7ba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "699d2b4a-79e3-495e-a50f-07278ae0c7ba" (UID: "699d2b4a-79e3-495e-a50f-07278ae0c7ba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:33:56.119702 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.119672    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/699d2b4a-79e3-495e-a50f-07278ae0c7ba-kube-api-access-6kkpf" (OuterVolumeSpecName: "kube-api-access-6kkpf") pod "699d2b4a-79e3-495e-a50f-07278ae0c7ba" (UID: "699d2b4a-79e3-495e-a50f-07278ae0c7ba"). InnerVolumeSpecName "kube-api-access-6kkpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:33:56.218715 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.218592    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-dshm\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:33:56.218715 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.218638    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-tmp-dir\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:33:56.218715 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.218656    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kkpf\" (UniqueName: \"kubernetes.io/projected/699d2b4a-79e3-495e-a50f-07278ae0c7ba-kube-api-access-6kkpf\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:33:56.218715 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.218670    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/699d2b4a-79e3-495e-a50f-07278ae0c7ba-model-cache\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:33:56.218715 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.218686    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/699d2b4a-79e3-495e-a50f-07278ae0c7ba-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:33:56.394691 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.394650    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"]
Jul 28 18:33:56.400793 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:56.400758    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"]
Jul 28 18:33:57.291944 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:33:57.291895    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="699d2b4a-79e3-495e-a50f-07278ae0c7ba" path="/var/lib/kubelet/pods/699d2b4a-79e3-495e-a50f-07278ae0c7ba/volumes"
Jul 28 18:34:22.051712 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:22.051674    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx"
Jul 28 18:34:24.558048 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.558000    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"]
Jul 28 18:34:24.558457 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.558317    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c211b14e-7d7e-4d80-a316-a7d0eef98d9e" containerName="istio-proxy"
Jul 28 18:34:24.558457 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.558327    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c211b14e-7d7e-4d80-a316-a7d0eef98d9e" containerName="istio-proxy"
Jul 28 18:34:24.558457 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.558342    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="699d2b4a-79e3-495e-a50f-07278ae0c7ba" containerName="main"
Jul 28 18:34:24.558457 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.558348    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="699d2b4a-79e3-495e-a50f-07278ae0c7ba" containerName="main"
Jul 28 18:34:24.558457 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.558393    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="699d2b4a-79e3-495e-a50f-07278ae0c7ba" containerName="main"
Jul 28 18:34:24.558457 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.558399    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c211b14e-7d7e-4d80-a316-a7d0eef98d9e" containerName="istio-proxy"
Jul 28 18:34:24.560582 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.560559    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:34:24.564182 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.564143    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-keda-lws-e541a132\"/\"kube-root-ca.crt\""
Jul 28 18:34:24.565615 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.565571    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-keda-lws-e541a132\"/\"openshift-service-ca.crt\""
Jul 28 18:34:24.565782 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.565578    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-keda-lws-e541a132\"/\"autoscale-keda-lws-kserve-self-signed-certs\""
Jul 28 18:34:24.565878 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.565584    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-keda-lws-e541a132\"/\"autoscale-keda-lws-kserve-mn-dockercfg-7gs5h\""
Jul 28 18:34:24.566065 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.566040    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-keda-lws-e541a132\"/\"default-dockercfg-472qz\""
Jul 28 18:34:24.571910 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.571866    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"]
Jul 28 18:34:24.660215 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.660175    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-tmp-dir\") pod \"autoscale-keda-lws-kserve-mn-0-1\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:34:24.660215 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.660214    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-dshm\") pod \"autoscale-keda-lws-kserve-mn-0-1\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:34:24.660585 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.660261    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-model-cache\") pod \"autoscale-keda-lws-kserve-mn-0-1\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:34:24.660585 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.660289    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e87911f-78f4-4c0c-8964-b653ced0dc3e-tls-certs\") pod \"autoscale-keda-lws-kserve-mn-0-1\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:34:24.660585 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.660321    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-home\") pod \"autoscale-keda-lws-kserve-mn-0-1\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:34:24.660585 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.660412    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmrgs\" (UniqueName: \"kubernetes.io/projected/3e87911f-78f4-4c0c-8964-b653ced0dc3e-kube-api-access-xmrgs\") pod \"autoscale-keda-lws-kserve-mn-0-1\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:34:24.761497 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.761438    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-model-cache\") pod \"autoscale-keda-lws-kserve-mn-0-1\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:34:24.761497 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.761497    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e87911f-78f4-4c0c-8964-b653ced0dc3e-tls-certs\") pod \"autoscale-keda-lws-kserve-mn-0-1\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:34:24.761786 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.761552    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-home\") pod \"autoscale-keda-lws-kserve-mn-0-1\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:34:24.761786 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.761612    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmrgs\" (UniqueName: \"kubernetes.io/projected/3e87911f-78f4-4c0c-8964-b653ced0dc3e-kube-api-access-xmrgs\") pod \"autoscale-keda-lws-kserve-mn-0-1\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:34:24.761786 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.761657    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-tmp-dir\") pod \"autoscale-keda-lws-kserve-mn-0-1\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:34:24.761786 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.761680    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-dshm\") pod \"autoscale-keda-lws-kserve-mn-0-1\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:34:24.762008 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.761927    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-model-cache\") pod \"autoscale-keda-lws-kserve-mn-0-1\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:34:24.762300 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.762264    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-home\") pod \"autoscale-keda-lws-kserve-mn-0-1\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:34:24.762494 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.762472    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-tmp-dir\") pod \"autoscale-keda-lws-kserve-mn-0-1\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:34:24.764214 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.764185    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-dshm\") pod \"autoscale-keda-lws-kserve-mn-0-1\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:34:24.764421 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.764402    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e87911f-78f4-4c0c-8964-b653ced0dc3e-tls-certs\") pod \"autoscale-keda-lws-kserve-mn-0-1\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:34:24.770618 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.770589    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmrgs\" (UniqueName: \"kubernetes.io/projected/3e87911f-78f4-4c0c-8964-b653ced0dc3e-kube-api-access-xmrgs\") pod \"autoscale-keda-lws-kserve-mn-0-1\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:34:24.873336 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:24.873247    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:34:25.011638 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:25.011603    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"]
Jul 28 18:34:25.013900 ip-10-0-133-215 kubenswrapper[2567]: W0728 18:34:25.013865    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e87911f_78f4_4c0c_8964_b653ced0dc3e.slice/crio-96d405e1411371cf67382529b063e3acae0f0b295faf0073781b50d183197d98 WatchSource:0}: Error finding container 96d405e1411371cf67382529b063e3acae0f0b295faf0073781b50d183197d98: Status 404 returned error can't find the container with id 96d405e1411371cf67382529b063e3acae0f0b295faf0073781b50d183197d98
Jul 28 18:34:25.213270 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:25.213159    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1" event={"ID":"3e87911f-78f4-4c0c-8964-b653ced0dc3e","Type":"ContainerStarted","Data":"70b8ae96f88afe929ac975fbaa876424b2f29fea41bd91431a5582eb6aef1de6"}
Jul 28 18:34:25.213270 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:25.213230    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1" event={"ID":"3e87911f-78f4-4c0c-8964-b653ced0dc3e","Type":"ContainerStarted","Data":"96d405e1411371cf67382529b063e3acae0f0b295faf0073781b50d183197d98"}
Jul 28 18:34:25.231018 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:34:25.230937    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1" podStartSLOduration=1.230920424 podStartE2EDuration="1.230920424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:34:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:34:25.229683135 +0000 UTC m=+5020.475785344" watchObservedRunningTime="2026-07-28 18:34:25.230920424 +0000 UTC m=+5020.477022631"
Jul 28 18:35:46.419353 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:35:46.419324    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:35:46.437252 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:35:46.437225    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:35:54.418529 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:35:54.418478    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx"]
Jul 28 18:36:02.051372 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:02.051326    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx" podUID="09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 18:36:09.437998 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:09.437894    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx" podUID="09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed" containerName="main" containerID="cri-o://f65c1502fdbcb98d0ece1cbd66f0c01a5e6e90825806727edb0ce46a7870f8b6" gracePeriod=45
Jul 28 18:36:09.693933 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:09.693865    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx"
Jul 28 18:36:09.730608 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:09.730565    2567 generic.go:356] "Generic (PLEG): container finished" podID="09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed" containerID="f65c1502fdbcb98d0ece1cbd66f0c01a5e6e90825806727edb0ce46a7870f8b6" exitCode=0
Jul 28 18:36:09.730608 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:09.730609    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx"
Jul 28 18:36:09.730877 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:09.730621    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx" event={"ID":"09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed","Type":"ContainerDied","Data":"f65c1502fdbcb98d0ece1cbd66f0c01a5e6e90825806727edb0ce46a7870f8b6"}
Jul 28 18:36:09.730877 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:09.730664    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx" event={"ID":"09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed","Type":"ContainerDied","Data":"8f12f0f40578a67091b10cf81861082ee6e1dcf8d834c8dc0f3faf69e53e3565"}
Jul 28 18:36:09.730877 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:09.730686    2567 scope.go:117] "RemoveContainer" containerID="f65c1502fdbcb98d0ece1cbd66f0c01a5e6e90825806727edb0ce46a7870f8b6"
Jul 28 18:36:09.739078 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:09.739059    2567 scope.go:117] "RemoveContainer" containerID="f65c1502fdbcb98d0ece1cbd66f0c01a5e6e90825806727edb0ce46a7870f8b6"
Jul 28 18:36:09.739350 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:36:09.739326    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f65c1502fdbcb98d0ece1cbd66f0c01a5e6e90825806727edb0ce46a7870f8b6\": container with ID starting with f65c1502fdbcb98d0ece1cbd66f0c01a5e6e90825806727edb0ce46a7870f8b6 not found: ID does not exist" containerID="f65c1502fdbcb98d0ece1cbd66f0c01a5e6e90825806727edb0ce46a7870f8b6"
Jul 28 18:36:09.739426 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:09.739360    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f65c1502fdbcb98d0ece1cbd66f0c01a5e6e90825806727edb0ce46a7870f8b6"} err="failed to get container status \"f65c1502fdbcb98d0ece1cbd66f0c01a5e6e90825806727edb0ce46a7870f8b6\": rpc error: code = NotFound desc = could not find container \"f65c1502fdbcb98d0ece1cbd66f0c01a5e6e90825806727edb0ce46a7870f8b6\": container with ID starting with f65c1502fdbcb98d0ece1cbd66f0c01a5e6e90825806727edb0ce46a7870f8b6 not found: ID does not exist"
Jul 28 18:36:09.863785 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:09.863743    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4hqt\" (UniqueName: \"kubernetes.io/projected/09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed-kube-api-access-v4hqt\") pod \"09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed\" (UID: \"09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed\") "
Jul 28 18:36:09.863960 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:09.863846    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed-tls-certs\") pod \"09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed\" (UID: \"09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed\") "
Jul 28 18:36:09.866216 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:09.866188    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed" (UID: "09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:36:09.866283 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:09.866256    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed-kube-api-access-v4hqt" (OuterVolumeSpecName: "kube-api-access-v4hqt") pod "09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed" (UID: "09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed"). InnerVolumeSpecName "kube-api-access-v4hqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:36:09.965176 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:09.965080    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4hqt\" (UniqueName: \"kubernetes.io/projected/09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed-kube-api-access-v4hqt\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:36:09.965176 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:09.965112    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:36:10.053786 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:10.053752    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx"]
Jul 28 18:36:10.061692 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:10.061664    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schel8zrx"]
Jul 28 18:36:11.287321 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:11.287283    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed" path="/var/lib/kubelet/pods/09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed/volumes"
Jul 28 18:36:23.469888 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:23.469844    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr"]
Jul 28 18:36:23.470572 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:23.470255    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed" containerName="main"
Jul 28 18:36:23.470572 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:23.470274    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed" containerName="main"
Jul 28 18:36:23.470572 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:23.470341    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="09fcfe9b-d5d7-4698-92ed-1c6f8bb195ed" containerName="main"
Jul 28 18:36:23.475231 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:23.475197    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr"
Jul 28 18:36:23.478163 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:23.478132    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"default-dockercfg-cp56k\""
Jul 28 18:36:23.478343 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:23.478165    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"kube-root-ca.crt\""
Jul 28 18:36:23.479612 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:23.479592    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"openshift-service-ca.crt\""
Jul 28 18:36:23.479754 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:23.479730    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"custom-route-timeout-test-epp-sa-dockercfg-kbbml\""
Jul 28 18:36:23.479879 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:23.479826    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"custom-route-timeout-test-kserve-self-signed-certs\""
Jul 28 18:36:23.484363 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:23.483965    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr"]
Jul 28 18:36:23.579729 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:23.579678    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn87v\" (UniqueName: \"kubernetes.io/projected/29230a74-2fad-41f0-a99c-85a2b91521ae-kube-api-access-sn87v\") pod \"custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr\" (UID: \"29230a74-2fad-41f0-a99c-85a2b91521ae\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr"
Jul 28 18:36:23.579949 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:23.579739    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29230a74-2fad-41f0-a99c-85a2b91521ae-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr\" (UID: \"29230a74-2fad-41f0-a99c-85a2b91521ae\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr"
Jul 28 18:36:23.681063 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:23.681018    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn87v\" (UniqueName: \"kubernetes.io/projected/29230a74-2fad-41f0-a99c-85a2b91521ae-kube-api-access-sn87v\") pod \"custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr\" (UID: \"29230a74-2fad-41f0-a99c-85a2b91521ae\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr"
Jul 28 18:36:23.681063 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:23.681069    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29230a74-2fad-41f0-a99c-85a2b91521ae-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr\" (UID: \"29230a74-2fad-41f0-a99c-85a2b91521ae\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr"
Jul 28 18:36:23.683735 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:23.683696    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29230a74-2fad-41f0-a99c-85a2b91521ae-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr\" (UID: \"29230a74-2fad-41f0-a99c-85a2b91521ae\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr"
Jul 28 18:36:23.689618 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:23.689588    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn87v\" (UniqueName: \"kubernetes.io/projected/29230a74-2fad-41f0-a99c-85a2b91521ae-kube-api-access-sn87v\") pod \"custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr\" (UID: \"29230a74-2fad-41f0-a99c-85a2b91521ae\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr"
Jul 28 18:36:23.789431 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:23.789387    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr"
Jul 28 18:36:23.940345 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:23.940306    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr"]
Jul 28 18:36:23.941867 ip-10-0-133-215 kubenswrapper[2567]: W0728 18:36:23.941832    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29230a74_2fad_41f0_a99c_85a2b91521ae.slice/crio-fe0aac89698fb921379986d5c1fe1b4dea9b8729cab8ec4e59367ab8fefdbe15 WatchSource:0}: Error finding container fe0aac89698fb921379986d5c1fe1b4dea9b8729cab8ec4e59367ab8fefdbe15: Status 404 returned error can't find the container with id fe0aac89698fb921379986d5c1fe1b4dea9b8729cab8ec4e59367ab8fefdbe15
Jul 28 18:36:24.807548 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:24.807490    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr" event={"ID":"29230a74-2fad-41f0-a99c-85a2b91521ae","Type":"ContainerStarted","Data":"b67271869f5c8558a739b432887a6f42927f7f3fbf2ea256728953d2fb494828"}
Jul 28 18:36:24.808049 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:24.807579    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr" event={"ID":"29230a74-2fad-41f0-a99c-85a2b91521ae","Type":"ContainerStarted","Data":"fe0aac89698fb921379986d5c1fe1b4dea9b8729cab8ec4e59367ab8fefdbe15"}
Jul 28 18:36:24.808049 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:24.807635    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr"
Jul 28 18:36:24.826278 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:24.826192    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr" podStartSLOduration=1.826171851 podStartE2EDuration="1.826171851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:36:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:36:24.823618012 +0000 UTC m=+5140.069720229" watchObservedRunningTime="2026-07-28 18:36:24.826171851 +0000 UTC m=+5140.072274059"
Jul 28 18:36:55.814571 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:36:55.814534    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr"
Jul 28 18:38:27.813576 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:27.813529    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr"]
Jul 28 18:38:35.814495 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:35.814456    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr" podUID="29230a74-2fad-41f0-a99c-85a2b91521ae" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 18:38:42.833250 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:42.833181    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr" podUID="29230a74-2fad-41f0-a99c-85a2b91521ae" containerName="main" containerID="cri-o://b67271869f5c8558a739b432887a6f42927f7f3fbf2ea256728953d2fb494828" gracePeriod=45
Jul 28 18:38:43.083097 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:43.083063    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr"
Jul 28 18:38:43.118199 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:43.118105    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sn87v\" (UniqueName: \"kubernetes.io/projected/29230a74-2fad-41f0-a99c-85a2b91521ae-kube-api-access-sn87v\") pod \"29230a74-2fad-41f0-a99c-85a2b91521ae\" (UID: \"29230a74-2fad-41f0-a99c-85a2b91521ae\") "
Jul 28 18:38:43.118361 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:43.118222    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29230a74-2fad-41f0-a99c-85a2b91521ae-tls-certs\") pod \"29230a74-2fad-41f0-a99c-85a2b91521ae\" (UID: \"29230a74-2fad-41f0-a99c-85a2b91521ae\") "
Jul 28 18:38:43.120435 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:43.120403    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29230a74-2fad-41f0-a99c-85a2b91521ae-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "29230a74-2fad-41f0-a99c-85a2b91521ae" (UID: "29230a74-2fad-41f0-a99c-85a2b91521ae"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:38:43.120579 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:43.120422    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29230a74-2fad-41f0-a99c-85a2b91521ae-kube-api-access-sn87v" (OuterVolumeSpecName: "kube-api-access-sn87v") pod "29230a74-2fad-41f0-a99c-85a2b91521ae" (UID: "29230a74-2fad-41f0-a99c-85a2b91521ae"). InnerVolumeSpecName "kube-api-access-sn87v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:38:43.218986 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:43.218933    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29230a74-2fad-41f0-a99c-85a2b91521ae-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:38:43.218986 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:43.218979    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sn87v\" (UniqueName: \"kubernetes.io/projected/29230a74-2fad-41f0-a99c-85a2b91521ae-kube-api-access-sn87v\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:38:43.467062 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:43.466961    2567 generic.go:356] "Generic (PLEG): container finished" podID="29230a74-2fad-41f0-a99c-85a2b91521ae" containerID="b67271869f5c8558a739b432887a6f42927f7f3fbf2ea256728953d2fb494828" exitCode=0
Jul 28 18:38:43.467062 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:43.467010    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr"
Jul 28 18:38:43.467062 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:43.467040    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr" event={"ID":"29230a74-2fad-41f0-a99c-85a2b91521ae","Type":"ContainerDied","Data":"b67271869f5c8558a739b432887a6f42927f7f3fbf2ea256728953d2fb494828"}
Jul 28 18:38:43.467360 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:43.467094    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr" event={"ID":"29230a74-2fad-41f0-a99c-85a2b91521ae","Type":"ContainerDied","Data":"fe0aac89698fb921379986d5c1fe1b4dea9b8729cab8ec4e59367ab8fefdbe15"}
Jul 28 18:38:43.467360 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:43.467115    2567 scope.go:117] "RemoveContainer" containerID="b67271869f5c8558a739b432887a6f42927f7f3fbf2ea256728953d2fb494828"
Jul 28 18:38:43.475931 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:43.475909    2567 scope.go:117] "RemoveContainer" containerID="b67271869f5c8558a739b432887a6f42927f7f3fbf2ea256728953d2fb494828"
Jul 28 18:38:43.476254 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:38:43.476224    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b67271869f5c8558a739b432887a6f42927f7f3fbf2ea256728953d2fb494828\": container with ID starting with b67271869f5c8558a739b432887a6f42927f7f3fbf2ea256728953d2fb494828 not found: ID does not exist" containerID="b67271869f5c8558a739b432887a6f42927f7f3fbf2ea256728953d2fb494828"
Jul 28 18:38:43.476310 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:43.476267    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b67271869f5c8558a739b432887a6f42927f7f3fbf2ea256728953d2fb494828"} err="failed to get container status \"b67271869f5c8558a739b432887a6f42927f7f3fbf2ea256728953d2fb494828\": rpc error: code = NotFound desc = could not find container \"b67271869f5c8558a739b432887a6f42927f7f3fbf2ea256728953d2fb494828\": container with ID starting with b67271869f5c8558a739b432887a6f42927f7f3fbf2ea256728953d2fb494828 not found: ID does not exist"
Jul 28 18:38:43.485331 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:43.485295    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr"]
Jul 28 18:38:43.488914 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:43.488883    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb6689qghhr"]
Jul 28 18:38:45.287725 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:45.287694    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29230a74-2fad-41f0-a99c-85a2b91521ae" path="/var/lib/kubelet/pods/29230a74-2fad-41f0-a99c-85a2b91521ae/volumes"
Jul 28 18:38:48.028128 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.028089    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"]
Jul 28 18:38:48.028613 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.028597    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29230a74-2fad-41f0-a99c-85a2b91521ae" containerName="main"
Jul 28 18:38:48.028710 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.028617    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="29230a74-2fad-41f0-a99c-85a2b91521ae" containerName="main"
Jul 28 18:38:48.028768 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.028709    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="29230a74-2fad-41f0-a99c-85a2b91521ae" containerName="main"
Jul 28 18:38:48.033157 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.032961    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.036422 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.036382    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"kube-root-ca.crt\""
Jul 28 18:38:48.036604 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.036452    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"router-gateway-1-openshift-default-dockercfg-nsbsq\""
Jul 28 18:38:48.036604 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.036549    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"openshift-gw-ca-root-cert\""
Jul 28 18:38:48.036939 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.036912    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"openshift-service-ca.crt\""
Jul 28 18:38:48.046372 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.046340    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"]
Jul 28 18:38:48.064486 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.064447    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-token\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.064486 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.064489    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-podinfo\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.064767 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.064552    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-data\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.064767 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.064618    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-credential-socket\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.064767 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.064634    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.064767 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.064652    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w774j\" (UniqueName: \"kubernetes.io/projected/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-kube-api-access-w774j\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.064767 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.064744    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-workload-socket\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.065000 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.064842    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-envoy\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.065000 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.064876    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-workload-certs\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.165635 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.165597    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-podinfo\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.165853 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.165649    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-data\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.165853 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.165774    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-credential-socket\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.165853 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.165818    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.165853 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.165848    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w774j\" (UniqueName: \"kubernetes.io/projected/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-kube-api-access-w774j\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.166081 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.165893    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-workload-socket\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.166081 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.165982    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-envoy\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.166081 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.166016    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-workload-certs\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.166081 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.166047    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-token\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.166081 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.166063    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-data\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.166392 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.166120    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-credential-socket\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.166444 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.166396    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-workload-certs\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.166487 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.166474    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-workload-socket\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.166648 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.166618    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.168260 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.168237    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-envoy\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.168423 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.168406    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-podinfo\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.179217 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.179178    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-token\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.180361 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.180287    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w774j\" (UniqueName: \"kubernetes.io/projected/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-kube-api-access-w774j\") pod \"router-gateway-1-openshift-default-69c8b44b88-jfnsz\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.346603 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.346481    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:48.493017 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.492939    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"]
Jul 28 18:38:48.495300 ip-10-0-133-215 kubenswrapper[2567]: W0728 18:38:48.495270    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbd79d05_2637_4e9a_b8ae_5187a50a4c2a.slice/crio-57d9f243ad6b8d680c7903b4670efe9f9bdf43f422924e802e7c25824fad79c4 WatchSource:0}: Error finding container 57d9f243ad6b8d680c7903b4670efe9f9bdf43f422924e802e7c25824fad79c4: Status 404 returned error can't find the container with id 57d9f243ad6b8d680c7903b4670efe9f9bdf43f422924e802e7c25824fad79c4
Jul 28 18:38:48.497270 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.497241    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 18:38:48.497333 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.497310    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 18:38:48.497371 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:48.497341    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 18:38:49.498624 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:49.498581    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz" event={"ID":"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a","Type":"ContainerStarted","Data":"668d3d72f5e4fb0f173cc5d3d026b34394989f875ab9c2f778565510caa4e2dd"}
Jul 28 18:38:49.499051 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:49.498646    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz" event={"ID":"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a","Type":"ContainerStarted","Data":"57d9f243ad6b8d680c7903b4670efe9f9bdf43f422924e802e7c25824fad79c4"}
Jul 28 18:38:49.532618 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:49.532557    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz" podStartSLOduration=1.532533927 podStartE2EDuration="1.532533927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:38:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:38:49.530470289 +0000 UTC m=+5284.776572497" watchObservedRunningTime="2026-07-28 18:38:49.532533927 +0000 UTC m=+5284.778636136"
Jul 28 18:38:50.347076 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:50.347033    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:50.352368 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:50.352335    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:50.503647 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:50.503614    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:38:50.504620 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:38:50.504598    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:39:02.565704 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:02.565668    2567 generic.go:356] "Generic (PLEG): container finished" podID="cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" containerID="668d3d72f5e4fb0f173cc5d3d026b34394989f875ab9c2f778565510caa4e2dd" exitCode=0
Jul 28 18:39:02.566155 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:02.565744    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz" event={"ID":"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a","Type":"ContainerDied","Data":"668d3d72f5e4fb0f173cc5d3d026b34394989f875ab9c2f778565510caa4e2dd"}
Jul 28 18:39:02.566155 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:02.566119    2567 scope.go:117] "RemoveContainer" containerID="668d3d72f5e4fb0f173cc5d3d026b34394989f875ab9c2f778565510caa4e2dd"
Jul 28 18:39:02.567058 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:02.567029    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 18:39:02.567128 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:02.567101    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 18:39:02.567169 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:02.567138    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 18:39:03.573034 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:03.572993    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz" event={"ID":"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a","Type":"ContainerStarted","Data":"6377262859a73223763a08efcd4d61cdda9e30a6d23ec64dc0b150f10afe1e72"}
Jul 28 18:39:04.347767 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:04.347724    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:39:04.348962 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:04.348929    2567 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz" podUID="cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 28 18:39:05.347182 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:05.347138    2567 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz" podUID="cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 28 18:39:05.504450 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:05.504400    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:39:06.347712 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:06.347659    2567 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz" podUID="cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 28 18:39:07.347304 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:07.347257    2567 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz" podUID="cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 28 18:39:08.347066 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:08.347014    2567 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz" podUID="cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 28 18:39:09.351835 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:09.351800    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:39:09.352826 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:09.352805    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:39:13.390705 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:13.390658    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj"]
Jul 28 18:39:13.393529 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:13.393480    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj"
Jul 28 18:39:13.397886 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:13.397858    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"default-dockercfg-sbvs9\""
Jul 28 18:39:13.398076 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:13.397859    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"router-with-refs-test-epp-sa-dockercfg-27zxg\""
Jul 28 18:39:13.398076 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:13.397906    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"router-with-refs-test-kserve-self-signed-certs\""
Jul 28 18:39:13.404596 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:13.404566    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj"]
Jul 28 18:39:13.494869 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:13.494821    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf3a044c-6629-4da9-9f04-c8593f8a8673-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj\" (UID: \"cf3a044c-6629-4da9-9f04-c8593f8a8673\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj"
Jul 28 18:39:13.495068 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:13.494884    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m57dd\" (UniqueName: \"kubernetes.io/projected/cf3a044c-6629-4da9-9f04-c8593f8a8673-kube-api-access-m57dd\") pod \"router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj\" (UID: \"cf3a044c-6629-4da9-9f04-c8593f8a8673\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj"
Jul 28 18:39:13.596139 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:13.596083    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m57dd\" (UniqueName: \"kubernetes.io/projected/cf3a044c-6629-4da9-9f04-c8593f8a8673-kube-api-access-m57dd\") pod \"router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj\" (UID: \"cf3a044c-6629-4da9-9f04-c8593f8a8673\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj"
Jul 28 18:39:13.596341 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:13.596183    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf3a044c-6629-4da9-9f04-c8593f8a8673-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj\" (UID: \"cf3a044c-6629-4da9-9f04-c8593f8a8673\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj"
Jul 28 18:39:13.599158 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:13.599122    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf3a044c-6629-4da9-9f04-c8593f8a8673-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj\" (UID: \"cf3a044c-6629-4da9-9f04-c8593f8a8673\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj"
Jul 28 18:39:13.606197 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:13.606158    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m57dd\" (UniqueName: \"kubernetes.io/projected/cf3a044c-6629-4da9-9f04-c8593f8a8673-kube-api-access-m57dd\") pod \"router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj\" (UID: \"cf3a044c-6629-4da9-9f04-c8593f8a8673\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj"
Jul 28 18:39:13.707277 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:13.707178    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj"
Jul 28 18:39:13.858958 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:13.858928    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj"]
Jul 28 18:39:13.860659 ip-10-0-133-215 kubenswrapper[2567]: W0728 18:39:13.860628    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf3a044c_6629_4da9_9f04_c8593f8a8673.slice/crio-844eb96ba5b5d85038a501179ef71f3281f44f734e403faf79fd1388549a9473 WatchSource:0}: Error finding container 844eb96ba5b5d85038a501179ef71f3281f44f734e403faf79fd1388549a9473: Status 404 returned error can't find the container with id 844eb96ba5b5d85038a501179ef71f3281f44f734e403faf79fd1388549a9473
Jul 28 18:39:14.640957 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:14.640906    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj" event={"ID":"cf3a044c-6629-4da9-9f04-c8593f8a8673","Type":"ContainerStarted","Data":"7137f8dfbffc1618b9cc185e8545b968d79494666d0374cb1f45628773d69d05"}
Jul 28 18:39:14.641478 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:14.640978    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj" event={"ID":"cf3a044c-6629-4da9-9f04-c8593f8a8673","Type":"ContainerStarted","Data":"844eb96ba5b5d85038a501179ef71f3281f44f734e403faf79fd1388549a9473"}
Jul 28 18:39:14.641478 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:14.641076    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj"
Jul 28 18:39:14.659425 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:14.659366    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj" podStartSLOduration=1.659348394 podStartE2EDuration="1.659348394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:39:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:39:14.658408858 +0000 UTC m=+5309.904511102" watchObservedRunningTime="2026-07-28 18:39:14.659348394 +0000 UTC m=+5309.905450600"
Jul 28 18:39:45.647618 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:39:45.647585    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj"
Jul 28 18:40:46.449494 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:40:46.449461    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:40:46.474462 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:40:46.474429    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:45:31.831371 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:31.831277    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj"]
Jul 28 18:45:35.648101 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:35.648055    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj" podUID="cf3a044c-6629-4da9-9f04-c8593f8a8673" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 18:45:45.647730 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:45.647687    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj" podUID="cf3a044c-6629-4da9-9f04-c8593f8a8673" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 18:45:46.479392 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:46.479364    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:45:46.504099 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:46.504069    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:45:46.851118 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:46.851056    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj" podUID="cf3a044c-6629-4da9-9f04-c8593f8a8673" containerName="main" containerID="cri-o://7137f8dfbffc1618b9cc185e8545b968d79494666d0374cb1f45628773d69d05" gracePeriod=45
Jul 28 18:45:47.095223 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:47.095195    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj"
Jul 28 18:45:47.252636 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:47.252539    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf3a044c-6629-4da9-9f04-c8593f8a8673-tls-certs\") pod \"cf3a044c-6629-4da9-9f04-c8593f8a8673\" (UID: \"cf3a044c-6629-4da9-9f04-c8593f8a8673\") "
Jul 28 18:45:47.252636 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:47.252610    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m57dd\" (UniqueName: \"kubernetes.io/projected/cf3a044c-6629-4da9-9f04-c8593f8a8673-kube-api-access-m57dd\") pod \"cf3a044c-6629-4da9-9f04-c8593f8a8673\" (UID: \"cf3a044c-6629-4da9-9f04-c8593f8a8673\") "
Jul 28 18:45:47.254778 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:47.254744    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf3a044c-6629-4da9-9f04-c8593f8a8673-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cf3a044c-6629-4da9-9f04-c8593f8a8673" (UID: "cf3a044c-6629-4da9-9f04-c8593f8a8673"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:45:47.254893 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:47.254787    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf3a044c-6629-4da9-9f04-c8593f8a8673-kube-api-access-m57dd" (OuterVolumeSpecName: "kube-api-access-m57dd") pod "cf3a044c-6629-4da9-9f04-c8593f8a8673" (UID: "cf3a044c-6629-4da9-9f04-c8593f8a8673"). InnerVolumeSpecName "kube-api-access-m57dd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:45:47.353171 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:47.353129    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf3a044c-6629-4da9-9f04-c8593f8a8673-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:45:47.353171 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:47.353165    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m57dd\" (UniqueName: \"kubernetes.io/projected/cf3a044c-6629-4da9-9f04-c8593f8a8673-kube-api-access-m57dd\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:45:47.557473 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:47.557428    2567 generic.go:356] "Generic (PLEG): container finished" podID="cf3a044c-6629-4da9-9f04-c8593f8a8673" containerID="7137f8dfbffc1618b9cc185e8545b968d79494666d0374cb1f45628773d69d05" exitCode=0
Jul 28 18:45:47.557673 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:47.557478    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj"
Jul 28 18:45:47.557673 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:47.557487    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj" event={"ID":"cf3a044c-6629-4da9-9f04-c8593f8a8673","Type":"ContainerDied","Data":"7137f8dfbffc1618b9cc185e8545b968d79494666d0374cb1f45628773d69d05"}
Jul 28 18:45:47.557673 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:47.557560    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj" event={"ID":"cf3a044c-6629-4da9-9f04-c8593f8a8673","Type":"ContainerDied","Data":"844eb96ba5b5d85038a501179ef71f3281f44f734e403faf79fd1388549a9473"}
Jul 28 18:45:47.557673 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:47.557588    2567 scope.go:117] "RemoveContainer" containerID="7137f8dfbffc1618b9cc185e8545b968d79494666d0374cb1f45628773d69d05"
Jul 28 18:45:47.566179 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:47.566155    2567 scope.go:117] "RemoveContainer" containerID="7137f8dfbffc1618b9cc185e8545b968d79494666d0374cb1f45628773d69d05"
Jul 28 18:45:47.566487 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:45:47.566464    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7137f8dfbffc1618b9cc185e8545b968d79494666d0374cb1f45628773d69d05\": container with ID starting with 7137f8dfbffc1618b9cc185e8545b968d79494666d0374cb1f45628773d69d05 not found: ID does not exist" containerID="7137f8dfbffc1618b9cc185e8545b968d79494666d0374cb1f45628773d69d05"
Jul 28 18:45:47.566590 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:47.566498    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7137f8dfbffc1618b9cc185e8545b968d79494666d0374cb1f45628773d69d05"} err="failed to get container status \"7137f8dfbffc1618b9cc185e8545b968d79494666d0374cb1f45628773d69d05\": rpc error: code = NotFound desc = could not find container \"7137f8dfbffc1618b9cc185e8545b968d79494666d0374cb1f45628773d69d05\": container with ID starting with 7137f8dfbffc1618b9cc185e8545b968d79494666d0374cb1f45628773d69d05 not found: ID does not exist"
Jul 28 18:45:47.577420 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:47.577380    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj"]
Jul 28 18:45:47.581418 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:47.581393    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-d4zrj"]
Jul 28 18:45:49.286884 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:49.286842    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf3a044c-6629-4da9-9f04-c8593f8a8673" path="/var/lib/kubelet/pods/cf3a044c-6629-4da9-9f04-c8593f8a8673/volumes"
Jul 28 18:45:56.770412 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:56.770367    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"]
Jul 28 18:45:56.770937 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:45:56.770634    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz" podUID="cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" containerName="istio-proxy" containerID="cri-o://6377262859a73223763a08efcd4d61cdda9e30a6d23ec64dc0b150f10afe1e72" gracePeriod=30
Jul 28 18:46:02.028411 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.028382    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:46:02.074873 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.074779    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-workload-certs\") pod \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") "
Jul 28 18:46:02.074873 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.074831    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istiod-ca-cert\") pod \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") "
Jul 28 18:46:02.074873 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.074862    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-token\") pod \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") "
Jul 28 18:46:02.075233 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.074878    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-credential-socket\") pod \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") "
Jul 28 18:46:02.075233 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.074895    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-workload-socket\") pod \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") "
Jul 28 18:46:02.075233 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.074938    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-envoy\") pod \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") "
Jul 28 18:46:02.075233 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.074976    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-podinfo\") pod \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") "
Jul 28 18:46:02.075233 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.075053    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" (UID: "cbd79d05-2637-4e9a-b8ae-5187a50a4c2a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:46:02.075233 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.075194    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-workload-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:46:02.075584 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.075392    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" (UID: "cbd79d05-2637-4e9a-b8ae-5187a50a4c2a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:46:02.075584 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.075461    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" (UID: "cbd79d05-2637-4e9a-b8ae-5187a50a4c2a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:46:02.075584 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.075465    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" (UID: "cbd79d05-2637-4e9a-b8ae-5187a50a4c2a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:46:02.077181 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.077144    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-token" (OuterVolumeSpecName: "istio-token") pod "cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" (UID: "cbd79d05-2637-4e9a-b8ae-5187a50a4c2a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:46:02.077298 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.077231    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" (UID: "cbd79d05-2637-4e9a-b8ae-5187a50a4c2a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 18:46:02.077460 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.077427    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" (UID: "cbd79d05-2637-4e9a-b8ae-5187a50a4c2a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:46:02.176225 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.176184    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w774j\" (UniqueName: \"kubernetes.io/projected/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-kube-api-access-w774j\") pod \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") "
Jul 28 18:46:02.176410 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.176235    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-data\") pod \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\" (UID: \"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a\") "
Jul 28 18:46:02.176493 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.176476    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istiod-ca-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:46:02.176553 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.176499    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-token\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:46:02.176553 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.176527    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-credential-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:46:02.176553 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.176530    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-data" (OuterVolumeSpecName: "istio-data") pod "cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" (UID: "cbd79d05-2637-4e9a-b8ae-5187a50a4c2a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:46:02.176652 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.176541    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-workload-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:46:02.176652 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.176574    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-envoy\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:46:02.176652 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.176586    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-podinfo\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:46:02.178405 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.178379    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-kube-api-access-w774j" (OuterVolumeSpecName: "kube-api-access-w774j") pod "cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" (UID: "cbd79d05-2637-4e9a-b8ae-5187a50a4c2a"). InnerVolumeSpecName "kube-api-access-w774j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:46:02.277570 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.277530    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-istio-data\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:46:02.277570 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.277564    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w774j\" (UniqueName: \"kubernetes.io/projected/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a-kube-api-access-w774j\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:46:02.630808 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.630772    2567 generic.go:356] "Generic (PLEG): container finished" podID="cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" containerID="6377262859a73223763a08efcd4d61cdda9e30a6d23ec64dc0b150f10afe1e72" exitCode=0
Jul 28 18:46:02.631120 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.630820    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"
Jul 28 18:46:02.631120 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.630861    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz" event={"ID":"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a","Type":"ContainerDied","Data":"6377262859a73223763a08efcd4d61cdda9e30a6d23ec64dc0b150f10afe1e72"}
Jul 28 18:46:02.631120 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.630928    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz" event={"ID":"cbd79d05-2637-4e9a-b8ae-5187a50a4c2a","Type":"ContainerDied","Data":"57d9f243ad6b8d680c7903b4670efe9f9bdf43f422924e802e7c25824fad79c4"}
Jul 28 18:46:02.631120 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.630962    2567 scope.go:117] "RemoveContainer" containerID="6377262859a73223763a08efcd4d61cdda9e30a6d23ec64dc0b150f10afe1e72"
Jul 28 18:46:02.640120 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.640094    2567 scope.go:117] "RemoveContainer" containerID="668d3d72f5e4fb0f173cc5d3d026b34394989f875ab9c2f778565510caa4e2dd"
Jul 28 18:46:02.648481 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.648460    2567 scope.go:117] "RemoveContainer" containerID="6377262859a73223763a08efcd4d61cdda9e30a6d23ec64dc0b150f10afe1e72"
Jul 28 18:46:02.648817 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:46:02.648795    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6377262859a73223763a08efcd4d61cdda9e30a6d23ec64dc0b150f10afe1e72\": container with ID starting with 6377262859a73223763a08efcd4d61cdda9e30a6d23ec64dc0b150f10afe1e72 not found: ID does not exist" containerID="6377262859a73223763a08efcd4d61cdda9e30a6d23ec64dc0b150f10afe1e72"
Jul 28 18:46:02.648864 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.648830    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6377262859a73223763a08efcd4d61cdda9e30a6d23ec64dc0b150f10afe1e72"} err="failed to get container status \"6377262859a73223763a08efcd4d61cdda9e30a6d23ec64dc0b150f10afe1e72\": rpc error: code = NotFound desc = could not find container \"6377262859a73223763a08efcd4d61cdda9e30a6d23ec64dc0b150f10afe1e72\": container with ID starting with 6377262859a73223763a08efcd4d61cdda9e30a6d23ec64dc0b150f10afe1e72 not found: ID does not exist"
Jul 28 18:46:02.648864 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.648850    2567 scope.go:117] "RemoveContainer" containerID="668d3d72f5e4fb0f173cc5d3d026b34394989f875ab9c2f778565510caa4e2dd"
Jul 28 18:46:02.649124 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:46:02.649107    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"668d3d72f5e4fb0f173cc5d3d026b34394989f875ab9c2f778565510caa4e2dd\": container with ID starting with 668d3d72f5e4fb0f173cc5d3d026b34394989f875ab9c2f778565510caa4e2dd not found: ID does not exist" containerID="668d3d72f5e4fb0f173cc5d3d026b34394989f875ab9c2f778565510caa4e2dd"
Jul 28 18:46:02.649162 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.649130    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"668d3d72f5e4fb0f173cc5d3d026b34394989f875ab9c2f778565510caa4e2dd"} err="failed to get container status \"668d3d72f5e4fb0f173cc5d3d026b34394989f875ab9c2f778565510caa4e2dd\": rpc error: code = NotFound desc = could not find container \"668d3d72f5e4fb0f173cc5d3d026b34394989f875ab9c2f778565510caa4e2dd\": container with ID starting with 668d3d72f5e4fb0f173cc5d3d026b34394989f875ab9c2f778565510caa4e2dd not found: ID does not exist"
Jul 28 18:46:02.653652 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.653618    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"]
Jul 28 18:46:02.657229 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:02.657198    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-69c8b44b88-jfnsz"]
Jul 28 18:46:03.286967 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:03.286932    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" path="/var/lib/kubelet/pods/cbd79d05-2637-4e9a-b8ae-5187a50a4c2a/volumes"
Jul 28 18:46:09.389252 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.389209    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7"]
Jul 28 18:46:09.389779 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.389517    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf3a044c-6629-4da9-9f04-c8593f8a8673" containerName="main"
Jul 28 18:46:09.389779 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.389531    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf3a044c-6629-4da9-9f04-c8593f8a8673" containerName="main"
Jul 28 18:46:09.389779 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.389546    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" containerName="istio-proxy"
Jul 28 18:46:09.389779 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.389552    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" containerName="istio-proxy"
Jul 28 18:46:09.389779 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.389560    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" containerName="istio-proxy"
Jul 28 18:46:09.389779 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.389566    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" containerName="istio-proxy"
Jul 28 18:46:09.389779 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.389621    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" containerName="istio-proxy"
Jul 28 18:46:09.389779 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.389628    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf3a044c-6629-4da9-9f04-c8593f8a8673" containerName="main"
Jul 28 18:46:09.389779 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.389635    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbd79d05-2637-4e9a-b8ae-5187a50a4c2a" containerName="istio-proxy"
Jul 28 18:46:09.393104 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.393078    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7"
Jul 28 18:46:09.397753 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.397712    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"default-dockercfg-hhpzw\""
Jul 28 18:46:09.397753 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.397734    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"openshift-service-ca.crt\""
Jul 28 18:46:09.397753 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.397716    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs\""
Jul 28 18:46:09.398031 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.397712    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"kube-root-ca.crt\""
Jul 28 18:46:09.398031 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.397813    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"llmisvc-model-fb-opt-125m-route-50bc673d-epp-sa-dockercfg-qd5nd\""
Jul 28 18:46:09.402500 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.402471    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7"]
Jul 28 18:46:09.436345 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.436311    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5r58\" (UniqueName: \"kubernetes.io/projected/0219a1d5-d00e-47fb-8442-05a3c8767e14-kube-api-access-b5r58\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7\" (UID: \"0219a1d5-d00e-47fb-8442-05a3c8767e14\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7"
Jul 28 18:46:09.436573 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.436386    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0219a1d5-d00e-47fb-8442-05a3c8767e14-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7\" (UID: \"0219a1d5-d00e-47fb-8442-05a3c8767e14\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7"
Jul 28 18:46:09.537570 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.537493    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5r58\" (UniqueName: \"kubernetes.io/projected/0219a1d5-d00e-47fb-8442-05a3c8767e14-kube-api-access-b5r58\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7\" (UID: \"0219a1d5-d00e-47fb-8442-05a3c8767e14\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7"
Jul 28 18:46:09.537789 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.537600    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0219a1d5-d00e-47fb-8442-05a3c8767e14-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7\" (UID: \"0219a1d5-d00e-47fb-8442-05a3c8767e14\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7"
Jul 28 18:46:09.540078 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.540053    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0219a1d5-d00e-47fb-8442-05a3c8767e14-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7\" (UID: \"0219a1d5-d00e-47fb-8442-05a3c8767e14\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7"
Jul 28 18:46:09.546256 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.546226    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5r58\" (UniqueName: \"kubernetes.io/projected/0219a1d5-d00e-47fb-8442-05a3c8767e14-kube-api-access-b5r58\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7\" (UID: \"0219a1d5-d00e-47fb-8442-05a3c8767e14\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7"
Jul 28 18:46:09.706288 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.706178    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7"
Jul 28 18:46:09.841401 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:09.841372    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7"]
Jul 28 18:46:09.842892 ip-10-0-133-215 kubenswrapper[2567]: W0728 18:46:09.842861    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0219a1d5_d00e_47fb_8442_05a3c8767e14.slice/crio-709e68785698d9c9479fa98b4a66a227adfa2fe5c66e4b6dde354596fb9bc95b WatchSource:0}: Error finding container 709e68785698d9c9479fa98b4a66a227adfa2fe5c66e4b6dde354596fb9bc95b: Status 404 returned error can't find the container with id 709e68785698d9c9479fa98b4a66a227adfa2fe5c66e4b6dde354596fb9bc95b
Jul 28 18:46:10.670858 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:10.670814    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7" event={"ID":"0219a1d5-d00e-47fb-8442-05a3c8767e14","Type":"ContainerStarted","Data":"58e64f712b8b3a082fcd8543cde1381f7e4a0ea0c2178bd0b2791040a2f31723"}
Jul 28 18:46:10.671274 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:10.670913    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7" event={"ID":"0219a1d5-d00e-47fb-8442-05a3c8767e14","Type":"ContainerStarted","Data":"709e68785698d9c9479fa98b4a66a227adfa2fe5c66e4b6dde354596fb9bc95b"}
Jul 28 18:46:10.671274 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:10.670941    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7"
Jul 28 18:46:10.687999 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:10.687935    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7" podStartSLOduration=1.687915051 podStartE2EDuration="1.687915051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:46:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:46:10.687865437 +0000 UTC m=+5725.933967643" watchObservedRunningTime="2026-07-28 18:46:10.687915051 +0000 UTC m=+5725.934017257"
Jul 28 18:46:41.677526 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:46:41.677405    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7"
Jul 28 18:49:17.063212 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.063148    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"]
Jul 28 18:49:17.074132 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:49:17.074079    2567 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 28 18:49:17.074132 ip-10-0-133-215 kubenswrapper[2567]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 18:49:17.074132 ip-10-0-133-215 kubenswrapper[2567]:         , stdout: , stderr: , exit code -1
Jul 28 18:49:17.074132 ip-10-0-133-215 kubenswrapper[2567]:  > containerID="70b8ae96f88afe929ac975fbaa876424b2f29fea41bd91431a5582eb6aef1de6" cmd=["/bin/sleep","15"]
Jul 28 18:49:17.074435 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.074153    2567 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1" message="" err=<
Jul 28 18:49:17.074435 ip-10-0-133-215 kubenswrapper[2567]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 18:49:17.074435 ip-10-0-133-215 kubenswrapper[2567]:         , stdout: , stderr: , exit code -1
Jul 28 18:49:17.074435 ip-10-0-133-215 kubenswrapper[2567]:  >
Jul 28 18:49:17.074435 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:49:17.074166    2567 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 28 18:49:17.074435 ip-10-0-133-215 kubenswrapper[2567]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 18:49:17.074435 ip-10-0-133-215 kubenswrapper[2567]:         , stdout: , stderr: , exit code -1
Jul 28 18:49:17.074435 ip-10-0-133-215 kubenswrapper[2567]:  > pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1" podUID="3e87911f-78f4-4c0c-8964-b653ced0dc3e" containerName="main" containerID="cri-o://70b8ae96f88afe929ac975fbaa876424b2f29fea41bd91431a5582eb6aef1de6"
Jul 28 18:49:17.074435 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.074222    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1" podUID="3e87911f-78f4-4c0c-8964-b653ced0dc3e" containerName="main" containerID="cri-o://70b8ae96f88afe929ac975fbaa876424b2f29fea41bd91431a5582eb6aef1de6" gracePeriod=60
Jul 28 18:49:17.328002 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.327906    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:49:17.416768 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.416729    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmrgs\" (UniqueName: \"kubernetes.io/projected/3e87911f-78f4-4c0c-8964-b653ced0dc3e-kube-api-access-xmrgs\") pod \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") "
Jul 28 18:49:17.416996 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.416807    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-dshm\") pod \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") "
Jul 28 18:49:17.416996 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.416853    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-model-cache\") pod \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") "
Jul 28 18:49:17.416996 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.416885    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e87911f-78f4-4c0c-8964-b653ced0dc3e-tls-certs\") pod \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") "
Jul 28 18:49:17.416996 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.416913    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-home\") pod \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") "
Jul 28 18:49:17.416996 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.416946    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-tmp-dir\") pod \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\" (UID: \"3e87911f-78f4-4c0c-8964-b653ced0dc3e\") "
Jul 28 18:49:17.417288 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.417182    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-model-cache" (OuterVolumeSpecName: "model-cache") pod "3e87911f-78f4-4c0c-8964-b653ced0dc3e" (UID: "3e87911f-78f4-4c0c-8964-b653ced0dc3e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:49:17.417288 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.417216    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-home" (OuterVolumeSpecName: "home") pod "3e87911f-78f4-4c0c-8964-b653ced0dc3e" (UID: "3e87911f-78f4-4c0c-8964-b653ced0dc3e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:49:17.417465 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.417319    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3e87911f-78f4-4c0c-8964-b653ced0dc3e" (UID: "3e87911f-78f4-4c0c-8964-b653ced0dc3e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:49:17.419347 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.419312    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e87911f-78f4-4c0c-8964-b653ced0dc3e-kube-api-access-xmrgs" (OuterVolumeSpecName: "kube-api-access-xmrgs") pod "3e87911f-78f4-4c0c-8964-b653ced0dc3e" (UID: "3e87911f-78f4-4c0c-8964-b653ced0dc3e"). InnerVolumeSpecName "kube-api-access-xmrgs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:49:17.419347 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.419332    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-dshm" (OuterVolumeSpecName: "dshm") pod "3e87911f-78f4-4c0c-8964-b653ced0dc3e" (UID: "3e87911f-78f4-4c0c-8964-b653ced0dc3e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:49:17.419590 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.419365    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e87911f-78f4-4c0c-8964-b653ced0dc3e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3e87911f-78f4-4c0c-8964-b653ced0dc3e" (UID: "3e87911f-78f4-4c0c-8964-b653ced0dc3e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:49:17.517829 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.517787    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmrgs\" (UniqueName: \"kubernetes.io/projected/3e87911f-78f4-4c0c-8964-b653ced0dc3e-kube-api-access-xmrgs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:49:17.517829 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.517821    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-dshm\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:49:17.517829 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.517832    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-model-cache\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:49:17.518088 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.517843    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e87911f-78f4-4c0c-8964-b653ced0dc3e-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:49:17.518088 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.517851    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-home\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:49:17.518088 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.517860    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e87911f-78f4-4c0c-8964-b653ced0dc3e-tmp-dir\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:49:17.594137 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.594040    2567 generic.go:356] "Generic (PLEG): container finished" podID="3e87911f-78f4-4c0c-8964-b653ced0dc3e" containerID="70b8ae96f88afe929ac975fbaa876424b2f29fea41bd91431a5582eb6aef1de6" exitCode=0
Jul 28 18:49:17.594137 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.594091    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"
Jul 28 18:49:17.594137 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.594126    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1" event={"ID":"3e87911f-78f4-4c0c-8964-b653ced0dc3e","Type":"ContainerDied","Data":"70b8ae96f88afe929ac975fbaa876424b2f29fea41bd91431a5582eb6aef1de6"}
Jul 28 18:49:17.594360 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.594184    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1" event={"ID":"3e87911f-78f4-4c0c-8964-b653ced0dc3e","Type":"ContainerDied","Data":"96d405e1411371cf67382529b063e3acae0f0b295faf0073781b50d183197d98"}
Jul 28 18:49:17.594360 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.594207    2567 scope.go:117] "RemoveContainer" containerID="70b8ae96f88afe929ac975fbaa876424b2f29fea41bd91431a5582eb6aef1de6"
Jul 28 18:49:17.606908 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.606873    2567 scope.go:117] "RemoveContainer" containerID="70b8ae96f88afe929ac975fbaa876424b2f29fea41bd91431a5582eb6aef1de6"
Jul 28 18:49:17.607305 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:49:17.607283    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70b8ae96f88afe929ac975fbaa876424b2f29fea41bd91431a5582eb6aef1de6\": container with ID starting with 70b8ae96f88afe929ac975fbaa876424b2f29fea41bd91431a5582eb6aef1de6 not found: ID does not exist" containerID="70b8ae96f88afe929ac975fbaa876424b2f29fea41bd91431a5582eb6aef1de6"
Jul 28 18:49:17.607391 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.607319    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70b8ae96f88afe929ac975fbaa876424b2f29fea41bd91431a5582eb6aef1de6"} err="failed to get container status \"70b8ae96f88afe929ac975fbaa876424b2f29fea41bd91431a5582eb6aef1de6\": rpc error: code = NotFound desc = could not find container \"70b8ae96f88afe929ac975fbaa876424b2f29fea41bd91431a5582eb6aef1de6\": container with ID starting with 70b8ae96f88afe929ac975fbaa876424b2f29fea41bd91431a5582eb6aef1de6 not found: ID does not exist"
Jul 28 18:49:17.621314 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.621279    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"]
Jul 28 18:49:17.624843 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:17.624810    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0-1"]
Jul 28 18:49:19.288231 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:19.288189    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e87911f-78f4-4c0c-8964-b653ced0dc3e" path="/var/lib/kubelet/pods/3e87911f-78f4-4c0c-8964-b653ced0dc3e/volumes"
Jul 28 18:49:19.452656 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:19.452615    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7"]
Jul 28 18:49:21.677412 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:21.677371    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7" podUID="0219a1d5-d00e-47fb-8442-05a3c8767e14" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 18:49:31.677235 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:31.677195    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7" podUID="0219a1d5-d00e-47fb-8442-05a3c8767e14" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 18:49:34.471769 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:34.471724    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7" podUID="0219a1d5-d00e-47fb-8442-05a3c8767e14" containerName="main" containerID="cri-o://58e64f712b8b3a082fcd8543cde1381f7e4a0ea0c2178bd0b2791040a2f31723" gracePeriod=45
Jul 28 18:49:34.676926 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:34.676883    2567 generic.go:356] "Generic (PLEG): container finished" podID="0219a1d5-d00e-47fb-8442-05a3c8767e14" containerID="58e64f712b8b3a082fcd8543cde1381f7e4a0ea0c2178bd0b2791040a2f31723" exitCode=0
Jul 28 18:49:34.677098 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:34.676954    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7" event={"ID":"0219a1d5-d00e-47fb-8442-05a3c8767e14","Type":"ContainerDied","Data":"58e64f712b8b3a082fcd8543cde1381f7e4a0ea0c2178bd0b2791040a2f31723"}
Jul 28 18:49:34.723416 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:34.723328    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7"
Jul 28 18:49:34.860442 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:34.860387    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5r58\" (UniqueName: \"kubernetes.io/projected/0219a1d5-d00e-47fb-8442-05a3c8767e14-kube-api-access-b5r58\") pod \"0219a1d5-d00e-47fb-8442-05a3c8767e14\" (UID: \"0219a1d5-d00e-47fb-8442-05a3c8767e14\") "
Jul 28 18:49:34.860659 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:34.860491    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0219a1d5-d00e-47fb-8442-05a3c8767e14-tls-certs\") pod \"0219a1d5-d00e-47fb-8442-05a3c8767e14\" (UID: \"0219a1d5-d00e-47fb-8442-05a3c8767e14\") "
Jul 28 18:49:34.862802 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:34.862765    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0219a1d5-d00e-47fb-8442-05a3c8767e14-kube-api-access-b5r58" (OuterVolumeSpecName: "kube-api-access-b5r58") pod "0219a1d5-d00e-47fb-8442-05a3c8767e14" (UID: "0219a1d5-d00e-47fb-8442-05a3c8767e14"). InnerVolumeSpecName "kube-api-access-b5r58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:49:34.862944 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:34.862797    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0219a1d5-d00e-47fb-8442-05a3c8767e14-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0219a1d5-d00e-47fb-8442-05a3c8767e14" (UID: "0219a1d5-d00e-47fb-8442-05a3c8767e14"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:49:34.961336 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:34.961276    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0219a1d5-d00e-47fb-8442-05a3c8767e14-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:49:34.961336 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:34.961327    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5r58\" (UniqueName: \"kubernetes.io/projected/0219a1d5-d00e-47fb-8442-05a3c8767e14-kube-api-access-b5r58\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:49:35.683085 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:35.682981    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7" event={"ID":"0219a1d5-d00e-47fb-8442-05a3c8767e14","Type":"ContainerDied","Data":"709e68785698d9c9479fa98b4a66a227adfa2fe5c66e4b6dde354596fb9bc95b"}
Jul 28 18:49:35.683085 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:35.683005    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7"
Jul 28 18:49:35.683085 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:35.683055    2567 scope.go:117] "RemoveContainer" containerID="58e64f712b8b3a082fcd8543cde1381f7e4a0ea0c2178bd0b2791040a2f31723"
Jul 28 18:49:35.700519 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:35.700468    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7"]
Jul 28 18:49:35.708587 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:35.706145    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche8ghs7"]
Jul 28 18:49:37.286346 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:49:37.286303    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0219a1d5-d00e-47fb-8442-05a3c8767e14" path="/var/lib/kubelet/pods/0219a1d5-d00e-47fb-8442-05a3c8767e14/volumes"
Jul 28 18:50:04.979863 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:04.979822    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g"]
Jul 28 18:50:04.980472 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:04.980369    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0219a1d5-d00e-47fb-8442-05a3c8767e14" containerName="main"
Jul 28 18:50:04.980472 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:04.980390    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0219a1d5-d00e-47fb-8442-05a3c8767e14" containerName="main"
Jul 28 18:50:04.980472 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:04.980406    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e87911f-78f4-4c0c-8964-b653ced0dc3e" containerName="main"
Jul 28 18:50:04.980472 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:04.980413    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e87911f-78f4-4c0c-8964-b653ced0dc3e" containerName="main"
Jul 28 18:50:04.980751 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:04.980483    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0219a1d5-d00e-47fb-8442-05a3c8767e14" containerName="main"
Jul 28 18:50:04.980751 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:04.980495    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e87911f-78f4-4c0c-8964-b653ced0dc3e" containerName="main"
Jul 28 18:50:04.984782 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:04.984758    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g"
Jul 28 18:50:04.987657 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:04.987632    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"kube-root-ca.crt\""
Jul 28 18:50:04.988941 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:04.988912    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"default-dockercfg-zxjkm\""
Jul 28 18:50:04.989086 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:04.989041    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"openshift-service-ca.crt\""
Jul 28 18:50:04.989168 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:04.989135    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"custom-route-timeout-pd-test-kserve-self-signed-certs\""
Jul 28 18:50:04.989316 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:04.989293    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"custom-route-timeout-pd-test-epp-sa-dockercfg-t8b99\""
Jul 28 18:50:04.993299 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:04.993271    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g"]
Jul 28 18:50:05.001349 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:05.001302    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzt58\" (UniqueName: \"kubernetes.io/projected/eb6f69ae-bf47-4aa1-83ca-742b35e0e28c-kube-api-access-qzt58\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g\" (UID: \"eb6f69ae-bf47-4aa1-83ca-742b35e0e28c\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g"
Jul 28 18:50:05.001704 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:05.001674    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb6f69ae-bf47-4aa1-83ca-742b35e0e28c-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g\" (UID: \"eb6f69ae-bf47-4aa1-83ca-742b35e0e28c\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g"
Jul 28 18:50:05.102977 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:05.102935    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzt58\" (UniqueName: \"kubernetes.io/projected/eb6f69ae-bf47-4aa1-83ca-742b35e0e28c-kube-api-access-qzt58\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g\" (UID: \"eb6f69ae-bf47-4aa1-83ca-742b35e0e28c\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g"
Jul 28 18:50:05.102977 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:05.102978    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb6f69ae-bf47-4aa1-83ca-742b35e0e28c-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g\" (UID: \"eb6f69ae-bf47-4aa1-83ca-742b35e0e28c\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g"
Jul 28 18:50:05.105677 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:05.105644    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb6f69ae-bf47-4aa1-83ca-742b35e0e28c-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g\" (UID: \"eb6f69ae-bf47-4aa1-83ca-742b35e0e28c\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g"
Jul 28 18:50:05.113926 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:05.113888    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzt58\" (UniqueName: \"kubernetes.io/projected/eb6f69ae-bf47-4aa1-83ca-742b35e0e28c-kube-api-access-qzt58\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g\" (UID: \"eb6f69ae-bf47-4aa1-83ca-742b35e0e28c\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g"
Jul 28 18:50:05.296756 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:05.296716    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g"
Jul 28 18:50:05.444136 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:05.444097    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g"]
Jul 28 18:50:05.447457 ip-10-0-133-215 kubenswrapper[2567]: W0728 18:50:05.447426    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb6f69ae_bf47_4aa1_83ca_742b35e0e28c.slice/crio-ad972f5f963f8929f7e48df25b94c46762d95effcbc2f6eabe47840b45aca933 WatchSource:0}: Error finding container ad972f5f963f8929f7e48df25b94c46762d95effcbc2f6eabe47840b45aca933: Status 404 returned error can't find the container with id ad972f5f963f8929f7e48df25b94c46762d95effcbc2f6eabe47840b45aca933
Jul 28 18:50:05.836105 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:05.836055    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g" event={"ID":"eb6f69ae-bf47-4aa1-83ca-742b35e0e28c","Type":"ContainerStarted","Data":"ff4b69c2aed7428afd9146209381a851e6a109100bae736f2c01482d37472c35"}
Jul 28 18:50:05.836319 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:05.836155    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g" event={"ID":"eb6f69ae-bf47-4aa1-83ca-742b35e0e28c","Type":"ContainerStarted","Data":"ad972f5f963f8929f7e48df25b94c46762d95effcbc2f6eabe47840b45aca933"}
Jul 28 18:50:05.836319 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:05.836185    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g"
Jul 28 18:50:05.854817 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:05.854744    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g" podStartSLOduration=1.854723049 podStartE2EDuration="1.854723049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:50:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:50:05.853657688 +0000 UTC m=+5961.099759898" watchObservedRunningTime="2026-07-28 18:50:05.854723049 +0000 UTC m=+5961.100825259"
Jul 28 18:50:36.842796 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:36.842760    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g"
Jul 28 18:50:46.506918 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:46.506889    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:50:46.532661 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:50:46.532633    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:52:43.337701 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:52:43.337607    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g"]
Jul 28 18:52:46.842226 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:52:46.842182    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g" podUID="eb6f69ae-bf47-4aa1-83ca-742b35e0e28c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 18:52:56.842386 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:52:56.842345    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g" podUID="eb6f69ae-bf47-4aa1-83ca-742b35e0e28c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 28 18:52:58.357318 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:52:58.357273    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g" podUID="eb6f69ae-bf47-4aa1-83ca-742b35e0e28c" containerName="main" containerID="cri-o://ff4b69c2aed7428afd9146209381a851e6a109100bae736f2c01482d37472c35" gracePeriod=45
Jul 28 18:52:58.616659 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:52:58.616581    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g"
Jul 28 18:52:58.657432 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:52:58.657390    2567 generic.go:356] "Generic (PLEG): container finished" podID="eb6f69ae-bf47-4aa1-83ca-742b35e0e28c" containerID="ff4b69c2aed7428afd9146209381a851e6a109100bae736f2c01482d37472c35" exitCode=0
Jul 28 18:52:58.657644 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:52:58.657451    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g"
Jul 28 18:52:58.657644 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:52:58.657451    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g" event={"ID":"eb6f69ae-bf47-4aa1-83ca-742b35e0e28c","Type":"ContainerDied","Data":"ff4b69c2aed7428afd9146209381a851e6a109100bae736f2c01482d37472c35"}
Jul 28 18:52:58.657644 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:52:58.657604    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g" event={"ID":"eb6f69ae-bf47-4aa1-83ca-742b35e0e28c","Type":"ContainerDied","Data":"ad972f5f963f8929f7e48df25b94c46762d95effcbc2f6eabe47840b45aca933"}
Jul 28 18:52:58.657644 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:52:58.657632    2567 scope.go:117] "RemoveContainer" containerID="ff4b69c2aed7428afd9146209381a851e6a109100bae736f2c01482d37472c35"
Jul 28 18:52:58.666423 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:52:58.666399    2567 scope.go:117] "RemoveContainer" containerID="ff4b69c2aed7428afd9146209381a851e6a109100bae736f2c01482d37472c35"
Jul 28 18:52:58.666749 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:52:58.666721    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff4b69c2aed7428afd9146209381a851e6a109100bae736f2c01482d37472c35\": container with ID starting with ff4b69c2aed7428afd9146209381a851e6a109100bae736f2c01482d37472c35 not found: ID does not exist" containerID="ff4b69c2aed7428afd9146209381a851e6a109100bae736f2c01482d37472c35"
Jul 28 18:52:58.666875 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:52:58.666758    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff4b69c2aed7428afd9146209381a851e6a109100bae736f2c01482d37472c35"} err="failed to get container status \"ff4b69c2aed7428afd9146209381a851e6a109100bae736f2c01482d37472c35\": rpc error: code = NotFound desc = could not find container \"ff4b69c2aed7428afd9146209381a851e6a109100bae736f2c01482d37472c35\": container with ID starting with ff4b69c2aed7428afd9146209381a851e6a109100bae736f2c01482d37472c35 not found: ID does not exist"
Jul 28 18:52:58.729147 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:52:58.729108    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb6f69ae-bf47-4aa1-83ca-742b35e0e28c-tls-certs\") pod \"eb6f69ae-bf47-4aa1-83ca-742b35e0e28c\" (UID: \"eb6f69ae-bf47-4aa1-83ca-742b35e0e28c\") "
Jul 28 18:52:58.729355 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:52:58.729173    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzt58\" (UniqueName: \"kubernetes.io/projected/eb6f69ae-bf47-4aa1-83ca-742b35e0e28c-kube-api-access-qzt58\") pod \"eb6f69ae-bf47-4aa1-83ca-742b35e0e28c\" (UID: \"eb6f69ae-bf47-4aa1-83ca-742b35e0e28c\") "
Jul 28 18:52:58.731540 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:52:58.731476    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb6f69ae-bf47-4aa1-83ca-742b35e0e28c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "eb6f69ae-bf47-4aa1-83ca-742b35e0e28c" (UID: "eb6f69ae-bf47-4aa1-83ca-742b35e0e28c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:52:58.731540 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:52:58.731521    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb6f69ae-bf47-4aa1-83ca-742b35e0e28c-kube-api-access-qzt58" (OuterVolumeSpecName: "kube-api-access-qzt58") pod "eb6f69ae-bf47-4aa1-83ca-742b35e0e28c" (UID: "eb6f69ae-bf47-4aa1-83ca-742b35e0e28c"). InnerVolumeSpecName "kube-api-access-qzt58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:52:58.830717 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:52:58.830674    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb6f69ae-bf47-4aa1-83ca-742b35e0e28c-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:52:58.830717 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:52:58.830709    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzt58\" (UniqueName: \"kubernetes.io/projected/eb6f69ae-bf47-4aa1-83ca-742b35e0e28c-kube-api-access-qzt58\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:52:58.980950 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:52:58.980905    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g"]
Jul 28 18:52:58.984481 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:52:58.984442    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d745zj7g"]
Jul 28 18:52:59.287026 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:52:59.286986    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb6f69ae-bf47-4aa1-83ca-742b35e0e28c" path="/var/lib/kubelet/pods/eb6f69ae-bf47-4aa1-83ca-742b35e0e28c/volumes"
Jul 28 18:53:03.681488 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.681445    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"]
Jul 28 18:53:03.682153 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.681787    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb6f69ae-bf47-4aa1-83ca-742b35e0e28c" containerName="main"
Jul 28 18:53:03.682153 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.681800    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb6f69ae-bf47-4aa1-83ca-742b35e0e28c" containerName="main"
Jul 28 18:53:03.682153 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.681871    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb6f69ae-bf47-4aa1-83ca-742b35e0e28c" containerName="main"
Jul 28 18:53:03.686612 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.686581    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.689852 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.689825    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"openshift-service-ca.crt\""
Jul 28 18:53:03.689852 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.689842    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"openshift-gw-ca-root-cert\""
Jul 28 18:53:03.690087 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.689850    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"kube-root-ca.crt\""
Jul 28 18:53:03.690087 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.689909    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-gateway-2-openshift-default-dockercfg-jndnk\""
Jul 28 18:53:03.700780 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.700742    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"]
Jul 28 18:53:03.769431 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.769387    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-envoy\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.769683 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.769437    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbswl\" (UniqueName: \"kubernetes.io/projected/0d26f26a-4f29-476f-ab1b-8ebaad341154-kube-api-access-gbswl\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.769683 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.769531    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-data\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.769683 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.769554    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-podinfo\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.769683 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.769583    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-credential-socket\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.769683 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.769607    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-token\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.769683 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.769673    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-workload-socket\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.770023 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.769719    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-workload-certs\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.770023 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.769765    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d26f26a-4f29-476f-ab1b-8ebaad341154-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.870595 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.870546    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbswl\" (UniqueName: \"kubernetes.io/projected/0d26f26a-4f29-476f-ab1b-8ebaad341154-kube-api-access-gbswl\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.870595 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.870601    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-data\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.870862 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.870645    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-podinfo\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.870862 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.870694    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-credential-socket\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.870862 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.870712    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-token\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.870862 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.870729    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-workload-socket\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.870862 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.870769    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-workload-certs\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.870862 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.870819    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d26f26a-4f29-476f-ab1b-8ebaad341154-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.871122 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.870981    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-envoy\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.871122 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.871003    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-data\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.871272 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.871250    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-credential-socket\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.871359 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.871308    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-workload-certs\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.871419 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.871377    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-workload-socket\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.871714 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.871696    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d26f26a-4f29-476f-ab1b-8ebaad341154-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.873235 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.873212    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-envoy\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.873318 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.873263    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-podinfo\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.879971 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.879943    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-token\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.880113 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.880086    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbswl\" (UniqueName: \"kubernetes.io/projected/0d26f26a-4f29-476f-ab1b-8ebaad341154-kube-api-access-gbswl\") pod \"router-gateway-2-openshift-default-56678d4c48-x75jc\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:03.998653 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:03.998552    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:04.145426 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:04.145397    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"]
Jul 28 18:53:04.147399 ip-10-0-133-215 kubenswrapper[2567]: W0728 18:53:04.147374    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d26f26a_4f29_476f_ab1b_8ebaad341154.slice/crio-2036c035558ef86a65e389bb359fea844e62d9f75d51f693f8c67fadd1ff3a3e WatchSource:0}: Error finding container 2036c035558ef86a65e389bb359fea844e62d9f75d51f693f8c67fadd1ff3a3e: Status 404 returned error can't find the container with id 2036c035558ef86a65e389bb359fea844e62d9f75d51f693f8c67fadd1ff3a3e
Jul 28 18:53:04.150041 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:04.150003    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 18:53:04.150117 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:04.150099    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 18:53:04.150160 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:04.150142    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 18:53:04.688029 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:04.687992    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc" event={"ID":"0d26f26a-4f29-476f-ab1b-8ebaad341154","Type":"ContainerStarted","Data":"7b5c056d5b3fc9be934520e719ab95c02502462190d57196263df6ad344f84f6"}
Jul 28 18:53:04.688494 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:04.688046    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc" event={"ID":"0d26f26a-4f29-476f-ab1b-8ebaad341154","Type":"ContainerStarted","Data":"2036c035558ef86a65e389bb359fea844e62d9f75d51f693f8c67fadd1ff3a3e"}
Jul 28 18:53:04.719528 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:04.713846    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc" podStartSLOduration=1.713824066 podStartE2EDuration="1.713824066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:53:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:53:04.712741024 +0000 UTC m=+6139.958843287" watchObservedRunningTime="2026-07-28 18:53:04.713824066 +0000 UTC m=+6139.959926274"
Jul 28 18:53:04.999527 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:04.999405    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:06.004549 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:06.004496    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:06.697897 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:06.697853    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:06.698956 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:06.698930    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:15.751671 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:15.751634    2567 generic.go:356] "Generic (PLEG): container finished" podID="0d26f26a-4f29-476f-ab1b-8ebaad341154" containerID="7b5c056d5b3fc9be934520e719ab95c02502462190d57196263df6ad344f84f6" exitCode=0
Jul 28 18:53:15.752235 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:15.751712    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc" event={"ID":"0d26f26a-4f29-476f-ab1b-8ebaad341154","Type":"ContainerDied","Data":"7b5c056d5b3fc9be934520e719ab95c02502462190d57196263df6ad344f84f6"}
Jul 28 18:53:15.752235 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:15.752108    2567 scope.go:117] "RemoveContainer" containerID="7b5c056d5b3fc9be934520e719ab95c02502462190d57196263df6ad344f84f6"
Jul 28 18:53:15.753042 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:15.753010    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 18:53:15.753118 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:15.753102    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 18:53:15.753169 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:15.753153    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 18:53:16.758042 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:16.758003    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc" event={"ID":"0d26f26a-4f29-476f-ab1b-8ebaad341154","Type":"ContainerStarted","Data":"71bede6277a0d8ae5abde99f23897151e75dd3c08695dec7adc4d036879a1945"}
Jul 28 18:53:16.999308 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:16.999246    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:17.000569 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:17.000524    2567 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc" podUID="0d26f26a-4f29-476f-ab1b-8ebaad341154" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 28 18:53:17.999440 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:17.999379    2567 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc" podUID="0d26f26a-4f29-476f-ab1b-8ebaad341154" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 28 18:53:19.000128 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:19.000064    2567 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc" podUID="0d26f26a-4f29-476f-ab1b-8ebaad341154" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 28 18:53:19.999275 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:19.999225    2567 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc" podUID="0d26f26a-4f29-476f-ab1b-8ebaad341154" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 28 18:53:20.999335 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:20.999284    2567 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc" podUID="0d26f26a-4f29-476f-ab1b-8ebaad341154" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 28 18:53:21.698796 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:21.698750    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:22.003846 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:22.003752    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:53:22.004752 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:53:22.004733    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:55:46.537072 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:55:46.537043    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:55:46.564140 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:55:46.564111    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v552f_cd62d7f3-7620-47de-a1dd-4e3bc6ea5652/ovn-acl-logging/0.log"
Jul 28 18:56:10.343699 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:10.343653    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"]
Jul 28 18:56:10.344241 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:10.343878    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc" podUID="0d26f26a-4f29-476f-ab1b-8ebaad341154" containerName="istio-proxy" containerID="cri-o://71bede6277a0d8ae5abde99f23897151e75dd3c08695dec7adc4d036879a1945" gracePeriod=30
Jul 28 18:56:15.590864 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.590824    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:56:15.611732 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.611545    2567 generic.go:356] "Generic (PLEG): container finished" podID="0d26f26a-4f29-476f-ab1b-8ebaad341154" containerID="71bede6277a0d8ae5abde99f23897151e75dd3c08695dec7adc4d036879a1945" exitCode=0
Jul 28 18:56:15.611732 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.611621    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"
Jul 28 18:56:15.612009 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.611620    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc" event={"ID":"0d26f26a-4f29-476f-ab1b-8ebaad341154","Type":"ContainerDied","Data":"71bede6277a0d8ae5abde99f23897151e75dd3c08695dec7adc4d036879a1945"}
Jul 28 18:56:15.612009 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.611836    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc" event={"ID":"0d26f26a-4f29-476f-ab1b-8ebaad341154","Type":"ContainerDied","Data":"2036c035558ef86a65e389bb359fea844e62d9f75d51f693f8c67fadd1ff3a3e"}
Jul 28 18:56:15.612009 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.611885    2567 scope.go:117] "RemoveContainer" containerID="71bede6277a0d8ae5abde99f23897151e75dd3c08695dec7adc4d036879a1945"
Jul 28 18:56:15.624176 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.624149    2567 scope.go:117] "RemoveContainer" containerID="7b5c056d5b3fc9be934520e719ab95c02502462190d57196263df6ad344f84f6"
Jul 28 18:56:15.634149 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.634123    2567 scope.go:117] "RemoveContainer" containerID="71bede6277a0d8ae5abde99f23897151e75dd3c08695dec7adc4d036879a1945"
Jul 28 18:56:15.634565 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:56:15.634533    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71bede6277a0d8ae5abde99f23897151e75dd3c08695dec7adc4d036879a1945\": container with ID starting with 71bede6277a0d8ae5abde99f23897151e75dd3c08695dec7adc4d036879a1945 not found: ID does not exist" containerID="71bede6277a0d8ae5abde99f23897151e75dd3c08695dec7adc4d036879a1945"
Jul 28 18:56:15.634710 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.634571    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71bede6277a0d8ae5abde99f23897151e75dd3c08695dec7adc4d036879a1945"} err="failed to get container status \"71bede6277a0d8ae5abde99f23897151e75dd3c08695dec7adc4d036879a1945\": rpc error: code = NotFound desc = could not find container \"71bede6277a0d8ae5abde99f23897151e75dd3c08695dec7adc4d036879a1945\": container with ID starting with 71bede6277a0d8ae5abde99f23897151e75dd3c08695dec7adc4d036879a1945 not found: ID does not exist"
Jul 28 18:56:15.634710 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.634595    2567 scope.go:117] "RemoveContainer" containerID="7b5c056d5b3fc9be934520e719ab95c02502462190d57196263df6ad344f84f6"
Jul 28 18:56:15.634906 ip-10-0-133-215 kubenswrapper[2567]: E0728 18:56:15.634880    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b5c056d5b3fc9be934520e719ab95c02502462190d57196263df6ad344f84f6\": container with ID starting with 7b5c056d5b3fc9be934520e719ab95c02502462190d57196263df6ad344f84f6 not found: ID does not exist" containerID="7b5c056d5b3fc9be934520e719ab95c02502462190d57196263df6ad344f84f6"
Jul 28 18:56:15.635026 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.634909    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b5c056d5b3fc9be934520e719ab95c02502462190d57196263df6ad344f84f6"} err="failed to get container status \"7b5c056d5b3fc9be934520e719ab95c02502462190d57196263df6ad344f84f6\": rpc error: code = NotFound desc = could not find container \"7b5c056d5b3fc9be934520e719ab95c02502462190d57196263df6ad344f84f6\": container with ID starting with 7b5c056d5b3fc9be934520e719ab95c02502462190d57196263df6ad344f84f6 not found: ID does not exist"
Jul 28 18:56:15.654359 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.654323    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-workload-socket\") pod \"0d26f26a-4f29-476f-ab1b-8ebaad341154\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") "
Jul 28 18:56:15.654359 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.654372    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-token\") pod \"0d26f26a-4f29-476f-ab1b-8ebaad341154\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") "
Jul 28 18:56:15.654649 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.654389    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-credential-socket\") pod \"0d26f26a-4f29-476f-ab1b-8ebaad341154\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") "
Jul 28 18:56:15.654649 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.654410    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-podinfo\") pod \"0d26f26a-4f29-476f-ab1b-8ebaad341154\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") "
Jul 28 18:56:15.654649 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.654435    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d26f26a-4f29-476f-ab1b-8ebaad341154-istiod-ca-cert\") pod \"0d26f26a-4f29-476f-ab1b-8ebaad341154\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") "
Jul 28 18:56:15.654649 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.654475    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-workload-certs\") pod \"0d26f26a-4f29-476f-ab1b-8ebaad341154\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") "
Jul 28 18:56:15.654649 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.654638    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-envoy\") pod \"0d26f26a-4f29-476f-ab1b-8ebaad341154\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") "
Jul 28 18:56:15.654918 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.654664    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-data\") pod \"0d26f26a-4f29-476f-ab1b-8ebaad341154\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") "
Jul 28 18:56:15.654918 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.654688    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0d26f26a-4f29-476f-ab1b-8ebaad341154" (UID: "0d26f26a-4f29-476f-ab1b-8ebaad341154"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:56:15.654918 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.654752    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbswl\" (UniqueName: \"kubernetes.io/projected/0d26f26a-4f29-476f-ab1b-8ebaad341154-kube-api-access-gbswl\") pod \"0d26f26a-4f29-476f-ab1b-8ebaad341154\" (UID: \"0d26f26a-4f29-476f-ab1b-8ebaad341154\") "
Jul 28 18:56:15.655073 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.654922    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0d26f26a-4f29-476f-ab1b-8ebaad341154" (UID: "0d26f26a-4f29-476f-ab1b-8ebaad341154"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:56:15.655134 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.655037    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0d26f26a-4f29-476f-ab1b-8ebaad341154" (UID: "0d26f26a-4f29-476f-ab1b-8ebaad341154"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:56:15.655134 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.655119    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-data" (OuterVolumeSpecName: "istio-data") pod "0d26f26a-4f29-476f-ab1b-8ebaad341154" (UID: "0d26f26a-4f29-476f-ab1b-8ebaad341154"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:56:15.655534 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.655485    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d26f26a-4f29-476f-ab1b-8ebaad341154-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0d26f26a-4f29-476f-ab1b-8ebaad341154" (UID: "0d26f26a-4f29-476f-ab1b-8ebaad341154"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:56:15.655733 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.655616    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-workload-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:56:15.655857 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.655747    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-credential-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:56:15.655857 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.655769    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-workload-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:56:15.655857 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.655784    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-data\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:56:15.657438 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.657396    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d26f26a-4f29-476f-ab1b-8ebaad341154-kube-api-access-gbswl" (OuterVolumeSpecName: "kube-api-access-gbswl") pod "0d26f26a-4f29-476f-ab1b-8ebaad341154" (UID: "0d26f26a-4f29-476f-ab1b-8ebaad341154"). InnerVolumeSpecName "kube-api-access-gbswl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:56:15.657576 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.657443    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-token" (OuterVolumeSpecName: "istio-token") pod "0d26f26a-4f29-476f-ab1b-8ebaad341154" (UID: "0d26f26a-4f29-476f-ab1b-8ebaad341154"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:56:15.657576 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.657451    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0d26f26a-4f29-476f-ab1b-8ebaad341154" (UID: "0d26f26a-4f29-476f-ab1b-8ebaad341154"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:56:15.657702 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.657593    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0d26f26a-4f29-476f-ab1b-8ebaad341154" (UID: "0d26f26a-4f29-476f-ab1b-8ebaad341154"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 18:56:15.756332 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.756284    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-token\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:56:15.756332 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.756323    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-podinfo\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:56:15.756332 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.756334    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d26f26a-4f29-476f-ab1b-8ebaad341154-istiod-ca-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:56:15.756332 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.756344    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d26f26a-4f29-476f-ab1b-8ebaad341154-istio-envoy\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:56:15.756670 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.756354    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbswl\" (UniqueName: \"kubernetes.io/projected/0d26f26a-4f29-476f-ab1b-8ebaad341154-kube-api-access-gbswl\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 28 18:56:15.935080 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.935040    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"]
Jul 28 18:56:15.941216 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:15.941180    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-56678d4c48-x75jc"]
Jul 28 18:56:17.287555 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:17.287492    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d26f26a-4f29-476f-ab1b-8ebaad341154" path="/var/lib/kubelet/pods/0d26f26a-4f29-476f-ab1b-8ebaad341154/volumes"
Jul 28 18:56:31.441636 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:31.441597    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-x4v6g_54a9bb44-0a50-48f3-a005-f9f3fc84c9e6/discovery/0.log"
Jul 28 18:56:31.455181 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:31.455147    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-8699b59bbc-nvf9s_be331145-6d91-44a5-aa68-247f0ab1262b/istio-proxy/0.log"
Jul 28 18:56:31.496803 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:31.496762    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-8699b59bbc-nvf9s_be331145-6d91-44a5-aa68-247f0ab1262b/istio-proxy/1.log"
Jul 28 18:56:32.437142 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:32.437112    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-x4v6g_54a9bb44-0a50-48f3-a005-f9f3fc84c9e6/discovery/0.log"
Jul 28 18:56:32.503302 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:32.503272    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-8699b59bbc-nvf9s_be331145-6d91-44a5-aa68-247f0ab1262b/istio-proxy/0.log"
Jul 28 18:56:32.548199 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:32.548151    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-8699b59bbc-nvf9s_be331145-6d91-44a5-aa68-247f0ab1262b/istio-proxy/1.log"
Jul 28 18:56:33.725292 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:33.725256    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-x4v6g_54a9bb44-0a50-48f3-a005-f9f3fc84c9e6/discovery/0.log"
Jul 28 18:56:33.805204 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:33.805169    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-8699b59bbc-nvf9s_be331145-6d91-44a5-aa68-247f0ab1262b/istio-proxy/0.log"
Jul 28 18:56:33.847758 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:33.847716    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-8699b59bbc-nvf9s_be331145-6d91-44a5-aa68-247f0ab1262b/istio-proxy/1.log"
Jul 28 18:56:34.849614 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:34.849576    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-x4v6g_54a9bb44-0a50-48f3-a005-f9f3fc84c9e6/discovery/0.log"
Jul 28 18:56:34.924068 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:34.924008    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-8699b59bbc-nvf9s_be331145-6d91-44a5-aa68-247f0ab1262b/istio-proxy/0.log"
Jul 28 18:56:34.961494 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:34.961458    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-8699b59bbc-nvf9s_be331145-6d91-44a5-aa68-247f0ab1262b/istio-proxy/1.log"
Jul 28 18:56:35.940185 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:35.940147    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-nq65w_1ae9718c-eecf-4360-afd2-76105c0ef626/manager/0.log"
Jul 28 18:56:36.161547 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:36.161484    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-66866f5fcc-7phb4_b9985ba3-0799-40cb-8e84-d747050273d6/manager/0.log"
Jul 28 18:56:36.170601 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:36.170576    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-glmwj_a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b/limitador/0.log"
Jul 28 18:56:37.135220 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:37.135171    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-nq65w_1ae9718c-eecf-4360-afd2-76105c0ef626/manager/0.log"
Jul 28 18:56:37.346752 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:37.346706    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-66866f5fcc-7phb4_b9985ba3-0799-40cb-8e84-d747050273d6/manager/0.log"
Jul 28 18:56:37.371217 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:37.371184    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-glmwj_a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b/limitador/0.log"
Jul 28 18:56:38.312978 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:38.312942    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-nq65w_1ae9718c-eecf-4360-afd2-76105c0ef626/manager/0.log"
Jul 28 18:56:38.515027 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:38.514996    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-66866f5fcc-7phb4_b9985ba3-0799-40cb-8e84-d747050273d6/manager/0.log"
Jul 28 18:56:38.532311 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:38.532278    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-glmwj_a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b/limitador/0.log"
Jul 28 18:56:39.518877 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:39.518844    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-nq65w_1ae9718c-eecf-4360-afd2-76105c0ef626/manager/0.log"
Jul 28 18:56:39.723157 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:39.723120    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-66866f5fcc-7phb4_b9985ba3-0799-40cb-8e84-d747050273d6/manager/0.log"
Jul 28 18:56:39.738894 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:39.738869    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-glmwj_a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b/limitador/0.log"
Jul 28 18:56:40.670558 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:40.670520    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-nq65w_1ae9718c-eecf-4360-afd2-76105c0ef626/manager/0.log"
Jul 28 18:56:40.883341 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:40.883274    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-66866f5fcc-7phb4_b9985ba3-0799-40cb-8e84-d747050273d6/manager/0.log"
Jul 28 18:56:40.900858 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:40.900828    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-glmwj_a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b/limitador/0.log"
Jul 28 18:56:41.828393 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:41.828357    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-nq65w_1ae9718c-eecf-4360-afd2-76105c0ef626/manager/0.log"
Jul 28 18:56:42.029252 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:42.029211    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-66866f5fcc-7phb4_b9985ba3-0799-40cb-8e84-d747050273d6/manager/0.log"
Jul 28 18:56:42.044877 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:42.044849    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-glmwj_a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b/limitador/0.log"
Jul 28 18:56:42.994833 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:42.994806    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-nq65w_1ae9718c-eecf-4360-afd2-76105c0ef626/manager/0.log"
Jul 28 18:56:43.202107 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:43.202061    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-66866f5fcc-7phb4_b9985ba3-0799-40cb-8e84-d747050273d6/manager/0.log"
Jul 28 18:56:43.216875 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:43.216845    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-glmwj_a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b/limitador/0.log"
Jul 28 18:56:44.158735 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:44.158702    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-nq65w_1ae9718c-eecf-4360-afd2-76105c0ef626/manager/0.log"
Jul 28 18:56:44.366477 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:44.366433    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-66866f5fcc-7phb4_b9985ba3-0799-40cb-8e84-d747050273d6/manager/0.log"
Jul 28 18:56:44.381627 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:44.381599    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-glmwj_a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b/limitador/0.log"
Jul 28 18:56:45.335375 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:45.335334    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-nq65w_1ae9718c-eecf-4360-afd2-76105c0ef626/manager/0.log"
Jul 28 18:56:45.538409 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:45.538363    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-66866f5fcc-7phb4_b9985ba3-0799-40cb-8e84-d747050273d6/manager/0.log"
Jul 28 18:56:45.554628 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:45.554599    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-glmwj_a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b/limitador/0.log"
Jul 28 18:56:46.516470 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:46.516441    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-nq65w_1ae9718c-eecf-4360-afd2-76105c0ef626/manager/0.log"
Jul 28 18:56:46.725993 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:46.725955    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-66866f5fcc-7phb4_b9985ba3-0799-40cb-8e84-d747050273d6/manager/0.log"
Jul 28 18:56:46.743412 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:46.743380    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-glmwj_a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b/limitador/0.log"
Jul 28 18:56:47.700996 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:47.700961    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-nq65w_1ae9718c-eecf-4360-afd2-76105c0ef626/manager/0.log"
Jul 28 18:56:47.912356 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:47.912317    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-66866f5fcc-7phb4_b9985ba3-0799-40cb-8e84-d747050273d6/manager/0.log"
Jul 28 18:56:47.928377 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:47.928346    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-glmwj_a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b/limitador/0.log"
Jul 28 18:56:48.890273 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:48.890234    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-nq65w_1ae9718c-eecf-4360-afd2-76105c0ef626/manager/0.log"
Jul 28 18:56:49.091738 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:49.091693    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-66866f5fcc-7phb4_b9985ba3-0799-40cb-8e84-d747050273d6/manager/0.log"
Jul 28 18:56:49.107356 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:49.107328    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-glmwj_a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b/limitador/0.log"
Jul 28 18:56:50.068244 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:50.068212    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-nq65w_1ae9718c-eecf-4360-afd2-76105c0ef626/manager/0.log"
Jul 28 18:56:50.293470 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:50.293430    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-66866f5fcc-7phb4_b9985ba3-0799-40cb-8e84-d747050273d6/manager/0.log"
Jul 28 18:56:50.323323 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:50.323223    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-glmwj_a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b/limitador/0.log"
Jul 28 18:56:52.498413 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:52.498372    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-67kr8/must-gather-k5rkf"]
Jul 28 18:56:52.498844 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:52.498705    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d26f26a-4f29-476f-ab1b-8ebaad341154" containerName="istio-proxy"
Jul 28 18:56:52.498844 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:52.498717    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d26f26a-4f29-476f-ab1b-8ebaad341154" containerName="istio-proxy"
Jul 28 18:56:52.498844 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:52.498773    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d26f26a-4f29-476f-ab1b-8ebaad341154" containerName="istio-proxy"
Jul 28 18:56:52.498844 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:52.498820    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d26f26a-4f29-476f-ab1b-8ebaad341154" containerName="istio-proxy"
Jul 28 18:56:52.498844 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:52.498826    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d26f26a-4f29-476f-ab1b-8ebaad341154" containerName="istio-proxy"
Jul 28 18:56:52.499015 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:52.498875    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d26f26a-4f29-476f-ab1b-8ebaad341154" containerName="istio-proxy"
Jul 28 18:56:52.501800 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:52.501780    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-67kr8/must-gather-k5rkf"
Jul 28 18:56:52.504800 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:52.504773    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-67kr8\"/\"default-dockercfg-5smz9\""
Jul 28 18:56:52.504800 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:52.504785    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-67kr8\"/\"openshift-service-ca.crt\""
Jul 28 18:56:52.504994 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:52.504856    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-67kr8\"/\"kube-root-ca.crt\""
Jul 28 18:56:52.513363 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:52.513331    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-67kr8/must-gather-k5rkf"]
Jul 28 18:56:52.634815 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:52.634781    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs6zs\" (UniqueName: \"kubernetes.io/projected/5413a258-96b3-40c9-be56-28632a69bb81-kube-api-access-hs6zs\") pod \"must-gather-k5rkf\" (UID: \"5413a258-96b3-40c9-be56-28632a69bb81\") " pod="openshift-must-gather-67kr8/must-gather-k5rkf"
Jul 28 18:56:52.635001 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:52.634844    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5413a258-96b3-40c9-be56-28632a69bb81-must-gather-output\") pod \"must-gather-k5rkf\" (UID: \"5413a258-96b3-40c9-be56-28632a69bb81\") " pod="openshift-must-gather-67kr8/must-gather-k5rkf"
Jul 28 18:56:52.735741 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:52.735699    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs6zs\" (UniqueName: \"kubernetes.io/projected/5413a258-96b3-40c9-be56-28632a69bb81-kube-api-access-hs6zs\") pod \"must-gather-k5rkf\" (UID: \"5413a258-96b3-40c9-be56-28632a69bb81\") " pod="openshift-must-gather-67kr8/must-gather-k5rkf"
Jul 28 18:56:52.735989 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:52.735856    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5413a258-96b3-40c9-be56-28632a69bb81-must-gather-output\") pod \"must-gather-k5rkf\" (UID: \"5413a258-96b3-40c9-be56-28632a69bb81\") " pod="openshift-must-gather-67kr8/must-gather-k5rkf"
Jul 28 18:56:52.736169 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:52.736153    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5413a258-96b3-40c9-be56-28632a69bb81-must-gather-output\") pod \"must-gather-k5rkf\" (UID: \"5413a258-96b3-40c9-be56-28632a69bb81\") " pod="openshift-must-gather-67kr8/must-gather-k5rkf"
Jul 28 18:56:52.749208 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:52.749144    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs6zs\" (UniqueName: \"kubernetes.io/projected/5413a258-96b3-40c9-be56-28632a69bb81-kube-api-access-hs6zs\") pod \"must-gather-k5rkf\" (UID: \"5413a258-96b3-40c9-be56-28632a69bb81\") " pod="openshift-must-gather-67kr8/must-gather-k5rkf"
Jul 28 18:56:52.811459 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:52.811425    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-67kr8/must-gather-k5rkf"
Jul 28 18:56:52.946387 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:52.946360    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-67kr8/must-gather-k5rkf"]
Jul 28 18:56:52.948596 ip-10-0-133-215 kubenswrapper[2567]: W0728 18:56:52.948562    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5413a258_96b3_40c9_be56_28632a69bb81.slice/crio-d2e0c0d2ab637fce055536829a4b4b01a5cbc599de5e39bf8a2d699943daf89b WatchSource:0}: Error finding container d2e0c0d2ab637fce055536829a4b4b01a5cbc599de5e39bf8a2d699943daf89b: Status 404 returned error can't find the container with id d2e0c0d2ab637fce055536829a4b4b01a5cbc599de5e39bf8a2d699943daf89b
Jul 28 18:56:52.950575 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:52.950555    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 18:56:53.801674 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:53.801642    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-67kr8/must-gather-k5rkf" event={"ID":"5413a258-96b3-40c9-be56-28632a69bb81","Type":"ContainerStarted","Data":"d61524b72c5245108f35e835d842b52098f77de6b2b099f5deec981b79804270"}
Jul 28 18:56:53.802113 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:53.801686    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-67kr8/must-gather-k5rkf" event={"ID":"5413a258-96b3-40c9-be56-28632a69bb81","Type":"ContainerStarted","Data":"d2e0c0d2ab637fce055536829a4b4b01a5cbc599de5e39bf8a2d699943daf89b"}
Jul 28 18:56:54.810081 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:54.810036    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-67kr8/must-gather-k5rkf" event={"ID":"5413a258-96b3-40c9-be56-28632a69bb81","Type":"ContainerStarted","Data":"67c15e2bd7d4ad1c45f71fe77dca624f8e08356c75164e4507129b0e935828a4"}
Jul 28 18:56:54.828107 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:54.828042    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-67kr8/must-gather-k5rkf" podStartSLOduration=2.144387428 podStartE2EDuration="2.828019351s" totalImagesPullingTime="683.631923ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:56:54.826879936 +0000 UTC m=+6370.072982144" watchObservedRunningTime="2026-07-28 18:56:54.828019351 +0000 UTC m=+6370.074121559"
Jul 28 18:56:55.159927 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:55.159820    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vcff5_482929b8-7c61-4fee-bd3b-0f9ac86e61df/global-pull-secret-syncer/0.log"
Jul 28 18:56:55.371563 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:55.371522    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-l82pq_3f8e6a3b-524e-403d-bfdf-338bf7c5defc/konnectivity-agent/0.log"
Jul 28 18:56:55.395747 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:56:55.395715    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-215.ec2.internal_f36b97c92bac0bfb582fa26faaad0625/haproxy/0.log"
Jul 28 18:57:00.551193 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:00.551151    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-nq65w_1ae9718c-eecf-4360-afd2-76105c0ef626/manager/0.log"
Jul 28 18:57:00.868827 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:00.868305    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-66866f5fcc-7phb4_b9985ba3-0799-40cb-8e84-d747050273d6/manager/0.log"
Jul 28 18:57:00.894293 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:00.894265    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-glmwj_a4e05dcd-6cb8-4e63-8c53-3ff7b4bf520b/limitador/0.log"
Jul 28 18:57:02.797602 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:02.797551    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mfm74_b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2/node-exporter/0.log"
Jul 28 18:57:02.825849 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:02.825818    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mfm74_b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2/kube-rbac-proxy/0.log"
Jul 28 18:57:02.856311 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:02.856268    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mfm74_b0c92dfd-b79b-4cf0-ab2b-5c8823ace7e2/init-textfile/0.log"
Jul 28 18:57:05.843780 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:05.843737    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"]
Jul 28 18:57:05.849749 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:05.849717    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"
Jul 28 18:57:05.856099 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:05.856018    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"]
Jul 28 18:57:05.964181 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:05.964092    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/78d605f3-69f6-4ede-bd9f-2b311163fb80-lib-modules\") pod \"perf-node-gather-daemonset-9zqjs\" (UID: \"78d605f3-69f6-4ede-bd9f-2b311163fb80\") " pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"
Jul 28 18:57:05.964563 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:05.964500    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q58qg\" (UniqueName: \"kubernetes.io/projected/78d605f3-69f6-4ede-bd9f-2b311163fb80-kube-api-access-q58qg\") pod \"perf-node-gather-daemonset-9zqjs\" (UID: \"78d605f3-69f6-4ede-bd9f-2b311163fb80\") " pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"
Jul 28 18:57:05.964723 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:05.964709    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/78d605f3-69f6-4ede-bd9f-2b311163fb80-podres\") pod \"perf-node-gather-daemonset-9zqjs\" (UID: \"78d605f3-69f6-4ede-bd9f-2b311163fb80\") " pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"
Jul 28 18:57:05.964863 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:05.964850    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78d605f3-69f6-4ede-bd9f-2b311163fb80-sys\") pod \"perf-node-gather-daemonset-9zqjs\" (UID: \"78d605f3-69f6-4ede-bd9f-2b311163fb80\") " pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"
Jul 28 18:57:05.964993 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:05.964981    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/78d605f3-69f6-4ede-bd9f-2b311163fb80-proc\") pod \"perf-node-gather-daemonset-9zqjs\" (UID: \"78d605f3-69f6-4ede-bd9f-2b311163fb80\") " pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"
Jul 28 18:57:06.066024 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:06.065968    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/78d605f3-69f6-4ede-bd9f-2b311163fb80-lib-modules\") pod \"perf-node-gather-daemonset-9zqjs\" (UID: \"78d605f3-69f6-4ede-bd9f-2b311163fb80\") " pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"
Jul 28 18:57:06.066224 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:06.066065    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q58qg\" (UniqueName: \"kubernetes.io/projected/78d605f3-69f6-4ede-bd9f-2b311163fb80-kube-api-access-q58qg\") pod \"perf-node-gather-daemonset-9zqjs\" (UID: \"78d605f3-69f6-4ede-bd9f-2b311163fb80\") " pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"
Jul 28 18:57:06.066224 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:06.066102    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/78d605f3-69f6-4ede-bd9f-2b311163fb80-podres\") pod \"perf-node-gather-daemonset-9zqjs\" (UID: \"78d605f3-69f6-4ede-bd9f-2b311163fb80\") " pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"
Jul 28 18:57:06.066224 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:06.066121    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78d605f3-69f6-4ede-bd9f-2b311163fb80-sys\") pod \"perf-node-gather-daemonset-9zqjs\" (UID: \"78d605f3-69f6-4ede-bd9f-2b311163fb80\") " pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"
Jul 28 18:57:06.066224 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:06.066195    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/78d605f3-69f6-4ede-bd9f-2b311163fb80-lib-modules\") pod \"perf-node-gather-daemonset-9zqjs\" (UID: \"78d605f3-69f6-4ede-bd9f-2b311163fb80\") " pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"
Jul 28 18:57:06.066420 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:06.066263    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/78d605f3-69f6-4ede-bd9f-2b311163fb80-podres\") pod \"perf-node-gather-daemonset-9zqjs\" (UID: \"78d605f3-69f6-4ede-bd9f-2b311163fb80\") " pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"
Jul 28 18:57:06.066420 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:06.066284    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78d605f3-69f6-4ede-bd9f-2b311163fb80-sys\") pod \"perf-node-gather-daemonset-9zqjs\" (UID: \"78d605f3-69f6-4ede-bd9f-2b311163fb80\") " pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"
Jul 28 18:57:06.066420 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:06.066338    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/78d605f3-69f6-4ede-bd9f-2b311163fb80-proc\") pod \"perf-node-gather-daemonset-9zqjs\" (UID: \"78d605f3-69f6-4ede-bd9f-2b311163fb80\") " pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"
Jul 28 18:57:06.066420 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:06.066413    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/78d605f3-69f6-4ede-bd9f-2b311163fb80-proc\") pod \"perf-node-gather-daemonset-9zqjs\" (UID: \"78d605f3-69f6-4ede-bd9f-2b311163fb80\") " pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"
Jul 28 18:57:06.075394 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:06.075358    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q58qg\" (UniqueName: \"kubernetes.io/projected/78d605f3-69f6-4ede-bd9f-2b311163fb80-kube-api-access-q58qg\") pod \"perf-node-gather-daemonset-9zqjs\" (UID: \"78d605f3-69f6-4ede-bd9f-2b311163fb80\") " pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"
Jul 28 18:57:06.163300 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:06.163190    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"
Jul 28 18:57:06.340782 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:06.340749    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"]
Jul 28 18:57:06.343043 ip-10-0-133-215 kubenswrapper[2567]: W0728 18:57:06.343009    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod78d605f3_69f6_4ede_bd9f_2b311163fb80.slice/crio-dee173e5ef9cee8488419261baacd902e7458ba9bc026da39937fc2d292adba1 WatchSource:0}: Error finding container dee173e5ef9cee8488419261baacd902e7458ba9bc026da39937fc2d292adba1: Status 404 returned error can't find the container with id dee173e5ef9cee8488419261baacd902e7458ba9bc026da39937fc2d292adba1
Jul 28 18:57:06.931810 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:06.931754    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs" event={"ID":"78d605f3-69f6-4ede-bd9f-2b311163fb80","Type":"ContainerStarted","Data":"a59d2bc0f7aa6c9332da166b3ab449ad737187d79a3fd953773f5e9cf4596043"}
Jul 28 18:57:06.932312 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:06.931837    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs" event={"ID":"78d605f3-69f6-4ede-bd9f-2b311163fb80","Type":"ContainerStarted","Data":"dee173e5ef9cee8488419261baacd902e7458ba9bc026da39937fc2d292adba1"}
Jul 28 18:57:06.932312 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:06.932084    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"
Jul 28 18:57:06.949590 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:06.949527    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs" podStartSLOduration=1.949486286 podStartE2EDuration="1.949486286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:57:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:57:06.948989373 +0000 UTC m=+6382.195091581" watchObservedRunningTime="2026-07-28 18:57:06.949486286 +0000 UTC m=+6382.195588496"
Jul 28 18:57:07.419213 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:07.419172    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wfpdb_14fdd663-eddf-46d8-ab05-2c125ec570fe/dns/0.log"
Jul 28 18:57:07.440606 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:07.440567    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wfpdb_14fdd663-eddf-46d8-ab05-2c125ec570fe/kube-rbac-proxy/0.log"
Jul 28 18:57:07.488283 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:07.488247    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-92rx8_7ad540e6-ff3a-4366-9176-0063eb45135c/dns-node-resolver/0.log"
Jul 28 18:57:08.084525 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:08.084473    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cxrrh_1b94f336-770a-4f5a-8ef9-cf867e720989/node-ca/0.log"
Jul 28 18:57:09.370773 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:09.370743    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-x4v6g_54a9bb44-0a50-48f3-a005-f9f3fc84c9e6/discovery/0.log"
Jul 28 18:57:09.401670 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:09.401634    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-8699b59bbc-nvf9s_be331145-6d91-44a5-aa68-247f0ab1262b/istio-proxy/0.log"
Jul 28 18:57:09.454137 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:09.454101    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-8699b59bbc-nvf9s_be331145-6d91-44a5-aa68-247f0ab1262b/istio-proxy/1.log"
Jul 28 18:57:10.015048 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:10.015009    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rrkx5_8c7cadd8-f3b6-4499-96c4-e3a3c2c3f03e/serve-healthcheck-canary/0.log"
Jul 28 18:57:10.547663 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:10.547633    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8dpn4_63377241-ccfc-4a7a-844d-6bc00d6f7db5/kube-rbac-proxy/0.log"
Jul 28 18:57:10.572584 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:10.572550    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8dpn4_63377241-ccfc-4a7a-844d-6bc00d6f7db5/exporter/0.log"
Jul 28 18:57:10.595871 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:10.595845    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8dpn4_63377241-ccfc-4a7a-844d-6bc00d6f7db5/extractor/0.log"
Jul 28 18:57:12.952339 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:12.952297    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-67kr8/perf-node-gather-daemonset-9zqjs"
Jul 28 18:57:16.427526 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:16.427104    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-x4v6g_54a9bb44-0a50-48f3-a005-f9f3fc84c9e6/discovery/0.log"
Jul 28 18:57:16.441853 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:16.441814    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-8699b59bbc-nvf9s_be331145-6d91-44a5-aa68-247f0ab1262b/istio-proxy/0.log"
Jul 28 18:57:16.494746 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:16.494716    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-8699b59bbc-nvf9s_be331145-6d91-44a5-aa68-247f0ab1262b/istio-proxy/1.log"
Jul 28 18:57:17.920035 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:17.919997    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-x4v6g_54a9bb44-0a50-48f3-a005-f9f3fc84c9e6/discovery/0.log"
Jul 28 18:57:17.947692 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:17.947667    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-8699b59bbc-nvf9s_be331145-6d91-44a5-aa68-247f0ab1262b/istio-proxy/0.log"
Jul 28 18:57:18.000300 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:18.000266    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-8699b59bbc-nvf9s_be331145-6d91-44a5-aa68-247f0ab1262b/istio-proxy/1.log"
Jul 28 18:57:22.854404 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:22.854374    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zxsfx_f363ee59-3092-46f1-a3e5-a01b034a1e14/kube-multus-additional-cni-plugins/0.log"
Jul 28 18:57:22.897732 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:22.897701    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zxsfx_f363ee59-3092-46f1-a3e5-a01b034a1e14/egress-router-binary-copy/0.log"
Jul 28 18:57:22.947301 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:22.947274    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zxsfx_f363ee59-3092-46f1-a3e5-a01b034a1e14/cni-plugins/0.log"
Jul 28 18:57:22.977828 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:22.977800    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zxsfx_f363ee59-3092-46f1-a3e5-a01b034a1e14/bond-cni-plugin/0.log"
Jul 28 18:57:22.999023 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:22.998994    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zxsfx_f363ee59-3092-46f1-a3e5-a01b034a1e14/routeoverride-cni/0.log"
Jul 28 18:57:23.020608 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:23.020581    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zxsfx_f363ee59-3092-46f1-a3e5-a01b034a1e14/whereabouts-cni-bincopy/0.log"
Jul 28 18:57:23.044399 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:23.044364    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zxsfx_f363ee59-3092-46f1-a3e5-a01b034a1e14/whereabouts-cni/0.log"
Jul 28 18:57:23.137258 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:23.137161    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q4vcf_8337ae34-e335-4033-8ecd-49dd32a2ea88/kube-multus/0.log"
Jul 28 18:57:23.233151 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:23.233122    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fr9xb_97f130a6-e3b7-4882-b992-0d5eef10897b/network-metrics-daemon/0.log"
Jul 28 18:57:23.251487 ip-10-0-133-215 kubenswrapper[2567]: I0728 18:57:23.251457    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fr9xb_97f130a6-e3b7-4882-b992-0d5eef10897b/kube-rbac-proxy/0.log"