Jul 27 19:37:52.372547 ip-10-0-140-36 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 27 19:37:52.372557 ip-10-0-140-36 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 27 19:37:52.372564 ip-10-0-140-36 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 27 19:37:52.372778 ip-10-0-140-36 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 27 19:38:02.114278 ip-10-0-140-36 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Jul 27 19:38:02.114293 ip-10-0-140-36 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot a5d50954135e424fb17c5fd5879b7b8f --
Jul 27 19:39:31.644136 ip-10-0-140-36 systemd[1]: Starting Kubernetes Kubelet...
Jul 27 19:39:32.086128 ip-10-0-140-36 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 19:39:32.086128 ip-10-0-140-36 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 27 19:39:32.086128 ip-10-0-140-36 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 19:39:32.086128 ip-10-0-140-36 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 27 19:39:32.086758 ip-10-0-140-36 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 19:39:32.087189 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.087093    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 27 19:39:32.093632 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093618    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:39:32.093669 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093633    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:39:32.093669 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093637    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:39:32.093669 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093640    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:39:32.093669 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093643    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:39:32.093669 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093646    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:39:32.093669 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093648    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:39:32.093669 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093651    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:39:32.093669 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093655    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:39:32.093669 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093657    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:39:32.093669 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093661    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:39:32.093669 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093663    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:39:32.093669 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093666    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:39:32.093669 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093668    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:39:32.093669 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093671    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:39:32.094056 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093678    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:39:32.094056 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093681    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:39:32.094056 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093684    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:39:32.094056 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093687    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:39:32.094056 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093689    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:39:32.094056 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093692    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:39:32.094056 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093694    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:39:32.094056 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093697    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:39:32.094056 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093699    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:39:32.094056 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093702    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:39:32.094056 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093704    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:39:32.094056 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093707    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:39:32.094056 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093710    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:39:32.094056 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093714    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:39:32.094056 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093717    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:39:32.094056 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093720    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:39:32.094056 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093723    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:39:32.094056 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093725    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:39:32.094056 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093728    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:39:32.094056 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093730    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:39:32.094631 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093733    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:39:32.094631 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093735    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:39:32.094631 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093738    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:39:32.094631 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093741    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:39:32.094631 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093743    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:39:32.094631 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093746    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:39:32.094631 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093749    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:39:32.094631 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093752    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:39:32.094631 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093754    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:39:32.094631 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093757    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:39:32.094631 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093759    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:39:32.094631 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093762    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:39:32.094631 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093764    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:39:32.094631 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093767    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:39:32.094631 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093770    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:39:32.094631 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093773    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:39:32.094631 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093775    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:39:32.094631 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093777    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:39:32.094631 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093780    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:39:32.094631 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093782    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:39:32.095149 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093785    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:39:32.095149 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093787    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:39:32.095149 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093790    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:39:32.095149 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093793    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:39:32.095149 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093795    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:39:32.095149 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093797    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:39:32.095149 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093800    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:39:32.095149 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093802    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:39:32.095149 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093805    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:39:32.095149 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093807    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:39:32.095149 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093810    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:39:32.095149 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093812    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:39:32.095149 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093814    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:39:32.095149 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093817    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:39:32.095149 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093820    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:39:32.095149 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093823    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:39:32.095149 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093826    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:39:32.095149 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093828    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:39:32.095149 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093830    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:39:32.095595 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093833    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:39:32.095595 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093835    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:39:32.095595 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093838    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:39:32.095595 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093840    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:39:32.095595 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093844    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:39:32.095595 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093846    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:39:32.095595 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093850    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:39:32.095595 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093868    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:39:32.095595 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093872    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:39:32.095595 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093875    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:39:32.095595 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093878    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:39:32.095595 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093881    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:39:32.095595 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093884    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:39:32.095595 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093887    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:39:32.095595 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093891    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:39:32.095595 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093895    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:39:32.095595 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093898    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:39:32.095595 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093901    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:39:32.095595 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093904    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:39:32.096066 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093907    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:39:32.096066 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093910    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:39:32.096066 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093912    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:39:32.096066 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093916    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:39:32.096066 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093918    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:39:32.096066 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.093921    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:39:32.096066 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094570    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:39:32.096066 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094576    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:39:32.096066 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094579    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:39:32.096066 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094581    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:39:32.096066 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094584    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:39:32.096066 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094588    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:39:32.096066 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094591    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:39:32.096066 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094593    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:39:32.096066 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094596    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:39:32.096066 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094599    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:39:32.096066 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094601    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:39:32.096066 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094604    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:39:32.096066 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094607    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:39:32.096066 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094609    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:39:32.096572 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094612    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:39:32.096572 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094614    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:39:32.096572 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094617    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:39:32.096572 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094620    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:39:32.096572 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094622    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:39:32.096572 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094625    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:39:32.096572 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094630    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:39:32.096572 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094633    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:39:32.096572 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094637    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:39:32.096572 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094640    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:39:32.096572 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094642    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:39:32.096572 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094645    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:39:32.096572 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094648    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:39:32.096572 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094651    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:39:32.096572 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094653    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:39:32.096572 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094656    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:39:32.096572 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094658    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:39:32.096572 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094661    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:39:32.096572 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094663    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:39:32.097041 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094666    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:39:32.097041 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094668    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:39:32.097041 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094671    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:39:32.097041 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094673    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:39:32.097041 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094676    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:39:32.097041 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094679    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:39:32.097041 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094682    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:39:32.097041 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094684    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:39:32.097041 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094687    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:39:32.097041 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094689    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:39:32.097041 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094692    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:39:32.097041 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094695    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:39:32.097041 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094698    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:39:32.097041 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094700    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:39:32.097041 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094703    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:39:32.097041 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094705    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:39:32.097041 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094708    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:39:32.097041 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094710    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:39:32.097041 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094713    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:39:32.097041 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094715    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:39:32.097526 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094719    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:39:32.097526 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094721    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:39:32.097526 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094723    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:39:32.097526 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094727    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:39:32.097526 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094731    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:39:32.097526 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094734    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:39:32.097526 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094736    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:39:32.097526 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094739    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:39:32.097526 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094741    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:39:32.097526 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094744    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:39:32.097526 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094746    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:39:32.097526 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094748    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:39:32.097526 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094751    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:39:32.097526 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094753    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:39:32.097526 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094756    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:39:32.097526 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094759    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:39:32.097526 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094761    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:39:32.097526 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094764    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:39:32.097526 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094767    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:39:32.097526 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094769    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:39:32.098043 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094772    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:39:32.098043 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094774    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:39:32.098043 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094777    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:39:32.098043 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094780    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:39:32.098043 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094782    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:39:32.098043 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094785    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:39:32.098043 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094787    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:39:32.098043 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094790    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:39:32.098043 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094792    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:39:32.098043 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094795    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:39:32.098043 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094798    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:39:32.098043 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094802    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:39:32.098043 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094804    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:39:32.098043 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094807    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:39:32.098043 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094809    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:39:32.098043 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094812    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:39:32.098043 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094814    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:39:32.098043 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094817    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:39:32.098043 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094819    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:39:32.098043 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094822    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:39:32.098521 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094824    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:39:32.098521 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094826    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:39:32.098521 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094829    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:39:32.098521 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094831    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:39:32.098521 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094835    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:39:32.098521 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.094838    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:39:32.098521 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095256    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:39:32.098521 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095262    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:39:32.098521 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095265    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:39:32.098521 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095268    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:39:32.098521 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095273    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:39:32.098521 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095277    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:39:32.098521 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095279    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:39:32.098521 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095282    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:39:32.098521 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095285    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:39:32.098521 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095289    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:39:32.098521 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095292    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:39:32.098521 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095295    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:39:32.098521 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095298    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:39:32.099025 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095301    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:39:32.099025 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095304    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:39:32.099025 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095306    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:39:32.099025 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095309    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:39:32.099025 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095312    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:39:32.099025 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095314    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:39:32.099025 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095317    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:39:32.099025 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095319    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:39:32.099025 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095322    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:39:32.099025 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095324    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:39:32.099025 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095327    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:39:32.099025 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095329    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:39:32.099025 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095332    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:39:32.099025 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095334    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:39:32.099025 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095337    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:39:32.099025 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095339    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:39:32.099025 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095342    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:39:32.099025 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095345    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:39:32.099025 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095347    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:39:32.099025 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095350    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:39:32.099509 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095352    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:39:32.099509 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095354    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:39:32.099509 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095357    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:39:32.099509 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095360    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:39:32.099509 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095363    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:39:32.099509 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095366    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:39:32.099509 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095368    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:39:32.099509 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095371    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:39:32.099509 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095373    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:39:32.099509 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095376    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:39:32.099509 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095379    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:39:32.099509 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095381    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:39:32.099509 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095384    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:39:32.099509 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095386    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:39:32.099509 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095390    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:39:32.099509 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095394    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:39:32.099509 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095397    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:39:32.099509 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095400    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:39:32.099509 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095404    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:39:32.099995 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095406    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:39:32.099995 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095409    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:39:32.099995 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095412    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:39:32.099995 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095415    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:39:32.099995 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095417    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:39:32.099995 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095420    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:39:32.099995 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095423    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:39:32.099995 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095426    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:39:32.099995 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095428    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:39:32.099995 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095431    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:39:32.099995 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095433    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:39:32.099995 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095436    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:39:32.099995 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095438    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:39:32.099995 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095441    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:39:32.099995 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095443    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:39:32.099995 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095446    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:39:32.099995 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095449    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:39:32.099995 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095452    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:39:32.099995 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095455    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:39:32.099995 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095458    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:39:32.100519 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095461    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:39:32.100519 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095465    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:39:32.100519 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095467    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:39:32.100519 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095469    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:39:32.100519 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095472    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:39:32.100519 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095474    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:39:32.100519 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095477    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:39:32.100519 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095480    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:39:32.100519 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095482    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:39:32.100519 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095485    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:39:32.100519 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095487    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:39:32.100519 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095489    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:39:32.100519 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095492    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:39:32.100519 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095494    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:39:32.100519 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095497    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:39:32.100519 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095499    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:39:32.100519 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095502    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:39:32.100519 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095504    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:39:32.100519 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095506    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:39:32.100519 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095509    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095511    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095514    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095516    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095519    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095521    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095523    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.095526    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095592    2579 flags.go:64] FLAG: --address="0.0.0.0"
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095599    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095604    2579 flags.go:64] FLAG: --anonymous-auth="true"
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095608    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095613    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095616    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095619    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095623    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095628    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095631    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095637    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095640    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095647    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095652    2579 flags.go:64] FLAG: --cgroup-root=""
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095656    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 27 19:39:32.101033 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095659    2579 flags.go:64] FLAG: --client-ca-file=""
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095662    2579 flags.go:64] FLAG: --cloud-provider="external"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095665    2579 flags.go:64] FLAG: --cluster-dns="[]"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095670    2579 flags.go:64] FLAG: --cluster-domain=""
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095672    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095675    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095678    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095682    2579 flags.go:64] FLAG: --container-log-max-files="5"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095685    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095689    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095692    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095694    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095697    2579 flags.go:64] FLAG: --contention-profiling="false"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095700    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095703    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095706    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095708    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095713    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095717    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095720    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095723    2579 flags.go:64] FLAG: --enable-load-reader="false"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095725    2579 flags.go:64] FLAG: --enable-server="true"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095729    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095733    2579 flags.go:64] FLAG: --event-burst="100"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095735    2579 flags.go:64] FLAG: --event-qps="50"
Jul 27 19:39:32.101606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095738    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095741    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095744    2579 flags.go:64] FLAG: --eviction-hard=""
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095747    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095751    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095754    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095757    2579 flags.go:64] FLAG: --eviction-soft=""
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095760    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095762    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095765    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095768    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095770    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095773    2579 flags.go:64] FLAG: --fail-swap-on="true"
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095776    2579 flags.go:64] FLAG: --feature-gates=""
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095779    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095782    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095785    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095788    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095791    2579 flags.go:64] FLAG: --healthz-port="10248"
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095793    2579 flags.go:64] FLAG: --help="false"
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095796    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-140-36.ec2.internal"
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095799    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095802    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 27 19:39:32.102213 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095804    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095808    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095812    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095815    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095817    2579 flags.go:64] FLAG: --image-service-endpoint=""
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095820    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095823    2579 flags.go:64] FLAG: --kube-api-burst="100"
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095826    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095829    2579 flags.go:64] FLAG: --kube-api-qps="50"
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095831    2579 flags.go:64] FLAG: --kube-reserved=""
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095834    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095837    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095840    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095843    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095846    2579 flags.go:64] FLAG: --lock-file=""
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095849    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095865    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095868    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095873    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095876    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095879    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095882    2579 flags.go:64] FLAG: --logging-format="text"
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095884    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095888    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 27 19:39:32.102774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095890    2579 flags.go:64] FLAG: --manifest-url=""
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095893    2579 flags.go:64] FLAG: --manifest-url-header=""
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095897    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095900    2579 flags.go:64] FLAG: --max-open-files="1000000"
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095904    2579 flags.go:64] FLAG: --max-pods="110"
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095906    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095909    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095912    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095914    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095917    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095921    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095924    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095930    2579 flags.go:64] FLAG: --node-status-max-images="50"
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095934    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095936    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095939    2579 flags.go:64] FLAG: --pod-cidr=""
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095942    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095947    2579 flags.go:64] FLAG: --pod-manifest-path=""
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095950    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095953    2579 flags.go:64] FLAG: --pods-per-core="0"
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095956    2579 flags.go:64] FLAG: --port="10250"
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095960    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095963    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00043bbb69fc0e8df"
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095966    2579 flags.go:64] FLAG: --qos-reserved=""
Jul 27 19:39:32.103399 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095969    2579 flags.go:64] FLAG: --read-only-port="10255"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095972    2579 flags.go:64] FLAG: --register-node="true"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095975    2579 flags.go:64] FLAG: --register-with-taints=""
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095978    2579 flags.go:64] FLAG: --registry-burst="10"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095981    2579 flags.go:64] FLAG: --registry-qps="5"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095984    2579 flags.go:64] FLAG: --reserved-cpus=""
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095986    2579 flags.go:64] FLAG: --reserved-memory=""
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095990    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095993    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095995    2579 flags.go:64] FLAG: --rotate-certificates="false"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.095998    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096001    2579 flags.go:64] FLAG: --runonce="false"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096004    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096007    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096009    2579 flags.go:64] FLAG: --seccomp-default="false"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096012    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096015    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096017    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096020    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096026    2579 flags.go:64] FLAG: --storage-driver-password="root"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096029    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096032    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096034    2579 flags.go:64] FLAG: --storage-driver-user="root"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096037    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096040    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 27 19:39:32.104043 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096043    2579 flags.go:64] FLAG: --system-cgroups=""
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096046    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096051    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096054    2579 flags.go:64] FLAG: --tls-cert-file=""
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096058    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096062    2579 flags.go:64] FLAG: --tls-min-version=""
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096065    2579 flags.go:64] FLAG: --tls-private-key-file=""
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096068    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096071    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096074    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096077    2579 flags.go:64] FLAG: --v="2"
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096081    2579 flags.go:64] FLAG: --version="false"
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096084    2579 flags.go:64] FLAG: --vmodule=""
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096088    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.096091    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096208    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096212    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096215    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096218    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096221    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096224    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096227    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096229    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:39:32.104658 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096232    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:39:32.105221 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096235    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:39:32.105221 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096237    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:39:32.105221 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096241    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:39:32.105221 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096244    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:39:32.105221 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096246    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:39:32.105221 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096249    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:39:32.105221 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096252    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:39:32.105221 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096254    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:39:32.105221 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096257    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:39:32.105221 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096260    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:39:32.105221 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096262    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:39:32.105221 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096265    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:39:32.105221 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096268    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:39:32.105221 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096272    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:39:32.105221 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096274    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:39:32.105221 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096277    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:39:32.105221 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096279    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:39:32.105221 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096282    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:39:32.105221 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096284    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:39:32.105221 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096287    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:39:32.105720 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096289    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:39:32.105720 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096292    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:39:32.105720 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096294    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:39:32.105720 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096297    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:39:32.105720 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096299    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:39:32.105720 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096302    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:39:32.105720 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096305    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:39:32.105720 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096307    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:39:32.105720 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096310    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:39:32.105720 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096312    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:39:32.105720 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096315    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:39:32.105720 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096317    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:39:32.105720 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096320    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:39:32.105720 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096322    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:39:32.105720 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096326    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:39:32.105720 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096328    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:39:32.105720 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096332    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:39:32.105720 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096335    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:39:32.105720 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096338    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:39:32.105720 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096341    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:39:32.106216 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096344    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:39:32.106216 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096346    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:39:32.106216 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096349    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:39:32.106216 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096352    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:39:32.106216 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096356    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:39:32.106216 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096358    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:39:32.106216 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096360    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:39:32.106216 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096363    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:39:32.106216 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096365    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:39:32.106216 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096368    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:39:32.106216 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096370    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:39:32.106216 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096373    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:39:32.106216 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096375    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:39:32.106216 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096378    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:39:32.106216 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096380    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:39:32.106216 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096383    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:39:32.106216 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096385    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:39:32.106216 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096387    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:39:32.106216 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096390    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:39:32.106216 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096393    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:39:32.106705 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096395    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:39:32.106705 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096398    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:39:32.106705 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096400    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:39:32.106705 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096402    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:39:32.106705 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096405    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:39:32.106705 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096409    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:39:32.106705 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096413    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:39:32.106705 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096417    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:39:32.106705 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096420    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:39:32.106705 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096423    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:39:32.106705 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096426    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:39:32.106705 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096428    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:39:32.106705 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096431    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:39:32.106705 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096434    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:39:32.106705 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096437    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:39:32.106705 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096440    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:39:32.106705 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096443    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:39:32.106705 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096446    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:39:32.106705 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096448    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:39:32.107193 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096451    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:39:32.107193 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096453    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:39:32.107193 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096455    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:39:32.107193 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096458    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:39:32.107193 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096461    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:39:32.107193 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096463    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:39:32.107193 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096466    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:39:32.107193 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096468    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:39:32.107193 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096471    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:39:32.107193 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096473    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:39:32.107193 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.096476    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:39:32.107193 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.097465    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 19:39:32.107193 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.105050    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 27 19:39:32.107193 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.105076    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 27 19:39:32.107193 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.105090    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 27 19:39:32.107570 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.105099    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 27 19:39:32.107570 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105160    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:39:32.107570 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105165    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:39:32.107570 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105168    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:39:32.107570 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105171    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:39:32.107570 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105174    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:39:32.107570 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105177    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:39:32.107570 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105180    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:39:32.107570 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105183    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:39:32.107570 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105187    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:39:32.107570 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105189    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:39:32.107570 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105192    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:39:32.107570 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105194    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:39:32.107570 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105197    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:39:32.107570 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105200    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:39:32.107570 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105202    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:39:32.107570 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105205    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:39:32.107570 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105208    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:39:32.107570 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105210    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:39:32.108044 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105213    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:39:32.108044 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105215    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:39:32.108044 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105218    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:39:32.108044 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105220    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:39:32.108044 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105223    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:39:32.108044 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105225    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:39:32.108044 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105228    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:39:32.108044 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105230    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:39:32.108044 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105233    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:39:32.108044 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105236    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:39:32.108044 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105239    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:39:32.108044 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105249    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:39:32.108044 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105252    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:39:32.108044 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105255    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:39:32.108044 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105257    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:39:32.108044 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105260    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:39:32.108044 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105262    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:39:32.108044 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105265    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:39:32.108044 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105268    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:39:32.108554 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105271    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:39:32.108554 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105273    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:39:32.108554 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105276    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:39:32.108554 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105278    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:39:32.108554 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105281    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:39:32.108554 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105284    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:39:32.108554 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105286    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:39:32.108554 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105289    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:39:32.108554 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105291    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:39:32.108554 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105293    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:39:32.108554 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105296    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:39:32.108554 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105298    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:39:32.108554 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105301    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:39:32.108554 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105303    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:39:32.108554 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105306    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:39:32.108554 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105308    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:39:32.108554 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105310    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:39:32.108554 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105313    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:39:32.108554 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105315    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:39:32.108554 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105318    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:39:32.109092 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105320    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:39:32.109092 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105322    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:39:32.109092 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105325    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:39:32.109092 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105327    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:39:32.109092 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105330    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:39:32.109092 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105340    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:39:32.109092 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105344    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:39:32.109092 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105346    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:39:32.109092 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105349    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:39:32.109092 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105351    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:39:32.109092 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105354    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:39:32.109092 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105356    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:39:32.109092 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105359    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:39:32.109092 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105361    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:39:32.109092 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105363    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:39:32.109092 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105366    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:39:32.109092 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105368    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:39:32.109092 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105370    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:39:32.109092 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105373    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:39:32.109092 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105375    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:39:32.109630 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105378    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:39:32.109630 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105380    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:39:32.109630 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105382    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:39:32.109630 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105385    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:39:32.109630 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105388    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:39:32.109630 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105390    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:39:32.109630 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105393    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:39:32.109630 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105395    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:39:32.109630 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105398    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:39:32.109630 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105400    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:39:32.109630 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105402    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:39:32.109630 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105405    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:39:32.109630 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105407    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:39:32.109630 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105410    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:39:32.109630 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105412    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:39:32.109630 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105416    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:39:32.109630 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105419    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:39:32.109630 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105422    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:39:32.109630 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105430    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:39:32.110124 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105433    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:39:32.110124 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105436    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:39:32.110124 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105439    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:39:32.110124 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.105444    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 19:39:32.110124 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105588    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:39:32.110124 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105593    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:39:32.110124 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105596    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:39:32.110124 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105599    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:39:32.110124 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105601    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:39:32.110124 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105604    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:39:32.110124 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105606    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:39:32.110124 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105608    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:39:32.110124 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105611    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:39:32.110124 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105613    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:39:32.110124 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105616    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:39:32.110514 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105618    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:39:32.110514 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105620    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:39:32.110514 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105623    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:39:32.110514 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105625    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:39:32.110514 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105628    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:39:32.110514 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105630    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:39:32.110514 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105634    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:39:32.110514 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105637    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:39:32.110514 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105640    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:39:32.110514 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105643    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:39:32.110514 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105645    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:39:32.110514 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105648    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:39:32.110514 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105650    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:39:32.110514 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105653    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:39:32.110514 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105656    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:39:32.110514 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105658    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:39:32.110514 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105661    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:39:32.110514 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105668    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:39:32.110514 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105672    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:39:32.111010 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105674    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:39:32.111010 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105676    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:39:32.111010 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105679    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:39:32.111010 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105681    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:39:32.111010 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105683    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:39:32.111010 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105686    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:39:32.111010 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105688    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:39:32.111010 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105690    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:39:32.111010 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105693    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:39:32.111010 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105695    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:39:32.111010 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105697    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:39:32.111010 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105700    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:39:32.111010 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105702    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:39:32.111010 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105705    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:39:32.111010 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105707    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:39:32.111010 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105711    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:39:32.111010 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105714    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:39:32.111010 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105716    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:39:32.111010 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105719    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:39:32.111478 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105722    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:39:32.111478 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105724    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:39:32.111478 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105727    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:39:32.111478 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105730    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:39:32.111478 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105733    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:39:32.111478 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105736    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:39:32.111478 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105738    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:39:32.111478 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105741    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:39:32.111478 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105743    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:39:32.111478 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105746    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:39:32.111478 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105748    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:39:32.111478 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105751    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:39:32.111478 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105759    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:39:32.111478 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105761    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:39:32.111478 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105764    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:39:32.111478 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105766    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:39:32.111478 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105769    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:39:32.111478 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105771    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:39:32.111478 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105774    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:39:32.111978 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105776    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:39:32.111978 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105778    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:39:32.111978 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105781    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:39:32.111978 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105783    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:39:32.111978 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105785    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:39:32.111978 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105788    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:39:32.111978 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105790    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:39:32.111978 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105792    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:39:32.111978 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105795    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:39:32.111978 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105797    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:39:32.111978 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105800    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:39:32.111978 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105802    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:39:32.111978 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105804    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:39:32.111978 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105807    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:39:32.111978 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105809    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:39:32.111978 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105811    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:39:32.111978 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105814    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:39:32.111978 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105816    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:39:32.111978 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105818    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:39:32.111978 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105821    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:39:32.112450 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105823    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:39:32.112450 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105827    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:39:32.112450 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105830    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:39:32.112450 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105832    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:39:32.112450 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105835    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:39:32.112450 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105837    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:39:32.112450 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105845    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:39:32.112450 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105848    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:39:32.112450 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105851    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:39:32.112450 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105869    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:39:32.112450 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:32.105873    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:39:32.112450 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.105879    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 19:39:32.112450 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.105990    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 27 19:39:32.112450 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.108436    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 27 19:39:32.112808 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.110128    2579 server.go:1019] "Starting client certificate rotation"
Jul 27 19:39:32.112808 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.110232    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 19:39:32.112808 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.110269    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 19:39:32.114925 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.114906    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 19:39:32.117741 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.117726    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 19:39:32.131737 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.131721    2579 log.go:25] "Validated CRI v1 runtime API"
Jul 27 19:39:32.134400 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.134385    2579 log.go:25] "Validated CRI v1 image API"
Jul 27 19:39:32.135211 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.135196    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 27 19:39:32.142789 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.142770    2579 fs.go:135] Filesystem UUIDs: map[73f2449a-a695-45a1-8c31-79b08ec2b9a5:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b6b29854-353d-445c-b9c0-dee0d3d2dcc7:/dev/nvme0n1p3]
Jul 27 19:39:32.142881 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.142787    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 27 19:39:32.143600 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.143581    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 19:39:32.148587 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.148456    2579 manager.go:216] Machine: {Timestamp:2026-07-27 19:39:32.147050713 +0000 UTC m=+0.376292903 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3108342 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d5f8df906341a6ca88833a59e62c1 SystemUUID:ec2d5f8d-f906-341a-6ca8-8833a59e62c1 BootID:a5d50954-135e-424f-b17c-5fd5879b7b8f Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3f:a1:d3:a1:eb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3f:a1:d3:a1:eb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:05:cd:d2:28:f9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 27 19:39:32.148587 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.148578    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 27 19:39:32.148713 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.148649    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 27 19:39:32.150703 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.150674    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 27 19:39:32.150827 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.150707    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-36.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 27 19:39:32.151559 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.151548    2579 topology_manager.go:138] "Creating topology manager with none policy"
Jul 27 19:39:32.151596 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.151562    2579 container_manager_linux.go:309] "Creating device plugin manager"
Jul 27 19:39:32.151596 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.151570    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 19:39:32.152243 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.152233    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 19:39:32.153522 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.153512    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 27 19:39:32.156412 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.156401    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 19:39:32.156539 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.156529    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 27 19:39:32.156577 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.156558    2579 kubelet.go:486] "Attempting to sync node with API server"
Jul 27 19:39:32.156577 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.156571    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 27 19:39:32.156637 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.156587    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 27 19:39:32.156637 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.156634    2579 kubelet.go:398] "Adding apiserver pod source"
Jul 27 19:39:32.156700 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.156651    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 27 19:39:32.157163 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.157146    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 19:39:32.157247 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.157182    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 19:39:32.160741 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.160708    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 27 19:39:32.163271 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.163256    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 27 19:39:32.163347 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.163286    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 27 19:39:32.165198 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.165175    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 27 19:39:32.165198 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.165193    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 27 19:39:32.165198 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.165199    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 27 19:39:32.165312 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.165205    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 27 19:39:32.165312 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.165213    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 27 19:39:32.165312 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.165218    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 27 19:39:32.165312 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.165226    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 27 19:39:32.165312 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.165231    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 27 19:39:32.165312 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.165237    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 27 19:39:32.165312 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.165242    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 27 19:39:32.165312 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.165250    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 27 19:39:32.165312 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.165255    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 27 19:39:32.166173 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.166163    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 27 19:39:32.166173 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.166172    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 27 19:39:32.167977 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.167960    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-36.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:39:32.168033 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.167998    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-36.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 19:39:32.168033 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.167998    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 19:39:32.169134 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.169122    2579 server.go:1268] "Started kubelet"
Jul 27 19:39:32.169263 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.169228    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 27 19:39:32.169799 ip-10-0-140-36 systemd[1]: Started Kubernetes Kubelet.
Jul 27 19:39:32.170317 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.169886    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 27 19:39:32.170317 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.169946    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 27 19:39:32.171236 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.171221    2579 server.go:310] "Adding debug handlers to kubelet server"
Jul 27 19:39:32.172053 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.172036    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 27 19:39:32.178694 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.178678    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 27 19:39:32.181419 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.181391    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 27 19:39:32.181419 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.181405    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 27 19:39:32.181555 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.181490    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-36.ec2.internal\" not found"
Jul 27 19:39:32.182194 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.182175    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 27 19:39:32.182277 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.182187    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 27 19:39:32.182277 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.182210    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 27 19:39:32.182386 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.182343    2579 reconstruct.go:97] "Volume reconstruction finished"
Jul 27 19:39:32.182386 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.182351    2579 reconciler.go:29] "Reconciler: start to sync state"
Jul 27 19:39:32.182509 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.182486    2579 factory.go:55] Registering systemd factory
Jul 27 19:39:32.182562 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.182527    2579 factory.go:223] Registration of the systemd container factory successfully
Jul 27 19:39:32.182758 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.182740    2579 factory.go:153] Registering CRI-O factory
Jul 27 19:39:32.182758 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.182756    2579 factory.go:223] Registration of the crio container factory successfully
Jul 27 19:39:32.182927 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.182883    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 27 19:39:32.182927 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.182911    2579 factory.go:103] Registering Raw factory
Jul 27 19:39:32.182927 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.182927    2579 manager.go:1196] Started watching for new ooms in manager
Jul 27 19:39:32.183415 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.183401    2579 manager.go:349] Starting recovery of all containers
Jul 27 19:39:32.183979 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.182758    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-36.ec2.internal.18c63cd15adac0ce  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-36.ec2.internal,UID:ip-10-0-140-36.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-36.ec2.internal,},FirstTimestamp:2026-07-27 19:39:32.169097422 +0000 UTC m=+0.398339611,LastTimestamp:2026-07-27 19:39:32.169097422 +0000 UTC m=+0.398339611,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-36.ec2.internal,}"
Jul 27 19:39:32.195743 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.195723    2579 manager.go:354] Recovery completed
Jul 27 19:39:32.196279 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.196245    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 27 19:39:32.196349 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.196299    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-36.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 27 19:39:32.200534 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.200513    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-94s72"
Jul 27 19:39:32.201125 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.201113    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:39:32.202827 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.202810    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:39:32.202890 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.202840    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:39:32.202890 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.202868    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:39:32.203365 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.203352    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 27 19:39:32.203445 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.203365    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 27 19:39:32.203445 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.203382    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 19:39:32.203445 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.203387    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:39:32.204024 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.204007    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:39:32.204112 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.204034    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:39:32.204112 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.204046    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:39:32.206197 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.206185    2579 policy_none.go:49] "None policy: Start"
Jul 27 19:39:32.206255 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.206201    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 27 19:39:32.206255 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.206211    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 27 19:39:32.208126 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.208113    2579 policy_none.go:47] "Start"
Jul 27 19:39:32.208199 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.208185    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-94s72"
Jul 27 19:39:32.214212 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.214056    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-36.ec2.internal.18c63cd15cdd6994  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-36.ec2.internal,UID:ip-10-0-140-36.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-36.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-36.ec2.internal,},FirstTimestamp:2026-07-27 19:39:32.202826132 +0000 UTC m=+0.432068321,LastTimestamp:2026-07-27 19:39:32.202826132 +0000 UTC m=+0.432068321,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-36.ec2.internal,}"
Jul 27 19:39:32.236968 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.236956    2579 manager.go:337] "Starting Device Plugin manager"
Jul 27 19:39:32.237028 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.236986    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 27 19:39:32.237028 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.236996    2579 server.go:87] "Starting device plugin registration server"
Jul 27 19:39:32.237223 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.237212    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 27 19:39:32.237262 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.237226    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 27 19:39:32.237687 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.237346    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 27 19:39:32.237687 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.237428    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 27 19:39:32.237687 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.237439    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 27 19:39:32.237939 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.237923    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 27 19:39:32.238024 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.237974    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-36.ec2.internal\" not found"
Jul 27 19:39:32.253825 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.253801    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 27 19:39:32.265741 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.255042    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 27 19:39:32.265741 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.255088    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 27 19:39:32.265741 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.255111    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 27 19:39:32.265741 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.255150    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 27 19:39:32.265741 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.257056    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 19:39:32.338298 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.338236    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:39:32.339063 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.339046    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:39:32.339145 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.339075    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:39:32.339145 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.339086    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:39:32.339145 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.339110    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-36.ec2.internal"
Jul 27 19:39:32.348446 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.348430    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-36.ec2.internal"
Jul 27 19:39:32.348499 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.348450    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-36.ec2.internal\": node \"ip-10-0-140-36.ec2.internal\" not found"
Jul 27 19:39:32.356131 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.356113    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-36.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-36.ec2.internal"]
Jul 27 19:39:32.356194 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.356161    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:39:32.356838 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.356824    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:39:32.356921 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.356849    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:39:32.356921 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.356880    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:39:32.359291 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.359278    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:39:32.359417 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.359403    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-36.ec2.internal"
Jul 27 19:39:32.359473 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.359434    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:39:32.361312 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.361298    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:39:32.361365 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.361302    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:39:32.361365 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.361316    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:39:32.361365 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.361347    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:39:32.361365 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.361353    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:39:32.361365 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.361360    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:39:32.363641 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.363625    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-36.ec2.internal"
Jul 27 19:39:32.363722 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.363650    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:39:32.364324 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.364309    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:39:32.364407 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.364335    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:39:32.364407 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.364345    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:39:32.378328 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.378311    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-36.ec2.internal\" not found"
Jul 27 19:39:32.383515 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.383499    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/65272394e66c19ba2f850aea94ec8a01-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-36.ec2.internal\" (UID: \"65272394e66c19ba2f850aea94ec8a01\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-36.ec2.internal"
Jul 27 19:39:32.383574 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.383523    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65272394e66c19ba2f850aea94ec8a01-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-36.ec2.internal\" (UID: \"65272394e66c19ba2f850aea94ec8a01\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-36.ec2.internal"
Jul 27 19:39:32.383574 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.383538    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8f425195a1bad05a939cf1834470c7d1-config\") pod \"kube-apiserver-proxy-ip-10-0-140-36.ec2.internal\" (UID: \"8f425195a1bad05a939cf1834470c7d1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-36.ec2.internal"
Jul 27 19:39:32.390377 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.390358    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-36.ec2.internal\" not found" node="ip-10-0-140-36.ec2.internal"
Jul 27 19:39:32.394717 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.394701    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-36.ec2.internal\" not found" node="ip-10-0-140-36.ec2.internal"
Jul 27 19:39:32.479318 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.479299    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-36.ec2.internal\" not found"
Jul 27 19:39:32.484623 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.484607    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8f425195a1bad05a939cf1834470c7d1-config\") pod \"kube-apiserver-proxy-ip-10-0-140-36.ec2.internal\" (UID: \"8f425195a1bad05a939cf1834470c7d1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-36.ec2.internal"
Jul 27 19:39:32.484697 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.484636    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/65272394e66c19ba2f850aea94ec8a01-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-36.ec2.internal\" (UID: \"65272394e66c19ba2f850aea94ec8a01\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-36.ec2.internal"
Jul 27 19:39:32.484697 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.484660    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65272394e66c19ba2f850aea94ec8a01-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-36.ec2.internal\" (UID: \"65272394e66c19ba2f850aea94ec8a01\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-36.ec2.internal"
Jul 27 19:39:32.484795 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.484698    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65272394e66c19ba2f850aea94ec8a01-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-36.ec2.internal\" (UID: \"65272394e66c19ba2f850aea94ec8a01\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-36.ec2.internal"
Jul 27 19:39:32.484795 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.484712    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8f425195a1bad05a939cf1834470c7d1-config\") pod \"kube-apiserver-proxy-ip-10-0-140-36.ec2.internal\" (UID: \"8f425195a1bad05a939cf1834470c7d1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-36.ec2.internal"
Jul 27 19:39:32.484795 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.484718    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/65272394e66c19ba2f850aea94ec8a01-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-36.ec2.internal\" (UID: \"65272394e66c19ba2f850aea94ec8a01\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-36.ec2.internal"
Jul 27 19:39:32.579801 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.579772    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-36.ec2.internal\" not found"
Jul 27 19:39:32.680595 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.680543    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-36.ec2.internal\" not found"
Jul 27 19:39:32.694034 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.694015    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-36.ec2.internal"
Jul 27 19:39:32.699710 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:32.699407    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-36.ec2.internal"
Jul 27 19:39:32.780800 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.780771    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-36.ec2.internal\" not found"
Jul 27 19:39:32.881298 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.881279    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-36.ec2.internal\" not found"
Jul 27 19:39:32.981759 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:32.981710    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-36.ec2.internal\" not found"
Jul 27 19:39:33.082137 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:33.082108    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-36.ec2.internal\" not found"
Jul 27 19:39:33.110496 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:33.110479    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 27 19:39:33.110834 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:33.110601    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 27 19:39:33.182169 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:33.182148    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-36.ec2.internal\" not found"
Jul 27 19:39:33.214945 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:33.214911    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-26 19:34:32 +0000 UTC" deadline="2028-04-15 18:11:28.36314901 +0000 UTC"
Jul 27 19:39:33.214945 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:33.214939    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15070h31m55.148214231s"
Jul 27 19:39:33.233057 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:33.233013    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:39:33.234066 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:33.234049    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:39:33.234162 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:33.234082    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:39:33.234162 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:33.234091    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:39:33.282435 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:33.282405    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-36.ec2.internal\" not found"
Jul 27 19:39:33.323106 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:33.323082    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 19:39:33.382777 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:33.382760    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-36.ec2.internal\" not found"
Jul 27 19:39:33.386759 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:33.386739    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65272394e66c19ba2f850aea94ec8a01.slice/crio-66857ca378b0398e7cdd89af1413cbb2cc6599c7c8d69f6cd1d6a947ede50918 WatchSource:0}: Error finding container 66857ca378b0398e7cdd89af1413cbb2cc6599c7c8d69f6cd1d6a947ede50918: Status 404 returned error can't find the container with id 66857ca378b0398e7cdd89af1413cbb2cc6599c7c8d69f6cd1d6a947ede50918
Jul 27 19:39:33.387024 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:33.387003    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f425195a1bad05a939cf1834470c7d1.slice/crio-7a582382b564f8384b073bdda008c3d4470414d4ec3bd7cb5d402d337fff03a3 WatchSource:0}: Error finding container 7a582382b564f8384b073bdda008c3d4470414d4ec3bd7cb5d402d337fff03a3: Status 404 returned error can't find the container with id 7a582382b564f8384b073bdda008c3d4470414d4ec3bd7cb5d402d337fff03a3
Jul 27 19:39:33.391234 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:33.391220    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 19:39:33.419112 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:33.419090    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 19:39:33.483730 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:33.483688    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-36.ec2.internal\" not found"
Jul 27 19:39:33.584191 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:33.584162    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-36.ec2.internal\" not found"
Jul 27 19:39:33.684931 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:33.684903    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-36.ec2.internal\" not found"
Jul 27 19:39:33.751665 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:33.751611    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 19:39:33.782280 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:33.782260    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-36.ec2.internal"
Jul 27 19:39:33.798616 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:33.798595    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 19:39:33.799687 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:33.799671    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-36.ec2.internal"
Jul 27 19:39:33.808764 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:33.808748    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 19:39:34.157985 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.157913    2579 apiserver.go:52] "Watching apiserver"
Jul 27 19:39:34.164291 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.164266    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 27 19:39:34.165841 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.165808    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-7gt55","kube-system/kube-apiserver-proxy-ip-10-0-140-36.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v","openshift-cluster-node-tuning-operator/tuned-7jxkt","openshift-multus/multus-additional-cni-plugins-lwcg2","openshift-multus/network-metrics-daemon-sdk4x","openshift-network-operator/iptables-alerter-96x6d","kube-system/konnectivity-agent-tktlt","openshift-dns/node-resolver-grpbk","openshift-image-registry/node-ca-m2pzk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-36.ec2.internal","openshift-multus/multus-nqrzj","openshift-network-diagnostics/network-check-target-fpbzl","openshift-ovn-kubernetes/ovnkube-node-nwhgp"]
Jul 27 19:39:34.168926 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.168895    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tktlt"
Jul 27 19:39:34.171603 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.171373    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 27 19:39:34.171709 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.171631    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 27 19:39:34.171709 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.171631    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-l7tx2\""
Jul 27 19:39:34.172223 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.172184    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m2pzk"
Jul 27 19:39:34.174839 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.174106    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-x9fcl\""
Jul 27 19:39:34.174839 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.174126    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 27 19:39:34.174839 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.174236    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 27 19:39:34.174839 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.174306    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 27 19:39:34.175572 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.175108    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.177339 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.177314    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:34.177427 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:34.177385    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gt55" podUID="6407e54e-d37a-4344-91d3-4494c81246c4"
Jul 27 19:39:34.178125 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.178100    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cqw8x\""
Jul 27 19:39:34.178370 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.178351    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:39:34.178618 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.178602    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 27 19:39:34.180551 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.179858    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.182232 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.182083    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 27 19:39:34.182232 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.182101    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 27 19:39:34.182232 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.182129    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 27 19:39:34.182232 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.182184    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 27 19:39:34.182479 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.182103    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 27 19:39:34.182750 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.182732    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-b7t4v\""
Jul 27 19:39:34.182833 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.182800    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 27 19:39:34.183265 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.183249    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:34.183359 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:34.183337    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdk4x" podUID="74f89d7b-f93b-4dc3-9e18-41cf466d90bd"
Jul 27 19:39:34.186824 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.185947    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-96x6d"
Jul 27 19:39:34.188202 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.188036    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 27 19:39:34.188202 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.188055    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 27 19:39:34.188469 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.188440    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.188564 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.188527    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:39:34.188748 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.188730    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mphh9\""
Jul 27 19:39:34.190545 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.190391    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 27 19:39:34.191012 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.190715    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hl6mg\""
Jul 27 19:39:34.191012 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.190981    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 27 19:39:34.191249 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.191223    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 27 19:39:34.195076 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.194193    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-grpbk"
Jul 27 19:39:34.195462 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195156    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-etc-modprobe-d\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.195462 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195199    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-run\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.195462 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195226    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-sys\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.195462 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195248    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-host\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.195462 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195294    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret\") pod \"global-pull-secret-syncer-7gt55\" (UID: \"6407e54e-d37a-4344-91d3-4494c81246c4\") " pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:34.195462 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195331    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a76e8a73-6dac-43e0-92ec-6269ccd13593-system-cni-dir\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.195462 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195348    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a76e8a73-6dac-43e0-92ec-6269ccd13593-cni-binary-copy\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.195462 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195366    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgbgj\" (UniqueName: \"kubernetes.io/projected/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-kube-api-access-qgbgj\") pod \"network-metrics-daemon-sdk4x\" (UID: \"74f89d7b-f93b-4dc3-9e18-41cf466d90bd\") " pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:34.195462 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195416    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6407e54e-d37a-4344-91d3-4494c81246c4-dbus\") pod \"global-pull-secret-syncer-7gt55\" (UID: \"6407e54e-d37a-4344-91d3-4494c81246c4\") " pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:34.195971 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195473    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-etc-sysctl-conf\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.195971 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195504    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5f94ff14-c2a5-4b14-b958-d56ada45996c-etc-tuned\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.195971 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195529    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg5fn\" (UniqueName: \"kubernetes.io/projected/5f94ff14-c2a5-4b14-b958-d56ada45996c-kube-api-access-fg5fn\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.195971 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195588    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91f1914a-7ad4-45a2-949f-75375e34df1f-host\") pod \"node-ca-m2pzk\" (UID: \"91f1914a-7ad4-45a2-949f-75375e34df1f\") " pod="openshift-image-registry/node-ca-m2pzk"
Jul 27 19:39:34.195971 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195621    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91f1914a-7ad4-45a2-949f-75375e34df1f-serviceca\") pod \"node-ca-m2pzk\" (UID: \"91f1914a-7ad4-45a2-949f-75375e34df1f\") " pod="openshift-image-registry/node-ca-m2pzk"
Jul 27 19:39:34.195971 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195666    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxt65\" (UniqueName: \"kubernetes.io/projected/91f1914a-7ad4-45a2-949f-75375e34df1f-kube-api-access-sxt65\") pod \"node-ca-m2pzk\" (UID: \"91f1914a-7ad4-45a2-949f-75375e34df1f\") " pod="openshift-image-registry/node-ca-m2pzk"
Jul 27 19:39:34.195971 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195699    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f94ff14-c2a5-4b14-b958-d56ada45996c-tmp\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.195971 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195723    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6407e54e-d37a-4344-91d3-4494c81246c4-kubelet-config\") pod \"global-pull-secret-syncer-7gt55\" (UID: \"6407e54e-d37a-4344-91d3-4494c81246c4\") " pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:34.195971 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195765    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a76e8a73-6dac-43e0-92ec-6269ccd13593-cnibin\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.195971 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195792    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a76e8a73-6dac-43e0-92ec-6269ccd13593-os-release\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.195971 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195834    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a76e8a73-6dac-43e0-92ec-6269ccd13593-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.195971 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195886    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a76e8a73-6dac-43e0-92ec-6269ccd13593-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.195971 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195951    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv77p\" (UniqueName: \"kubernetes.io/projected/a76e8a73-6dac-43e0-92ec-6269ccd13593-kube-api-access-sv77p\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.196544 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.195984    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/967ad4b2-9b0f-4654-be1b-8bcceee7f7c3-agent-certs\") pod \"konnectivity-agent-tktlt\" (UID: \"967ad4b2-9b0f-4654-be1b-8bcceee7f7c3\") " pod="kube-system/konnectivity-agent-tktlt"
Jul 27 19:39:34.196544 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.196012    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-lib-modules\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.196544 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.196054    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a76e8a73-6dac-43e0-92ec-6269ccd13593-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.196544 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.196088    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs\") pod \"network-metrics-daemon-sdk4x\" (UID: \"74f89d7b-f93b-4dc3-9e18-41cf466d90bd\") " pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:34.196544 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.196122    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/967ad4b2-9b0f-4654-be1b-8bcceee7f7c3-konnectivity-ca\") pod \"konnectivity-agent-tktlt\" (UID: \"967ad4b2-9b0f-4654-be1b-8bcceee7f7c3\") " pod="kube-system/konnectivity-agent-tktlt"
Jul 27 19:39:34.196544 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.196174    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-etc-kubernetes\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.196544 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.196203    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-var-lib-kubelet\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.196544 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.196221    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-etc-sysconfig\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.196544 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.196248    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-etc-sysctl-d\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.196544 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.196272    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-etc-systemd\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.198751 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.198265    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 27 19:39:34.198751 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.198614    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 27 19:39:34.199562 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.198293    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-fn5xb\""
Jul 27 19:39:34.199673 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.199626    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.201477 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.201379    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 19:39:34.202732 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.201815    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 27 19:39:34.202732 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.202045    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-sxrpj\""
Jul 27 19:39:34.203809 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.203102    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:39:34.203809 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:34.203210    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fpbzl" podUID="f9f7d637-863f-4a7f-8327-ce46c5076d4d"
Jul 27 19:39:34.206972 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.206636    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.208553 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.208530    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 27 19:39:34.208806 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.208790    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 27 19:39:34.209338 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.209319    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 27 19:39:34.209776 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.209758    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tqzq6\""
Jul 27 19:39:34.210021 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.210006    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 27 19:39:34.210202 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.210172    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 27 19:39:34.210653 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.210415    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 27 19:39:34.223660 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.223642    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8gtbh"
Jul 27 19:39:34.232467 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.232450    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 19:39:34.236444 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.236424    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8gtbh"
Jul 27 19:39:34.259748 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.259696    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-36.ec2.internal" event={"ID":"65272394e66c19ba2f850aea94ec8a01","Type":"ContainerStarted","Data":"66857ca378b0398e7cdd89af1413cbb2cc6599c7c8d69f6cd1d6a947ede50918"}
Jul 27 19:39:34.260917 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.260892    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-36.ec2.internal" event={"ID":"8f425195a1bad05a939cf1834470c7d1","Type":"ContainerStarted","Data":"7a582382b564f8384b073bdda008c3d4470414d4ec3bd7cb5d402d337fff03a3"}
Jul 27 19:39:34.283758 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.283719    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 27 19:39:34.297522 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.297482    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f94ff14-c2a5-4b14-b958-d56ada45996c-tmp\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.297650 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.297527    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a76e8a73-6dac-43e0-92ec-6269ccd13593-cnibin\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.297650 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.297554    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a76e8a73-6dac-43e0-92ec-6269ccd13593-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.297650 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.297580    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/967ad4b2-9b0f-4654-be1b-8bcceee7f7c3-agent-certs\") pod \"konnectivity-agent-tktlt\" (UID: \"967ad4b2-9b0f-4654-be1b-8bcceee7f7c3\") " pod="kube-system/konnectivity-agent-tktlt"
Jul 27 19:39:34.297650 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.297607    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-hostroot\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.297650 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.297642    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a76e8a73-6dac-43e0-92ec-6269ccd13593-cnibin\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.297933 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.297632    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-host-slash\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.297933 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.297726    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8ck5\" (UniqueName: \"kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5\") pod \"network-check-target-fpbzl\" (UID: \"f9f7d637-863f-4a7f-8327-ce46c5076d4d\") " pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:39:34.297933 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.297759    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a76e8a73-6dac-43e0-92ec-6269ccd13593-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.297933 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.297787    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/967ad4b2-9b0f-4654-be1b-8bcceee7f7c3-konnectivity-ca\") pod \"konnectivity-agent-tktlt\" (UID: \"967ad4b2-9b0f-4654-be1b-8bcceee7f7c3\") " pod="kube-system/konnectivity-agent-tktlt"
Jul 27 19:39:34.297933 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.297829    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-run-openvswitch\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.297933 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.297861    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-host-run-ovn-kubernetes\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.297933 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.297914    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a2601d76-e153-4372-a871-3e20499e774a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rn87v\" (UID: \"a2601d76-e153-4372-a871-3e20499e774a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.297933 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.297890    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 27 19:39:34.298353 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.297945    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfdvj\" (UniqueName: \"kubernetes.io/projected/73b8094b-b342-4acd-9323-a5e71e0c3dfb-kube-api-access-cfdvj\") pod \"node-resolver-grpbk\" (UID: \"73b8094b-b342-4acd-9323-a5e71e0c3dfb\") " pod="openshift-dns/node-resolver-grpbk"
Jul 27 19:39:34.298353 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.297974    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-etc-kubernetes\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.298353 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298006    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-etc-systemd\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.298353 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298031    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-host-run-netns\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.298353 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298074    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbkhn\" (UniqueName: \"kubernetes.io/projected/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-kube-api-access-hbkhn\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.298353 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298100    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/580a1b09-1df6-48b5-9b0d-d72cb6806b06-host-slash\") pod \"iptables-alerter-96x6d\" (UID: \"580a1b09-1df6-48b5-9b0d-d72cb6806b06\") " pod="openshift-network-operator/iptables-alerter-96x6d"
Jul 27 19:39:34.298353 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298131    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-cni-binary-copy\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.298353 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298174    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-multus-socket-dir-parent\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.298353 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298211    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-host-run-netns\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.298353 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298238    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-host-var-lib-kubelet\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.298353 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298265    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.298353 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298295    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-etc-modprobe-d\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.298353 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298335    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-host\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.299133 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298361    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a76e8a73-6dac-43e0-92ec-6269ccd13593-system-cni-dir\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.299133 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298387    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a76e8a73-6dac-43e0-92ec-6269ccd13593-cni-binary-copy\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.299133 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298415    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgbgj\" (UniqueName: \"kubernetes.io/projected/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-kube-api-access-qgbgj\") pod \"network-metrics-daemon-sdk4x\" (UID: \"74f89d7b-f93b-4dc3-9e18-41cf466d90bd\") " pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:34.299133 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298449    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxt65\" (UniqueName: \"kubernetes.io/projected/91f1914a-7ad4-45a2-949f-75375e34df1f-kube-api-access-sxt65\") pod \"node-ca-m2pzk\" (UID: \"91f1914a-7ad4-45a2-949f-75375e34df1f\") " pod="openshift-image-registry/node-ca-m2pzk"
Jul 27 19:39:34.299133 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298476    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-var-lib-openvswitch\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.299133 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298503    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-ovnkube-config\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.299133 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298528    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-host-var-lib-cni-multus\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.299133 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298557    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-multus-daemon-config\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.299133 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298583    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxvg9\" (UniqueName: \"kubernetes.io/projected/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-kube-api-access-wxvg9\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.299133 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298609    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-systemd-units\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.299133 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298642    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6407e54e-d37a-4344-91d3-4494c81246c4-dbus\") pod \"global-pull-secret-syncer-7gt55\" (UID: \"6407e54e-d37a-4344-91d3-4494c81246c4\") " pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:34.299133 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298667    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5f94ff14-c2a5-4b14-b958-d56ada45996c-etc-tuned\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.299133 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298694    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a2601d76-e153-4372-a871-3e20499e774a-socket-dir\") pod \"aws-ebs-csi-driver-node-rn87v\" (UID: \"a2601d76-e153-4372-a871-3e20499e774a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.299133 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298743    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a2601d76-e153-4372-a871-3e20499e774a-device-dir\") pod \"aws-ebs-csi-driver-node-rn87v\" (UID: \"a2601d76-e153-4372-a871-3e20499e774a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.299133 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298771    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6407e54e-d37a-4344-91d3-4494c81246c4-kubelet-config\") pod \"global-pull-secret-syncer-7gt55\" (UID: \"6407e54e-d37a-4344-91d3-4494c81246c4\") " pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:34.299133 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298802    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a76e8a73-6dac-43e0-92ec-6269ccd13593-os-release\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.299951 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298832    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a76e8a73-6dac-43e0-92ec-6269ccd13593-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.299951 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298859    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv77p\" (UniqueName: \"kubernetes.io/projected/a76e8a73-6dac-43e0-92ec-6269ccd13593-kube-api-access-sv77p\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.299951 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298887    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-multus-cni-dir\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.299951 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298912    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73b8094b-b342-4acd-9323-a5e71e0c3dfb-tmp-dir\") pod \"node-resolver-grpbk\" (UID: \"73b8094b-b342-4acd-9323-a5e71e0c3dfb\") " pod="openshift-dns/node-resolver-grpbk"
Jul 27 19:39:34.299951 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.298973    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-lib-modules\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.299951 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.299038    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs\") pod \"network-metrics-daemon-sdk4x\" (UID: \"74f89d7b-f93b-4dc3-9e18-41cf466d90bd\") " pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:34.299951 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.299072    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-run-systemd\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.299951 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.299103    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-host-cni-netd\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.299951 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.299130    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-ovnkube-script-lib\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.299951 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.299155    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-node-log\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.299951 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.299180    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-env-overrides\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.299951 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.299206    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a2601d76-e153-4372-a871-3e20499e774a-registration-dir\") pod \"aws-ebs-csi-driver-node-rn87v\" (UID: \"a2601d76-e153-4372-a871-3e20499e774a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.299951 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.299267    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a76e8a73-6dac-43e0-92ec-6269ccd13593-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.299951 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.299409    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6407e54e-d37a-4344-91d3-4494c81246c4-dbus\") pod \"global-pull-secret-syncer-7gt55\" (UID: \"6407e54e-d37a-4344-91d3-4494c81246c4\") " pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:34.299951 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.299519    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a76e8a73-6dac-43e0-92ec-6269ccd13593-os-release\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.299951 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.299568    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-var-lib-kubelet\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.299951 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.299694    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-etc-kubernetes\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.301004 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:34.299795    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:39:34.301004 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.299906    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-host\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.301004 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:34.299947    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs podName:74f89d7b-f93b-4dc3-9e18-41cf466d90bd nodeName:}" failed. No retries permitted until 2026-07-27 19:39:34.799893156 +0000 UTC m=+3.029135333 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs") pod "network-metrics-daemon-sdk4x" (UID: "74f89d7b-f93b-4dc3-9e18-41cf466d90bd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:39:34.301004 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.299968    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/967ad4b2-9b0f-4654-be1b-8bcceee7f7c3-konnectivity-ca\") pod \"konnectivity-agent-tktlt\" (UID: \"967ad4b2-9b0f-4654-be1b-8bcceee7f7c3\") " pod="kube-system/konnectivity-agent-tktlt"
Jul 27 19:39:34.301004 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300009    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-etc-sysconfig\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.301004 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300041    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-etc-sysctl-d\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.301004 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300051    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6407e54e-d37a-4344-91d3-4494c81246c4-kubelet-config\") pod \"global-pull-secret-syncer-7gt55\" (UID: \"6407e54e-d37a-4344-91d3-4494c81246c4\") " pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:34.301004 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300072    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-host-run-k8s-cni-cncf-io\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.301004 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300098    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a76e8a73-6dac-43e0-92ec-6269ccd13593-system-cni-dir\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.301004 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300105    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-host-run-multus-certs\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.301004 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300238    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-etc-sysconfig\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.301004 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300244    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a76e8a73-6dac-43e0-92ec-6269ccd13593-cni-binary-copy\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.301004 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300313    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-etc-sysctl-d\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.301004 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300360    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-var-lib-kubelet\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.301004 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300366    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-lib-modules\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.301004 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300417    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-etc-systemd\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.301004 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300481    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-etc-modprobe-d\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.301893 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300497    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-log-socket\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.301893 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300542    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a76e8a73-6dac-43e0-92ec-6269ccd13593-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.301893 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300542    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-ovn-node-metrics-cert\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.301893 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300548    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a76e8a73-6dac-43e0-92ec-6269ccd13593-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.301893 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300600    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/580a1b09-1df6-48b5-9b0d-d72cb6806b06-iptables-alerter-script\") pod \"iptables-alerter-96x6d\" (UID: \"580a1b09-1df6-48b5-9b0d-d72cb6806b06\") " pod="openshift-network-operator/iptables-alerter-96x6d"
Jul 27 19:39:34.301893 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300637    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-cnibin\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.301893 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300675    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-host-var-lib-cni-bin\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.301893 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300717    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-multus-conf-dir\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.301893 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300752    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-etc-openvswitch\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.301893 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300822    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4mjg\" (UniqueName: \"kubernetes.io/projected/580a1b09-1df6-48b5-9b0d-d72cb6806b06-kube-api-access-k4mjg\") pod \"iptables-alerter-96x6d\" (UID: \"580a1b09-1df6-48b5-9b0d-d72cb6806b06\") " pod="openshift-network-operator/iptables-alerter-96x6d"
Jul 27 19:39:34.301893 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300897    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/73b8094b-b342-4acd-9323-a5e71e0c3dfb-hosts-file\") pod \"node-resolver-grpbk\" (UID: \"73b8094b-b342-4acd-9323-a5e71e0c3dfb\") " pod="openshift-dns/node-resolver-grpbk"
Jul 27 19:39:34.301893 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300949    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-host-cni-bin\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.301893 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.300976    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-os-release\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.301893 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.301010    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-host-kubelet\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.301893 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.301039    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-run\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.301893 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.301069    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-sys\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.301893 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.301117    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret\") pod \"global-pull-secret-syncer-7gt55\" (UID: \"6407e54e-d37a-4344-91d3-4494c81246c4\") " pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:34.302763 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.301128    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-sys\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.302763 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.301150    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91f1914a-7ad4-45a2-949f-75375e34df1f-host\") pod \"node-ca-m2pzk\" (UID: \"91f1914a-7ad4-45a2-949f-75375e34df1f\") " pod="openshift-image-registry/node-ca-m2pzk"
Jul 27 19:39:34.302763 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.301172    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91f1914a-7ad4-45a2-949f-75375e34df1f-serviceca\") pod \"node-ca-m2pzk\" (UID: \"91f1914a-7ad4-45a2-949f-75375e34df1f\") " pod="openshift-image-registry/node-ca-m2pzk"
Jul 27 19:39:34.302763 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.301198    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-run-ovn\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.302763 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.301219    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-system-cni-dir\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.302763 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:34.301241    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:39:34.302763 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.301286    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91f1914a-7ad4-45a2-949f-75375e34df1f-host\") pod \"node-ca-m2pzk\" (UID: \"91f1914a-7ad4-45a2-949f-75375e34df1f\") " pod="openshift-image-registry/node-ca-m2pzk"
Jul 27 19:39:34.302763 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:34.301372    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret podName:6407e54e-d37a-4344-91d3-4494c81246c4 nodeName:}" failed. No retries permitted until 2026-07-27 19:39:34.801352581 +0000 UTC m=+3.030594775 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret") pod "global-pull-secret-syncer-7gt55" (UID: "6407e54e-d37a-4344-91d3-4494c81246c4") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:39:34.302763 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.301240    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-etc-kubernetes\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.302763 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.301418    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-etc-sysctl-conf\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.302763 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.301451    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg5fn\" (UniqueName: \"kubernetes.io/projected/5f94ff14-c2a5-4b14-b958-d56ada45996c-kube-api-access-fg5fn\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.302763 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.301487    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a2601d76-e153-4372-a871-3e20499e774a-etc-selinux\") pod \"aws-ebs-csi-driver-node-rn87v\" (UID: \"a2601d76-e153-4372-a871-3e20499e774a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.302763 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.301510    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a2601d76-e153-4372-a871-3e20499e774a-sys-fs\") pod \"aws-ebs-csi-driver-node-rn87v\" (UID: \"a2601d76-e153-4372-a871-3e20499e774a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.302763 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.301535    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx2zs\" (UniqueName: \"kubernetes.io/projected/a2601d76-e153-4372-a871-3e20499e774a-kube-api-access-lx2zs\") pod \"aws-ebs-csi-driver-node-rn87v\" (UID: \"a2601d76-e153-4372-a871-3e20499e774a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.302763 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.301581    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-etc-sysctl-conf\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.302763 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.301174    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5f94ff14-c2a5-4b14-b958-d56ada45996c-run\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.302763 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.301744    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91f1914a-7ad4-45a2-949f-75375e34df1f-serviceca\") pod \"node-ca-m2pzk\" (UID: \"91f1914a-7ad4-45a2-949f-75375e34df1f\") " pod="openshift-image-registry/node-ca-m2pzk"
Jul 27 19:39:34.303574 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.302172    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/967ad4b2-9b0f-4654-be1b-8bcceee7f7c3-agent-certs\") pod \"konnectivity-agent-tktlt\" (UID: \"967ad4b2-9b0f-4654-be1b-8bcceee7f7c3\") " pod="kube-system/konnectivity-agent-tktlt"
Jul 27 19:39:34.303574 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.302358    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f94ff14-c2a5-4b14-b958-d56ada45996c-tmp\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.303574 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.302644    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5f94ff14-c2a5-4b14-b958-d56ada45996c-etc-tuned\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.312554 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.312529    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg5fn\" (UniqueName: \"kubernetes.io/projected/5f94ff14-c2a5-4b14-b958-d56ada45996c-kube-api-access-fg5fn\") pod \"tuned-7jxkt\" (UID: \"5f94ff14-c2a5-4b14-b958-d56ada45996c\") " pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.320241 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.320211    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgbgj\" (UniqueName: \"kubernetes.io/projected/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-kube-api-access-qgbgj\") pod \"network-metrics-daemon-sdk4x\" (UID: \"74f89d7b-f93b-4dc3-9e18-41cf466d90bd\") " pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:34.320241 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.320222    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxt65\" (UniqueName: \"kubernetes.io/projected/91f1914a-7ad4-45a2-949f-75375e34df1f-kube-api-access-sxt65\") pod \"node-ca-m2pzk\" (UID: \"91f1914a-7ad4-45a2-949f-75375e34df1f\") " pod="openshift-image-registry/node-ca-m2pzk"
Jul 27 19:39:34.320523 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.320502    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv77p\" (UniqueName: \"kubernetes.io/projected/a76e8a73-6dac-43e0-92ec-6269ccd13593-kube-api-access-sv77p\") pod \"multus-additional-cni-plugins-lwcg2\" (UID: \"a76e8a73-6dac-43e0-92ec-6269ccd13593\") " pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.402335 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.402293    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a2601d76-e153-4372-a871-3e20499e774a-etc-selinux\") pod \"aws-ebs-csi-driver-node-rn87v\" (UID: \"a2601d76-e153-4372-a871-3e20499e774a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.402489 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.402348    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a2601d76-e153-4372-a871-3e20499e774a-sys-fs\") pod \"aws-ebs-csi-driver-node-rn87v\" (UID: \"a2601d76-e153-4372-a871-3e20499e774a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.402489 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.402382    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx2zs\" (UniqueName: \"kubernetes.io/projected/a2601d76-e153-4372-a871-3e20499e774a-kube-api-access-lx2zs\") pod \"aws-ebs-csi-driver-node-rn87v\" (UID: \"a2601d76-e153-4372-a871-3e20499e774a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.402489 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.402411    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-hostroot\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.402489 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.402442    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-host-slash\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.402640 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.402497    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8ck5\" (UniqueName: \"kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5\") pod \"network-check-target-fpbzl\" (UID: \"f9f7d637-863f-4a7f-8327-ce46c5076d4d\") " pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:39:34.402640 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.402544    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-run-openvswitch\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.402640 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.402581    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-host-run-ovn-kubernetes\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.402640 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.402612    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a2601d76-e153-4372-a871-3e20499e774a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rn87v\" (UID: \"a2601d76-e153-4372-a871-3e20499e774a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.402767 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.402653    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfdvj\" (UniqueName: \"kubernetes.io/projected/73b8094b-b342-4acd-9323-a5e71e0c3dfb-kube-api-access-cfdvj\") pod \"node-resolver-grpbk\" (UID: \"73b8094b-b342-4acd-9323-a5e71e0c3dfb\") " pod="openshift-dns/node-resolver-grpbk"
Jul 27 19:39:34.402767 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.402685    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-host-run-netns\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.402767 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.402740    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbkhn\" (UniqueName: \"kubernetes.io/projected/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-kube-api-access-hbkhn\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.402871 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.402775    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/580a1b09-1df6-48b5-9b0d-d72cb6806b06-host-slash\") pod \"iptables-alerter-96x6d\" (UID: \"580a1b09-1df6-48b5-9b0d-d72cb6806b06\") " pod="openshift-network-operator/iptables-alerter-96x6d"
Jul 27 19:39:34.402871 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.402805    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-cni-binary-copy\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.402871 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.402834    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-multus-socket-dir-parent\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.403041 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.402877    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-host-run-netns\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.403041 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.402907    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-host-var-lib-kubelet\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.403041 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.402953    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.403041 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.403021    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-var-lib-openvswitch\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.403249 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.403060    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-ovnkube-config\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.403249 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.403096    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-host-var-lib-cni-multus\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.403249 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.403124    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-multus-daemon-config\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.403249 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.403210    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxvg9\" (UniqueName: \"kubernetes.io/projected/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-kube-api-access-wxvg9\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.403482 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.403285    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-systemd-units\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.403482 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.403222    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a2601d76-e153-4372-a871-3e20499e774a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rn87v\" (UID: \"a2601d76-e153-4372-a871-3e20499e774a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.403482 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.403328    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a2601d76-e153-4372-a871-3e20499e774a-socket-dir\") pod \"aws-ebs-csi-driver-node-rn87v\" (UID: \"a2601d76-e153-4372-a871-3e20499e774a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.403482 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.403387    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a2601d76-e153-4372-a871-3e20499e774a-device-dir\") pod \"aws-ebs-csi-driver-node-rn87v\" (UID: \"a2601d76-e153-4372-a871-3e20499e774a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.403482 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.403423    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-multus-cni-dir\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.403482 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.403457    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73b8094b-b342-4acd-9323-a5e71e0c3dfb-tmp-dir\") pod \"node-resolver-grpbk\" (UID: \"73b8094b-b342-4acd-9323-a5e71e0c3dfb\") " pod="openshift-dns/node-resolver-grpbk"
Jul 27 19:39:34.403790 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.403490    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a2601d76-e153-4372-a871-3e20499e774a-etc-selinux\") pod \"aws-ebs-csi-driver-node-rn87v\" (UID: \"a2601d76-e153-4372-a871-3e20499e774a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.403790 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.403557    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-run-systemd\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.403790 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.403589    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a2601d76-e153-4372-a871-3e20499e774a-sys-fs\") pod \"aws-ebs-csi-driver-node-rn87v\" (UID: \"a2601d76-e153-4372-a871-3e20499e774a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.403790 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.403638    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-host-cni-netd\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.403790 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.403702    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-host-cni-netd\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.403790 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.403731    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-ovnkube-script-lib\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.404132 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.403824    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-node-log\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.404132 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.403860    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-env-overrides\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.404132 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.403890    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a2601d76-e153-4372-a871-3e20499e774a-registration-dir\") pod \"aws-ebs-csi-driver-node-rn87v\" (UID: \"a2601d76-e153-4372-a871-3e20499e774a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.404132 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.403953    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-host-run-k8s-cni-cncf-io\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.404132 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404047    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-host-run-multus-certs\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.404132 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404091    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-host-run-netns\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.404132 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404116    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-log-socket\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.404492 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404159    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-ovn-node-metrics-cert\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.404492 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404181    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/580a1b09-1df6-48b5-9b0d-d72cb6806b06-host-slash\") pod \"iptables-alerter-96x6d\" (UID: \"580a1b09-1df6-48b5-9b0d-d72cb6806b06\") " pod="openshift-network-operator/iptables-alerter-96x6d"
Jul 27 19:39:34.404492 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404196    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/580a1b09-1df6-48b5-9b0d-d72cb6806b06-iptables-alerter-script\") pod \"iptables-alerter-96x6d\" (UID: \"580a1b09-1df6-48b5-9b0d-d72cb6806b06\") " pod="openshift-network-operator/iptables-alerter-96x6d"
Jul 27 19:39:34.404492 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404252    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-cnibin\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.404492 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404288    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-host-var-lib-cni-bin\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.404492 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404275    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-multus-socket-dir-parent\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.404492 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404330    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-multus-conf-dir\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.404492 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404349    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-host-run-netns\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.404492 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404392    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-etc-openvswitch\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.404492 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404429    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-host-var-lib-kubelet\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.404492 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404446    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4mjg\" (UniqueName: \"kubernetes.io/projected/580a1b09-1df6-48b5-9b0d-d72cb6806b06-kube-api-access-k4mjg\") pod \"iptables-alerter-96x6d\" (UID: \"580a1b09-1df6-48b5-9b0d-d72cb6806b06\") " pod="openshift-network-operator/iptables-alerter-96x6d"
Jul 27 19:39:34.404492 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404484    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.405136 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404503    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/73b8094b-b342-4acd-9323-a5e71e0c3dfb-hosts-file\") pod \"node-resolver-grpbk\" (UID: \"73b8094b-b342-4acd-9323-a5e71e0c3dfb\") " pod="openshift-dns/node-resolver-grpbk"
Jul 27 19:39:34.405136 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404534    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-var-lib-openvswitch\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.405136 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404670    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-host-cni-bin\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.405136 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404707    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-os-release\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.405136 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404757    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-host-kubelet\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.405136 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404870    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-host-kubelet\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.405136 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404927    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a2601d76-e153-4372-a871-3e20499e774a-socket-dir\") pod \"aws-ebs-csi-driver-node-rn87v\" (UID: \"a2601d76-e153-4372-a871-3e20499e774a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.405136 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.404989    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-run-ovn\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.405136 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.405015    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-system-cni-dir\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.405136 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.405037    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-etc-kubernetes\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.405647 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.405199    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-etc-kubernetes\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.405647 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.405244    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-host-var-lib-cni-multus\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.405647 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.405247    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-ovnkube-config\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.405647 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.405337    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-hostroot\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.405647 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.405368    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a2601d76-e153-4372-a871-3e20499e774a-device-dir\") pod \"aws-ebs-csi-driver-node-rn87v\" (UID: \"a2601d76-e153-4372-a871-3e20499e774a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.405647 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.405460    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-host-slash\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.405647 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.405469    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-multus-cni-dir\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.405647 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.405531    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-run-systemd\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.406056 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.405728    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-multus-daemon-config\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.406181 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.406157    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-run-openvswitch\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.406255 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.406199    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-host-run-ovn-kubernetes\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.406383 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.406365    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-cni-binary-copy\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.406523 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.406505    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-systemd-units\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.406589 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.406546    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-run-ovn\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.406642 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.406593    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-system-cni-dir\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.406642 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.406635    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/73b8094b-b342-4acd-9323-a5e71e0c3dfb-hosts-file\") pod \"node-resolver-grpbk\" (UID: \"73b8094b-b342-4acd-9323-a5e71e0c3dfb\") " pod="openshift-dns/node-resolver-grpbk"
Jul 27 19:39:34.406757 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.406667    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-cnibin\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.406757 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.406690    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-host-var-lib-cni-bin\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.406757 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.406714    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-multus-conf-dir\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.406757 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.406749    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-etc-openvswitch\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.407005 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.406955    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-os-release\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.407005 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.406970    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/580a1b09-1df6-48b5-9b0d-d72cb6806b06-iptables-alerter-script\") pod \"iptables-alerter-96x6d\" (UID: \"580a1b09-1df6-48b5-9b0d-d72cb6806b06\") " pod="openshift-network-operator/iptables-alerter-96x6d"
Jul 27 19:39:34.407005 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.406987    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-host-cni-bin\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.407198 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.407181    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73b8094b-b342-4acd-9323-a5e71e0c3dfb-tmp-dir\") pod \"node-resolver-grpbk\" (UID: \"73b8094b-b342-4acd-9323-a5e71e0c3dfb\") " pod="openshift-dns/node-resolver-grpbk"
Jul 27 19:39:34.407256 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.407237    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-host-run-k8s-cni-cncf-io\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.407407 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.407390    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a2601d76-e153-4372-a871-3e20499e774a-registration-dir\") pod \"aws-ebs-csi-driver-node-rn87v\" (UID: \"a2601d76-e153-4372-a871-3e20499e774a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.407522 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.407443    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-node-log\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.407749 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.407714    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-ovnkube-script-lib\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.407816 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.407788    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-log-socket\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.407876 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.407827    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-host-run-multus-certs\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.407950 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.407859    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-env-overrides\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.408862 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.408791    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-ovn-node-metrics-cert\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.414082 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.414061    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfdvj\" (UniqueName: \"kubernetes.io/projected/73b8094b-b342-4acd-9323-a5e71e0c3dfb-kube-api-access-cfdvj\") pod \"node-resolver-grpbk\" (UID: \"73b8094b-b342-4acd-9323-a5e71e0c3dfb\") " pod="openshift-dns/node-resolver-grpbk"
Jul 27 19:39:34.415756 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.415723    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx2zs\" (UniqueName: \"kubernetes.io/projected/a2601d76-e153-4372-a871-3e20499e774a-kube-api-access-lx2zs\") pod \"aws-ebs-csi-driver-node-rn87v\" (UID: \"a2601d76-e153-4372-a871-3e20499e774a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.417317 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.417295    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbkhn\" (UniqueName: \"kubernetes.io/projected/6d952ea0-8d44-4913-b659-8a6f6f0b8b25-kube-api-access-hbkhn\") pod \"ovnkube-node-nwhgp\" (UID: \"6d952ea0-8d44-4913-b659-8a6f6f0b8b25\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.417660 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.417641    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxvg9\" (UniqueName: \"kubernetes.io/projected/5d7a3c81-c00c-467a-92ac-ee6abb8b2d72-kube-api-access-wxvg9\") pod \"multus-nqrzj\" (UID: \"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72\") " pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.423055 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:34.423034    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:39:34.423055 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:34.423058    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:39:34.423199 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:34.423072    2579 projected.go:196] Error preparing data for projected volume kube-api-access-p8ck5 for pod openshift-network-diagnostics/network-check-target-fpbzl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:39:34.423199 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:34.423151    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5 podName:f9f7d637-863f-4a7f-8327-ce46c5076d4d nodeName:}" failed. No retries permitted until 2026-07-27 19:39:34.923132938 +0000 UTC m=+3.152375130 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-p8ck5" (UniqueName: "kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5") pod "network-check-target-fpbzl" (UID: "f9f7d637-863f-4a7f-8327-ce46c5076d4d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:39:34.424422 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.424391    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4mjg\" (UniqueName: \"kubernetes.io/projected/580a1b09-1df6-48b5-9b0d-d72cb6806b06-kube-api-access-k4mjg\") pod \"iptables-alerter-96x6d\" (UID: \"580a1b09-1df6-48b5-9b0d-d72cb6806b06\") " pod="openshift-network-operator/iptables-alerter-96x6d"
Jul 27 19:39:34.483760 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.483730    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tktlt"
Jul 27 19:39:34.494238 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.494205    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m2pzk"
Jul 27 19:39:34.510421 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.510398    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7jxkt"
Jul 27 19:39:34.517256 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.517232    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-grpbk"
Jul 27 19:39:34.524113 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.524095    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lwcg2"
Jul 27 19:39:34.537985 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.537962    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-96x6d"
Jul 27 19:39:34.546022 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.546000    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v"
Jul 27 19:39:34.552946 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.552924    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nqrzj"
Jul 27 19:39:34.559019 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.559002    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:39:34.808130 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.808034    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs\") pod \"network-metrics-daemon-sdk4x\" (UID: \"74f89d7b-f93b-4dc3-9e18-41cf466d90bd\") " pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:34.808130 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:34.808100    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret\") pod \"global-pull-secret-syncer-7gt55\" (UID: \"6407e54e-d37a-4344-91d3-4494c81246c4\") " pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:34.808322 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:34.808213    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:39:34.808322 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:34.808222    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:39:34.808322 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:34.808296    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret podName:6407e54e-d37a-4344-91d3-4494c81246c4 nodeName:}" failed. No retries permitted until 2026-07-27 19:39:35.808274542 +0000 UTC m=+4.037516735 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret") pod "global-pull-secret-syncer-7gt55" (UID: "6407e54e-d37a-4344-91d3-4494c81246c4") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:39:34.808322 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:34.808322    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs podName:74f89d7b-f93b-4dc3-9e18-41cf466d90bd nodeName:}" failed. No retries permitted until 2026-07-27 19:39:35.808306591 +0000 UTC m=+4.037548768 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs") pod "network-metrics-daemon-sdk4x" (UID: "74f89d7b-f93b-4dc3-9e18-41cf466d90bd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:39:35.009705 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:35.009665    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8ck5\" (UniqueName: \"kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5\") pod \"network-check-target-fpbzl\" (UID: \"f9f7d637-863f-4a7f-8327-ce46c5076d4d\") " pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:39:35.009835 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:35.009805    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:39:35.009835 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:35.009819    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:39:35.009835 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:35.009828    2579 projected.go:196] Error preparing data for projected volume kube-api-access-p8ck5 for pod openshift-network-diagnostics/network-check-target-fpbzl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:39:35.009966 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:35.009880    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5 podName:f9f7d637-863f-4a7f-8327-ce46c5076d4d nodeName:}" failed. No retries permitted until 2026-07-27 19:39:36.009865035 +0000 UTC m=+4.239107227 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-p8ck5" (UniqueName: "kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5") pod "network-check-target-fpbzl" (UID: "f9f7d637-863f-4a7f-8327-ce46c5076d4d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:39:35.240249 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:35.240163    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 19:34:34 +0000 UTC" deadline="2027-12-28 15:09:45.132156631 +0000 UTC"
Jul 27 19:39:35.240249 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:35.240194    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12451h30m9.891967384s"
Jul 27 19:39:35.255353 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:35.255327    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:39:35.255502 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:35.255449    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fpbzl" podUID="f9f7d637-863f-4a7f-8327-ce46c5076d4d"
Jul 27 19:39:35.325992 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:35.325967    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d952ea0_8d44_4913_b659_8a6f6f0b8b25.slice/crio-6a21343627880a60345930f9e83b9ef577146f3ad17c9fc4ad5a0d2376d35479 WatchSource:0}: Error finding container 6a21343627880a60345930f9e83b9ef577146f3ad17c9fc4ad5a0d2376d35479: Status 404 returned error can't find the container with id 6a21343627880a60345930f9e83b9ef577146f3ad17c9fc4ad5a0d2376d35479
Jul 27 19:39:35.327473 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:35.327448    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod580a1b09_1df6_48b5_9b0d_d72cb6806b06.slice/crio-2ce650b14cc4bece4c623f1f3436c63e2d7c3c189922a1df883ac1e180d96564 WatchSource:0}: Error finding container 2ce650b14cc4bece4c623f1f3436c63e2d7c3c189922a1df883ac1e180d96564: Status 404 returned error can't find the container with id 2ce650b14cc4bece4c623f1f3436c63e2d7c3c189922a1df883ac1e180d96564
Jul 27 19:39:35.357906 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:35.357879    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73b8094b_b342_4acd_9323_a5e71e0c3dfb.slice/crio-13a8499354cf7d56b21ad304f4e99de11b68c8d322400dfac96965fe22a77792 WatchSource:0}: Error finding container 13a8499354cf7d56b21ad304f4e99de11b68c8d322400dfac96965fe22a77792: Status 404 returned error can't find the container with id 13a8499354cf7d56b21ad304f4e99de11b68c8d322400dfac96965fe22a77792
Jul 27 19:39:35.358629 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:35.358606    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod967ad4b2_9b0f_4654_be1b_8bcceee7f7c3.slice/crio-8840862824e62b9fb16fbafaafe3113380db00f36ac335db9d95635acd4be37c WatchSource:0}: Error finding container 8840862824e62b9fb16fbafaafe3113380db00f36ac335db9d95635acd4be37c: Status 404 returned error can't find the container with id 8840862824e62b9fb16fbafaafe3113380db00f36ac335db9d95635acd4be37c
Jul 27 19:39:35.359703 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:39:35.359676    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2601d76_e153_4372_a871_3e20499e774a.slice/crio-f104900b4345a2547c4c31797b387f31046bb56630bb240f10e8135f19058ef1 WatchSource:0}: Error finding container f104900b4345a2547c4c31797b387f31046bb56630bb240f10e8135f19058ef1: Status 404 returned error can't find the container with id f104900b4345a2547c4c31797b387f31046bb56630bb240f10e8135f19058ef1
Jul 27 19:39:35.817322 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:35.817071    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs\") pod \"network-metrics-daemon-sdk4x\" (UID: \"74f89d7b-f93b-4dc3-9e18-41cf466d90bd\") " pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:35.817322 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:35.817312    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret\") pod \"global-pull-secret-syncer-7gt55\" (UID: \"6407e54e-d37a-4344-91d3-4494c81246c4\") " pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:35.817525 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:35.817460    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:39:35.817582 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:35.817523    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret podName:6407e54e-d37a-4344-91d3-4494c81246c4 nodeName:}" failed. No retries permitted until 2026-07-27 19:39:37.817504327 +0000 UTC m=+6.046746521 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret") pod "global-pull-secret-syncer-7gt55" (UID: "6407e54e-d37a-4344-91d3-4494c81246c4") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:39:35.817654 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:35.817603    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:39:35.817654 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:35.817647    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs podName:74f89d7b-f93b-4dc3-9e18-41cf466d90bd nodeName:}" failed. No retries permitted until 2026-07-27 19:39:37.817634603 +0000 UTC m=+6.046876795 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs") pod "network-metrics-daemon-sdk4x" (UID: "74f89d7b-f93b-4dc3-9e18-41cf466d90bd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:39:36.019632 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:36.019595    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8ck5\" (UniqueName: \"kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5\") pod \"network-check-target-fpbzl\" (UID: \"f9f7d637-863f-4a7f-8327-ce46c5076d4d\") " pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:39:36.019794 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:36.019750    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:39:36.019794 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:36.019769    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:39:36.019794 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:36.019781    2579 projected.go:196] Error preparing data for projected volume kube-api-access-p8ck5 for pod openshift-network-diagnostics/network-check-target-fpbzl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:39:36.019996 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:36.019866    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5 podName:f9f7d637-863f-4a7f-8327-ce46c5076d4d nodeName:}" failed. No retries permitted until 2026-07-27 19:39:38.019840935 +0000 UTC m=+6.249083131 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-p8ck5" (UniqueName: "kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5") pod "network-check-target-fpbzl" (UID: "f9f7d637-863f-4a7f-8327-ce46c5076d4d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:39:36.241001 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:36.240942    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 19:34:34 +0000 UTC" deadline="2027-12-26 03:09:41.87804459 +0000 UTC"
Jul 27 19:39:36.241001 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:36.240979    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12391h30m5.637071071s"
Jul 27 19:39:36.258866 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:36.258071    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:36.258866 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:36.258081    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:36.258866 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:36.258246    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdk4x" podUID="74f89d7b-f93b-4dc3-9e18-41cf466d90bd"
Jul 27 19:39:36.258866 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:36.258372    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gt55" podUID="6407e54e-d37a-4344-91d3-4494c81246c4"
Jul 27 19:39:36.303767 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:36.303665    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v" event={"ID":"a2601d76-e153-4372-a871-3e20499e774a","Type":"ContainerStarted","Data":"f104900b4345a2547c4c31797b387f31046bb56630bb240f10e8135f19058ef1"}
Jul 27 19:39:36.306672 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:36.306598    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tktlt" event={"ID":"967ad4b2-9b0f-4654-be1b-8bcceee7f7c3","Type":"ContainerStarted","Data":"8840862824e62b9fb16fbafaafe3113380db00f36ac335db9d95635acd4be37c"}
Jul 27 19:39:36.318775 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:36.318726    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nqrzj" event={"ID":"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72","Type":"ContainerStarted","Data":"d123024ffc31c70ccc34e8c42922d4fcc4231c50b62d32400459a0125ae4503e"}
Jul 27 19:39:36.327421 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:36.327388    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp" event={"ID":"6d952ea0-8d44-4913-b659-8a6f6f0b8b25","Type":"ContainerStarted","Data":"6a21343627880a60345930f9e83b9ef577146f3ad17c9fc4ad5a0d2376d35479"}
Jul 27 19:39:36.344428 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:36.344398    2579 generic.go:356] "Generic (PLEG): container finished" podID="65272394e66c19ba2f850aea94ec8a01" containerID="d77617bf1c92c56a9760fb42f6811c2ed86fa9948f2920c71933dc4ed4b5f042" exitCode=0
Jul 27 19:39:36.344530 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:36.344484    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-36.ec2.internal" event={"ID":"65272394e66c19ba2f850aea94ec8a01","Type":"ContainerDied","Data":"d77617bf1c92c56a9760fb42f6811c2ed86fa9948f2920c71933dc4ed4b5f042"}
Jul 27 19:39:36.358415 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:36.358383    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-36.ec2.internal" event={"ID":"8f425195a1bad05a939cf1834470c7d1","Type":"ContainerStarted","Data":"ffbf539f65c56935cce9943b098da75d672472e077122723136722841bcf4f9f"}
Jul 27 19:39:36.363383 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:36.363355    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-grpbk" event={"ID":"73b8094b-b342-4acd-9323-a5e71e0c3dfb","Type":"ContainerStarted","Data":"13a8499354cf7d56b21ad304f4e99de11b68c8d322400dfac96965fe22a77792"}
Jul 27 19:39:36.365658 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:36.365444    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-96x6d" event={"ID":"580a1b09-1df6-48b5-9b0d-d72cb6806b06","Type":"ContainerStarted","Data":"2ce650b14cc4bece4c623f1f3436c63e2d7c3c189922a1df883ac1e180d96564"}
Jul 27 19:39:36.381293 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:36.381218    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7jxkt" event={"ID":"5f94ff14-c2a5-4b14-b958-d56ada45996c","Type":"ContainerStarted","Data":"c2a864f43427f259f74fcdf842a430d14a09d931330957d1d685970d5c33f359"}
Jul 27 19:39:36.404948 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:36.404897    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m2pzk" event={"ID":"91f1914a-7ad4-45a2-949f-75375e34df1f","Type":"ContainerStarted","Data":"ce13e9d4e4e0de183a5761c717541365e349bba6bec05c21527eebeeddd6fe95"}
Jul 27 19:39:36.412546 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:36.412477    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lwcg2" event={"ID":"a76e8a73-6dac-43e0-92ec-6269ccd13593","Type":"ContainerStarted","Data":"9e0da4719d4a20db138b24f5fa17b5dde1db5bb025cbaebad31dd24f77b06ab9"}
Jul 27 19:39:37.255596 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:37.255562    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:39:37.256124 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:37.255690    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fpbzl" podUID="f9f7d637-863f-4a7f-8327-ce46c5076d4d"
Jul 27 19:39:37.433032 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:37.432995    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-36.ec2.internal" event={"ID":"65272394e66c19ba2f850aea94ec8a01","Type":"ContainerStarted","Data":"58434fe8d79b782308da8c4f99d107788870742f1402063051af2cae13ba4b94"}
Jul 27 19:39:37.452562 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:37.452351    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-36.ec2.internal" podStartSLOduration=3.45233175 podStartE2EDuration="3.45233175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:39:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:39:37.452214197 +0000 UTC m=+5.681456399" watchObservedRunningTime="2026-07-27 19:39:37.45233175 +0000 UTC m=+5.681573950"
Jul 27 19:39:37.452956 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:37.452901    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-36.ec2.internal" podStartSLOduration=3.452889382 podStartE2EDuration="3.452889382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:39:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:39:36.387627184 +0000 UTC m=+4.616869389" watchObservedRunningTime="2026-07-27 19:39:37.452889382 +0000 UTC m=+5.682131589"
Jul 27 19:39:37.834641 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:37.834604    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret\") pod \"global-pull-secret-syncer-7gt55\" (UID: \"6407e54e-d37a-4344-91d3-4494c81246c4\") " pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:37.834842 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:37.834704    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs\") pod \"network-metrics-daemon-sdk4x\" (UID: \"74f89d7b-f93b-4dc3-9e18-41cf466d90bd\") " pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:37.835031 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:37.834985    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:39:37.835115 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:37.835056    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret podName:6407e54e-d37a-4344-91d3-4494c81246c4 nodeName:}" failed. No retries permitted until 2026-07-27 19:39:41.835035828 +0000 UTC m=+10.064278021 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret") pod "global-pull-secret-syncer-7gt55" (UID: "6407e54e-d37a-4344-91d3-4494c81246c4") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:39:37.835588 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:37.835567    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:39:37.835686 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:37.835634    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs podName:74f89d7b-f93b-4dc3-9e18-41cf466d90bd nodeName:}" failed. No retries permitted until 2026-07-27 19:39:41.835616078 +0000 UTC m=+10.064858272 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs") pod "network-metrics-daemon-sdk4x" (UID: "74f89d7b-f93b-4dc3-9e18-41cf466d90bd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:39:38.036350 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:38.036312    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8ck5\" (UniqueName: \"kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5\") pod \"network-check-target-fpbzl\" (UID: \"f9f7d637-863f-4a7f-8327-ce46c5076d4d\") " pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:39:38.036600 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:38.036580    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:39:38.036676 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:38.036613    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:39:38.036676 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:38.036628    2579 projected.go:196] Error preparing data for projected volume kube-api-access-p8ck5 for pod openshift-network-diagnostics/network-check-target-fpbzl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:39:38.036773 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:38.036691    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5 podName:f9f7d637-863f-4a7f-8327-ce46c5076d4d nodeName:}" failed. No retries permitted until 2026-07-27 19:39:42.036669858 +0000 UTC m=+10.265912063 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-p8ck5" (UniqueName: "kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5") pod "network-check-target-fpbzl" (UID: "f9f7d637-863f-4a7f-8327-ce46c5076d4d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:39:38.257137 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:38.257058    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:38.257652 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:38.257203    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gt55" podUID="6407e54e-d37a-4344-91d3-4494c81246c4"
Jul 27 19:39:38.257652 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:38.257309    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:38.257652 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:38.257446    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdk4x" podUID="74f89d7b-f93b-4dc3-9e18-41cf466d90bd"
Jul 27 19:39:39.256339 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:39.256302    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:39:39.256525 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:39.256495    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fpbzl" podUID="f9f7d637-863f-4a7f-8327-ce46c5076d4d"
Jul 27 19:39:40.255974 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:40.255937    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:40.255974 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:40.255962    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:40.256510 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:40.256102    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdk4x" podUID="74f89d7b-f93b-4dc3-9e18-41cf466d90bd"
Jul 27 19:39:40.256510 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:40.256242    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gt55" podUID="6407e54e-d37a-4344-91d3-4494c81246c4"
Jul 27 19:39:41.256062 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:41.255980    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:39:41.256549 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:41.256107    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fpbzl" podUID="f9f7d637-863f-4a7f-8327-ce46c5076d4d"
Jul 27 19:39:41.873742 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:41.873703    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs\") pod \"network-metrics-daemon-sdk4x\" (UID: \"74f89d7b-f93b-4dc3-9e18-41cf466d90bd\") " pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:41.873967 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:41.873770    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret\") pod \"global-pull-secret-syncer-7gt55\" (UID: \"6407e54e-d37a-4344-91d3-4494c81246c4\") " pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:41.873967 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:41.873834    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:39:41.873967 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:41.873915    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:39:41.873967 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:41.873929    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs podName:74f89d7b-f93b-4dc3-9e18-41cf466d90bd nodeName:}" failed. No retries permitted until 2026-07-27 19:39:49.873909301 +0000 UTC m=+18.103151496 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs") pod "network-metrics-daemon-sdk4x" (UID: "74f89d7b-f93b-4dc3-9e18-41cf466d90bd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:39:41.873967 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:41.873969    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret podName:6407e54e-d37a-4344-91d3-4494c81246c4 nodeName:}" failed. No retries permitted until 2026-07-27 19:39:49.873952469 +0000 UTC m=+18.103194663 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret") pod "global-pull-secret-syncer-7gt55" (UID: "6407e54e-d37a-4344-91d3-4494c81246c4") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:39:42.075378 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:42.075342    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8ck5\" (UniqueName: \"kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5\") pod \"network-check-target-fpbzl\" (UID: \"f9f7d637-863f-4a7f-8327-ce46c5076d4d\") " pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:39:42.075541 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:42.075486    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:39:42.075541 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:42.075504    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:39:42.075541 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:42.075515    2579 projected.go:196] Error preparing data for projected volume kube-api-access-p8ck5 for pod openshift-network-diagnostics/network-check-target-fpbzl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:39:42.075708 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:42.075578    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5 podName:f9f7d637-863f-4a7f-8327-ce46c5076d4d nodeName:}" failed. No retries permitted until 2026-07-27 19:39:50.07555872 +0000 UTC m=+18.304800913 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-p8ck5" (UniqueName: "kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5") pod "network-check-target-fpbzl" (UID: "f9f7d637-863f-4a7f-8327-ce46c5076d4d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:39:42.256444 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:42.256346    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:42.257179 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:42.256998    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:42.257503 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:42.257452    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdk4x" podUID="74f89d7b-f93b-4dc3-9e18-41cf466d90bd"
Jul 27 19:39:42.257631 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:42.257607    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gt55" podUID="6407e54e-d37a-4344-91d3-4494c81246c4"
Jul 27 19:39:43.255748 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:43.255715    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:39:43.255926 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:43.255840    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fpbzl" podUID="f9f7d637-863f-4a7f-8327-ce46c5076d4d"
Jul 27 19:39:44.255421 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:44.255388    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:44.255877 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:44.255511    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdk4x" podUID="74f89d7b-f93b-4dc3-9e18-41cf466d90bd"
Jul 27 19:39:44.255983 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:44.255967    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:44.256109 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:44.256074    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gt55" podUID="6407e54e-d37a-4344-91d3-4494c81246c4"
Jul 27 19:39:45.255729 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:45.255699    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:39:45.256152 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:45.255817    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fpbzl" podUID="f9f7d637-863f-4a7f-8327-ce46c5076d4d"
Jul 27 19:39:46.255928 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:46.255627    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:46.255928 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:46.255750    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gt55" podUID="6407e54e-d37a-4344-91d3-4494c81246c4"
Jul 27 19:39:46.255928 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:46.255785    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:46.255928 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:46.255898    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdk4x" podUID="74f89d7b-f93b-4dc3-9e18-41cf466d90bd"
Jul 27 19:39:47.255890 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:47.255841    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:39:47.256061 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:47.255982    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fpbzl" podUID="f9f7d637-863f-4a7f-8327-ce46c5076d4d"
Jul 27 19:39:48.256458 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:48.256424    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:48.256458 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:48.256441    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:48.256972 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:48.256551    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gt55" podUID="6407e54e-d37a-4344-91d3-4494c81246c4"
Jul 27 19:39:48.256972 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:48.256699    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdk4x" podUID="74f89d7b-f93b-4dc3-9e18-41cf466d90bd"
Jul 27 19:39:49.256292 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:49.256260    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:39:49.256520 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:49.256352    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fpbzl" podUID="f9f7d637-863f-4a7f-8327-ce46c5076d4d"
Jul 27 19:39:49.940193 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:49.940163    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs\") pod \"network-metrics-daemon-sdk4x\" (UID: \"74f89d7b-f93b-4dc3-9e18-41cf466d90bd\") " pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:49.940460 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:49.940206    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret\") pod \"global-pull-secret-syncer-7gt55\" (UID: \"6407e54e-d37a-4344-91d3-4494c81246c4\") " pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:49.940460 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:49.940302    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:39:49.940460 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:49.940330    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:39:49.940460 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:49.940356    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret podName:6407e54e-d37a-4344-91d3-4494c81246c4 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:05.940339232 +0000 UTC m=+34.169581418 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret") pod "global-pull-secret-syncer-7gt55" (UID: "6407e54e-d37a-4344-91d3-4494c81246c4") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:39:49.940460 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:49.940392    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs podName:74f89d7b-f93b-4dc3-9e18-41cf466d90bd nodeName:}" failed. No retries permitted until 2026-07-27 19:40:05.940373055 +0000 UTC m=+34.169615244 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs") pod "network-metrics-daemon-sdk4x" (UID: "74f89d7b-f93b-4dc3-9e18-41cf466d90bd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:39:50.141744 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:50.141713    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8ck5\" (UniqueName: \"kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5\") pod \"network-check-target-fpbzl\" (UID: \"f9f7d637-863f-4a7f-8327-ce46c5076d4d\") " pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:39:50.141902 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:50.141836    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:39:50.141902 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:50.141851    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:39:50.141902 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:50.141874    2579 projected.go:196] Error preparing data for projected volume kube-api-access-p8ck5 for pod openshift-network-diagnostics/network-check-target-fpbzl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:39:50.142007 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:50.141926    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5 podName:f9f7d637-863f-4a7f-8327-ce46c5076d4d nodeName:}" failed. No retries permitted until 2026-07-27 19:40:06.141908304 +0000 UTC m=+34.371150493 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-p8ck5" (UniqueName: "kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5") pod "network-check-target-fpbzl" (UID: "f9f7d637-863f-4a7f-8327-ce46c5076d4d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:39:50.256331 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:50.256305    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:50.256447 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:50.256415    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gt55" podUID="6407e54e-d37a-4344-91d3-4494c81246c4"
Jul 27 19:39:50.256519 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:50.256492    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:50.256609 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:50.256590    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdk4x" podUID="74f89d7b-f93b-4dc3-9e18-41cf466d90bd"
Jul 27 19:39:51.255940 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:51.255906    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:39:51.256117 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:51.256013    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fpbzl" podUID="f9f7d637-863f-4a7f-8327-ce46c5076d4d"
Jul 27 19:39:52.259551 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:52.259515    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:52.260081 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:52.259653    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gt55" podUID="6407e54e-d37a-4344-91d3-4494c81246c4"
Jul 27 19:39:52.260141 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:52.260075    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:52.260254 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:52.260227    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdk4x" podUID="74f89d7b-f93b-4dc3-9e18-41cf466d90bd"
Jul 27 19:39:53.256302 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:53.256272    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:39:53.256469 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:53.256386    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fpbzl" podUID="f9f7d637-863f-4a7f-8327-ce46c5076d4d"
Jul 27 19:39:54.258168 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:54.258140    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:54.258610 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:54.258270    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdk4x" podUID="74f89d7b-f93b-4dc3-9e18-41cf466d90bd"
Jul 27 19:39:54.258610 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:54.258334    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:54.258610 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:54.258463    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gt55" podUID="6407e54e-d37a-4344-91d3-4494c81246c4"
Jul 27 19:39:55.255564 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:55.255281    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:39:55.255751 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:55.255639    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fpbzl" podUID="f9f7d637-863f-4a7f-8327-ce46c5076d4d"
Jul 27 19:39:55.482781 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:55.482744    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-grpbk" event={"ID":"73b8094b-b342-4acd-9323-a5e71e0c3dfb","Type":"ContainerStarted","Data":"7aff3beb1d3c6ac137ec2654a566128e142aea1adbd42a44455d0b6adfcb5929"}
Jul 27 19:39:55.484203 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:55.484182    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7jxkt" event={"ID":"5f94ff14-c2a5-4b14-b958-d56ada45996c","Type":"ContainerStarted","Data":"dcd7f73dd4e5141443f8ad454c381dc194631302e4629433dc947b1b953e13eb"}
Jul 27 19:39:55.485668 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:55.485635    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m2pzk" event={"ID":"91f1914a-7ad4-45a2-949f-75375e34df1f","Type":"ContainerStarted","Data":"abb14c162abc57997ee70f03b517dac33f9f3549176f474ac26ec14d86f1f32e"}
Jul 27 19:39:55.487099 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:55.487077    2579 generic.go:356] "Generic (PLEG): container finished" podID="a76e8a73-6dac-43e0-92ec-6269ccd13593" containerID="41a0a8e2fd8361ab8987ca48d091fc2118c40118dd6c0a5b763b0417c1f1828f" exitCode=0
Jul 27 19:39:55.487199 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:55.487157    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lwcg2" event={"ID":"a76e8a73-6dac-43e0-92ec-6269ccd13593","Type":"ContainerDied","Data":"41a0a8e2fd8361ab8987ca48d091fc2118c40118dd6c0a5b763b0417c1f1828f"}
Jul 27 19:39:55.488679 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:55.488660    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v" event={"ID":"a2601d76-e153-4372-a871-3e20499e774a","Type":"ContainerStarted","Data":"7d8e1be014a834a5a0bbf2d84089b1d69214ad084672fff3ab6042aa43167afa"}
Jul 27 19:39:55.490107 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:55.490084    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tktlt" event={"ID":"967ad4b2-9b0f-4654-be1b-8bcceee7f7c3","Type":"ContainerStarted","Data":"fb8fa53f35c2345e88249c4a0e263327ccfaa7094eb59c09160d66d96e670002"}
Jul 27 19:39:55.491453 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:55.491434    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nqrzj" event={"ID":"5d7a3c81-c00c-467a-92ac-ee6abb8b2d72","Type":"ContainerStarted","Data":"12e729c95cb0bae47a1ec9a3d676839c64a6bf0e41e3be3ff94face25c8808f9"}
Jul 27 19:39:55.494326 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:55.494292    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp" event={"ID":"6d952ea0-8d44-4913-b659-8a6f6f0b8b25","Type":"ContainerStarted","Data":"2ddc6d1882787ef8b745683f9389a7f4f8772cffba0b46c1375bb10fe3310f05"}
Jul 27 19:39:55.494413 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:55.494344    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp" event={"ID":"6d952ea0-8d44-4913-b659-8a6f6f0b8b25","Type":"ContainerStarted","Data":"9586226ac3d11c434943251f2e22258b2ad32e372c47a3cf198a6419e114b2a0"}
Jul 27 19:39:55.494413 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:55.494382    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp" event={"ID":"6d952ea0-8d44-4913-b659-8a6f6f0b8b25","Type":"ContainerStarted","Data":"f3edb5b32cbb0bc87eda18b48ac6e12fc78022ecf4e51ca485268d7028a15eb2"}
Jul 27 19:39:55.494413 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:55.494411    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp" event={"ID":"6d952ea0-8d44-4913-b659-8a6f6f0b8b25","Type":"ContainerStarted","Data":"0413a422f68ccca465193ef4d30c426e88b6c4beca582c86202c9a4325b19f49"}
Jul 27 19:39:55.494524 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:55.494438    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp" event={"ID":"6d952ea0-8d44-4913-b659-8a6f6f0b8b25","Type":"ContainerStarted","Data":"ea9fbdd8c28d74b338ebfe19237c28e4b2d3b0681d3f6caa049720871e067236"}
Jul 27 19:39:55.494524 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:55.494480    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp" event={"ID":"6d952ea0-8d44-4913-b659-8a6f6f0b8b25","Type":"ContainerStarted","Data":"542b65f55c19fd0b52f8a81a0c281dc698ddf86ab76e7a198a81122ecbf1ce44"}
Jul 27 19:39:55.501552 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:55.501516    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-grpbk" podStartSLOduration=7.665363074 podStartE2EDuration="22.50150399s" totalImagesPullingTime="14.836140916s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:39:55.501119365 +0000 UTC m=+23.730361563" watchObservedRunningTime="2026-07-27 19:39:55.50150399 +0000 UTC m=+23.730746188"
Jul 27 19:39:55.519727 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:55.519693    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tktlt" podStartSLOduration=8.684939231 podStartE2EDuration="23.519681991s" totalImagesPullingTime="14.83474276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:39:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:39:55.519178079 +0000 UTC m=+23.748420278" watchObservedRunningTime="2026-07-27 19:39:55.519681991 +0000 UTC m=+23.748924188"
Jul 27 19:39:55.533666 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:55.533631    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-m2pzk" podStartSLOduration=4.540419015 podStartE2EDuration="23.533619749s" totalImagesPullingTime="18.993200734s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:39:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:39:55.533300721 +0000 UTC m=+23.762542915" watchObservedRunningTime="2026-07-27 19:39:55.533619749 +0000 UTC m=+23.762861947"
Jul 27 19:39:55.550384 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:55.550351    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7jxkt" podStartSLOduration=3.5531934290000002 podStartE2EDuration="22.550341296s" totalImagesPullingTime="18.997147867s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:39:55.550084822 +0000 UTC m=+23.779327018" watchObservedRunningTime="2026-07-27 19:39:55.550341296 +0000 UTC m=+23.779583494"
Jul 27 19:39:55.859207 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:55.859178    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 27 19:39:56.249400 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:56.249232    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-27T19:39:55.859201881Z","UUID":"b33fe333-6abc-4da3-9360-c9f601a37310","Handler":null,"Name":"","Endpoint":""}
Jul 27 19:39:56.251310 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:56.251288    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 27 19:39:56.251421 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:56.251333    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 27 19:39:56.256095 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:56.256078    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:56.256095 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:56.256094    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:56.256236 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:56.256217    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gt55" podUID="6407e54e-d37a-4344-91d3-4494c81246c4"
Jul 27 19:39:56.256391 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:56.256356    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdk4x" podUID="74f89d7b-f93b-4dc3-9e18-41cf466d90bd"
Jul 27 19:39:56.499632 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:56.499523    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v" event={"ID":"a2601d76-e153-4372-a871-3e20499e774a","Type":"ContainerStarted","Data":"765dacaeb8c1aa36066e7361c5595711212fc290147c54f7531bf57dc68e4051"}
Jul 27 19:39:56.501817 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:56.501781    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-96x6d" event={"ID":"580a1b09-1df6-48b5-9b0d-d72cb6806b06","Type":"ContainerStarted","Data":"9830832c82ba1769d5858a4e789c7c333d0ea7f3e5fcde38a227d2a3a34fdba9"}
Jul 27 19:39:56.526661 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:56.526616    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-96x6d" podStartSLOduration=4.504769219 podStartE2EDuration="23.526601773s" totalImagesPullingTime="19.021832554s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:39:56.524885008 +0000 UTC m=+24.754127213" watchObservedRunningTime="2026-07-27 19:39:56.526601773 +0000 UTC m=+24.755843971"
Jul 27 19:39:56.526924 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:56.526897    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nqrzj" podStartSLOduration=4.437674354 podStartE2EDuration="23.526886492s" totalImagesPullingTime="19.089212138s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:39:55.612306813 +0000 UTC m=+23.841549009" watchObservedRunningTime="2026-07-27 19:39:56.526886492 +0000 UTC m=+24.756128690"
Jul 27 19:39:57.256120 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:57.256101    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:39:57.256278 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:57.256192    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fpbzl" podUID="f9f7d637-863f-4a7f-8327-ce46c5076d4d"
Jul 27 19:39:57.510491 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:57.510454    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v" event={"ID":"a2601d76-e153-4372-a871-3e20499e774a","Type":"ContainerStarted","Data":"cf7990135ca60f88744294af3390cdea2e94f626745d7ee43ba4228f73406a87"}
Jul 27 19:39:57.514407 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:57.514383    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp" event={"ID":"6d952ea0-8d44-4913-b659-8a6f6f0b8b25","Type":"ContainerStarted","Data":"c58c56fef6aff3ad5475f6ae352d612724e02293fb446c06f2c55d7f79fee318"}
Jul 27 19:39:57.533352 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:57.533306    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rn87v" podStartSLOduration=2.969762425 podStartE2EDuration="24.533294444s" totalImagesPullingTime="21.563532019s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:39:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:39:57.533039258 +0000 UTC m=+25.762281459" watchObservedRunningTime="2026-07-27 19:39:57.533294444 +0000 UTC m=+25.762536639"
Jul 27 19:39:58.259543 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:58.259330    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:39:58.259795 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:58.259362    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:39:58.259795 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:58.259660    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdk4x" podUID="74f89d7b-f93b-4dc3-9e18-41cf466d90bd"
Jul 27 19:39:58.259795 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:58.259689    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gt55" podUID="6407e54e-d37a-4344-91d3-4494c81246c4"
Jul 27 19:39:59.255790 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:39:59.255767    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:39:59.256246 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:39:59.255890    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fpbzl" podUID="f9f7d637-863f-4a7f-8327-ce46c5076d4d"
Jul 27 19:40:00.256247 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:00.256032    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:40:00.256915 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:00.256106    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:40:00.256915 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:00.256298    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gt55" podUID="6407e54e-d37a-4344-91d3-4494c81246c4"
Jul 27 19:40:00.256915 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:00.256371    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdk4x" podUID="74f89d7b-f93b-4dc3-9e18-41cf466d90bd"
Jul 27 19:40:00.427157 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:00.427134    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tktlt"
Jul 27 19:40:00.427705 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:00.427687    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tktlt"
Jul 27 19:40:00.524129 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:00.524068    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp" event={"ID":"6d952ea0-8d44-4913-b659-8a6f6f0b8b25","Type":"ContainerStarted","Data":"551041dd719fdef85e165852a6f75d5674fc678222fd83dd1fb0b229621153d8"}
Jul 27 19:40:00.524405 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:00.524390    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:40:00.524499 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:00.524476    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:40:00.524555 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:00.524537    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:40:00.525961 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:00.525938    2579 generic.go:356] "Generic (PLEG): container finished" podID="a76e8a73-6dac-43e0-92ec-6269ccd13593" containerID="cc6f7be2d5f8699d853df72fe417d7e0ab761923810fa09b77fa82943dc86a29" exitCode=0
Jul 27 19:40:00.526070 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:00.525973    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lwcg2" event={"ID":"a76e8a73-6dac-43e0-92ec-6269ccd13593","Type":"ContainerDied","Data":"cc6f7be2d5f8699d853df72fe417d7e0ab761923810fa09b77fa82943dc86a29"}
Jul 27 19:40:00.540927 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:00.540837    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:40:00.541009 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:00.540990    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:40:00.557359 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:00.557322    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp" podStartSLOduration=8.473195735000001 podStartE2EDuration="27.557310534s" totalImagesPullingTime="19.084114799s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:40:00.557194943 +0000 UTC m=+28.786437140" watchObservedRunningTime="2026-07-27 19:40:00.557310534 +0000 UTC m=+28.786552777"
Jul 27 19:40:01.255843 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:01.255810    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:40:01.256132 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:01.255949    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fpbzl" podUID="f9f7d637-863f-4a7f-8327-ce46c5076d4d"
Jul 27 19:40:01.825919 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:01.825824    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fpbzl"]
Jul 27 19:40:01.826263 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:01.825966    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:40:01.826263 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:01.826040    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fpbzl" podUID="f9f7d637-863f-4a7f-8327-ce46c5076d4d"
Jul 27 19:40:01.830605 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:01.830585    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7gt55"]
Jul 27 19:40:01.830684 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:01.830666    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:40:01.830742 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:01.830728    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gt55" podUID="6407e54e-d37a-4344-91d3-4494c81246c4"
Jul 27 19:40:01.833582 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:01.833558    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sdk4x"]
Jul 27 19:40:01.833672 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:01.833664    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:40:01.833763 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:01.833745    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdk4x" podUID="74f89d7b-f93b-4dc3-9e18-41cf466d90bd"
Jul 27 19:40:02.532508 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:02.532474    2579 generic.go:356] "Generic (PLEG): container finished" podID="a76e8a73-6dac-43e0-92ec-6269ccd13593" containerID="35fb5f3899163ddb165f9cf8bcb3f5443a2dd72cedda749657099897ad5c393d" exitCode=0
Jul 27 19:40:02.532655 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:02.532579    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lwcg2" event={"ID":"a76e8a73-6dac-43e0-92ec-6269ccd13593","Type":"ContainerDied","Data":"35fb5f3899163ddb165f9cf8bcb3f5443a2dd72cedda749657099897ad5c393d"}
Jul 27 19:40:03.255796 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:03.255763    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:40:03.255796 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:03.255791    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:40:03.256191 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:03.255900    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdk4x" podUID="74f89d7b-f93b-4dc3-9e18-41cf466d90bd"
Jul 27 19:40:03.256191 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:03.255951    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:40:03.256191 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:03.256028    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fpbzl" podUID="f9f7d637-863f-4a7f-8327-ce46c5076d4d"
Jul 27 19:40:03.256191 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:03.256172    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gt55" podUID="6407e54e-d37a-4344-91d3-4494c81246c4"
Jul 27 19:40:04.076943 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:04.076760    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tktlt"
Jul 27 19:40:04.077093 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:04.077081    2579 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 27 19:40:04.077348 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:04.077330    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tktlt"
Jul 27 19:40:04.539077 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:04.539043    2579 generic.go:356] "Generic (PLEG): container finished" podID="a76e8a73-6dac-43e0-92ec-6269ccd13593" containerID="89958b7fefa2cd98dac172ec72ed92676b2bac1772049614b6d1d89e883686e9" exitCode=0
Jul 27 19:40:04.539492 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:04.539102    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lwcg2" event={"ID":"a76e8a73-6dac-43e0-92ec-6269ccd13593","Type":"ContainerDied","Data":"89958b7fefa2cd98dac172ec72ed92676b2bac1772049614b6d1d89e883686e9"}
Jul 27 19:40:05.256000 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:05.255969    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:40:05.256173 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:05.256009    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:40:05.256173 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:05.255975    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:40:05.256173 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:05.256090    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdk4x" podUID="74f89d7b-f93b-4dc3-9e18-41cf466d90bd"
Jul 27 19:40:05.256173 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:05.256134    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gt55" podUID="6407e54e-d37a-4344-91d3-4494c81246c4"
Jul 27 19:40:05.256375 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:05.256209    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fpbzl" podUID="f9f7d637-863f-4a7f-8327-ce46c5076d4d"
Jul 27 19:40:05.969534 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:05.969499    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret\") pod \"global-pull-secret-syncer-7gt55\" (UID: \"6407e54e-d37a-4344-91d3-4494c81246c4\") " pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:40:05.970004 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:05.969561    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs\") pod \"network-metrics-daemon-sdk4x\" (UID: \"74f89d7b-f93b-4dc3-9e18-41cf466d90bd\") " pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:40:05.970004 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:05.969649    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:40:05.970004 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:05.969649    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:40:05.970004 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:05.969709    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs podName:74f89d7b-f93b-4dc3-9e18-41cf466d90bd nodeName:}" failed. No retries permitted until 2026-07-27 19:40:37.969694018 +0000 UTC m=+66.198936210 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs") pod "network-metrics-daemon-sdk4x" (UID: "74f89d7b-f93b-4dc3-9e18-41cf466d90bd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:40:05.970004 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:05.969725    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret podName:6407e54e-d37a-4344-91d3-4494c81246c4 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:37.969717052 +0000 UTC m=+66.198959241 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret") pod "global-pull-secret-syncer-7gt55" (UID: "6407e54e-d37a-4344-91d3-4494c81246c4") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:40:06.171187 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:06.171155    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8ck5\" (UniqueName: \"kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5\") pod \"network-check-target-fpbzl\" (UID: \"f9f7d637-863f-4a7f-8327-ce46c5076d4d\") " pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:40:06.171418 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:06.171399    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:40:06.171484 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:06.171474    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:40:06.171543 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:06.171488    2579 projected.go:196] Error preparing data for projected volume kube-api-access-p8ck5 for pod openshift-network-diagnostics/network-check-target-fpbzl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:40:06.171593 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:06.171545    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5 podName:f9f7d637-863f-4a7f-8327-ce46c5076d4d nodeName:}" failed. No retries permitted until 2026-07-27 19:40:38.171527233 +0000 UTC m=+66.400769474 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-p8ck5" (UniqueName: "kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5") pod "network-check-target-fpbzl" (UID: "f9f7d637-863f-4a7f-8327-ce46c5076d4d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:40:07.060373 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.060340    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-36.ec2.internal" event="NodeReady"
Jul 27 19:40:07.060762 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.060506    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 27 19:40:07.095105 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.095073    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-68dc8dbbdb-vrb2j"]
Jul 27 19:40:07.101725 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.101699    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"]
Jul 27 19:40:07.101879 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.101799    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68dc8dbbdb-vrb2j"
Jul 27 19:40:07.105256 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.105196    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 27 19:40:07.105362 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.105273    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c846f44cd-b9dq2"]
Jul 27 19:40:07.105438 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.105419    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 27 19:40:07.105531 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.105435    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"
Jul 27 19:40:07.105954 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.105935    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 27 19:40:07.106045 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.105937    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 27 19:40:07.107646 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.107576    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 27 19:40:07.107876 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.107832    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 27 19:40:07.107940 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.107877    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 27 19:40:07.107997 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.107944    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 27 19:40:07.109375 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.108563    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66df67d8d5-nx4mf"]
Jul 27 19:40:07.109375 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.108642    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c846f44cd-b9dq2"
Jul 27 19:40:07.110817 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.110793    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-jzfj6\""
Jul 27 19:40:07.111643 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.111621    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 27 19:40:07.114127 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.112939    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-68dc8dbbdb-vrb2j"]
Jul 27 19:40:07.114127 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.113024    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c846f44cd-b9dq2"]
Jul 27 19:40:07.114127 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.113097    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.114328 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.114151    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tct2c"]
Jul 27 19:40:07.114692 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.114669    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 27 19:40:07.114955 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.114935    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 27 19:40:07.115052 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.115034    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 27 19:40:07.117562 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.117540    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4lj5s\""
Jul 27 19:40:07.117801 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.117772    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"]
Jul 27 19:40:07.117909 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.117820    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66df67d8d5-nx4mf"]
Jul 27 19:40:07.117954 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.117933    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tct2c"
Jul 27 19:40:07.121668 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.119740    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 27 19:40:07.121668 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.120038    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 27 19:40:07.121668 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.120148    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mcmp4"]
Jul 27 19:40:07.121668 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.120206    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cqn6v\""
Jul 27 19:40:07.121668 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.120512    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 27 19:40:07.123051 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.123024    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 27 19:40:07.123458 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.123437    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:07.125770 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.125737    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 27 19:40:07.126644 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.125998    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tnv29\""
Jul 27 19:40:07.126644 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.126037    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 27 19:40:07.129135 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.129114    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tct2c"]
Jul 27 19:40:07.132148 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.132131    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mcmp4"]
Jul 27 19:40:07.178221 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.178199    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zxxv\" (UniqueName: \"kubernetes.io/projected/bc53579a-3dbd-4fa8-84e5-0407a1f00e5b-kube-api-access-6zxxv\") pod \"managed-serviceaccount-addon-agent-5c846f44cd-b9dq2\" (UID: \"bc53579a-3dbd-4fa8-84e5-0407a1f00e5b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c846f44cd-b9dq2"
Jul 27 19:40:07.178309 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.178246    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slgvn\" (UniqueName: \"kubernetes.io/projected/225f0ebd-cdb1-44c5-b705-19e759e4dd61-kube-api-access-slgvn\") pod \"ingress-canary-tct2c\" (UID: \"225f0ebd-cdb1-44c5-b705-19e759e4dd61\") " pod="openshift-ingress-canary/ingress-canary-tct2c"
Jul 27 19:40:07.178309 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.178266    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bc53579a-3dbd-4fa8-84e5-0407a1f00e5b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c846f44cd-b9dq2\" (UID: \"bc53579a-3dbd-4fa8-84e5-0407a1f00e5b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c846f44cd-b9dq2"
Jul 27 19:40:07.178390 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.178308    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf2z5\" (UniqueName: \"kubernetes.io/projected/aa006e25-4d61-4397-b366-87d4c776afae-kube-api-access-sf2z5\") pod \"klusterlet-addon-workmgr-68dc8dbbdb-vrb2j\" (UID: \"aa006e25-4d61-4397-b366-87d4c776afae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68dc8dbbdb-vrb2j"
Jul 27 19:40:07.178390 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.178344    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6bbc6b75-0747-42df-a099-8246656515e1-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-c67cbbd4c-bzkw9\" (UID: \"6bbc6b75-0747-42df-a099-8246656515e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"
Jul 27 19:40:07.178482 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.178412    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2343471b-c979-4422-a9d6-f30610099419-installation-pull-secrets\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.178482 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.178464    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6bbc6b75-0747-42df-a099-8246656515e1-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-c67cbbd4c-bzkw9\" (UID: \"6bbc6b75-0747-42df-a099-8246656515e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"
Jul 27 19:40:07.178577 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.178497    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd9dk\" (UniqueName: \"kubernetes.io/projected/66b96198-308c-49c0-bb97-bb8210624656-kube-api-access-zd9dk\") pod \"dns-default-mcmp4\" (UID: \"66b96198-308c-49c0-bb97-bb8210624656\") " pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:07.178577 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.178525    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6bbc6b75-0747-42df-a099-8246656515e1-hub\") pod \"cluster-proxy-proxy-agent-c67cbbd4c-bzkw9\" (UID: \"6bbc6b75-0747-42df-a099-8246656515e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"
Jul 27 19:40:07.178577 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.178547    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.178683 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.178588    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2343471b-c979-4422-a9d6-f30610099419-ca-trust-extracted\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.178683 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.178616    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4ll8\" (UniqueName: \"kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-kube-api-access-q4ll8\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.178683 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.178651    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66b96198-308c-49c0-bb97-bb8210624656-metrics-tls\") pod \"dns-default-mcmp4\" (UID: \"66b96198-308c-49c0-bb97-bb8210624656\") " pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:07.178884 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.178688    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2343471b-c979-4422-a9d6-f30610099419-trusted-ca\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.179003 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.178899    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-bound-sa-token\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.179003 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.178928    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/66b96198-308c-49c0-bb97-bb8210624656-config-volume\") pod \"dns-default-mcmp4\" (UID: \"66b96198-308c-49c0-bb97-bb8210624656\") " pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:07.179003 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.178967    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bbc6b75-0747-42df-a099-8246656515e1-additional-service-ca\") pod \"cluster-proxy-proxy-agent-c67cbbd4c-bzkw9\" (UID: \"6bbc6b75-0747-42df-a099-8246656515e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"
Jul 27 19:40:07.179169 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.179035    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2343471b-c979-4422-a9d6-f30610099419-image-registry-private-configuration\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.179169 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.179116    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/225f0ebd-cdb1-44c5-b705-19e759e4dd61-cert\") pod \"ingress-canary-tct2c\" (UID: \"225f0ebd-cdb1-44c5-b705-19e759e4dd61\") " pod="openshift-ingress-canary/ingress-canary-tct2c"
Jul 27 19:40:07.179169 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.179142    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/aa006e25-4d61-4397-b366-87d4c776afae-klusterlet-config\") pod \"klusterlet-addon-workmgr-68dc8dbbdb-vrb2j\" (UID: \"aa006e25-4d61-4397-b366-87d4c776afae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68dc8dbbdb-vrb2j"
Jul 27 19:40:07.179308 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.179191    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6bbc6b75-0747-42df-a099-8246656515e1-ca\") pod \"cluster-proxy-proxy-agent-c67cbbd4c-bzkw9\" (UID: \"6bbc6b75-0747-42df-a099-8246656515e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"
Jul 27 19:40:07.179308 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.179214    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2343471b-c979-4422-a9d6-f30610099419-registry-certificates\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.179308 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.179250    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66b96198-308c-49c0-bb97-bb8210624656-tmp-dir\") pod \"dns-default-mcmp4\" (UID: \"66b96198-308c-49c0-bb97-bb8210624656\") " pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:07.179308 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.179283    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa006e25-4d61-4397-b366-87d4c776afae-tmp\") pod \"klusterlet-addon-workmgr-68dc8dbbdb-vrb2j\" (UID: \"aa006e25-4d61-4397-b366-87d4c776afae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68dc8dbbdb-vrb2j"
Jul 27 19:40:07.179308 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.179306    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnr59\" (UniqueName: \"kubernetes.io/projected/6bbc6b75-0747-42df-a099-8246656515e1-kube-api-access-dnr59\") pod \"cluster-proxy-proxy-agent-c67cbbd4c-bzkw9\" (UID: \"6bbc6b75-0747-42df-a099-8246656515e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"
Jul 27 19:40:07.256121 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.256094    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:40:07.256295 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.256134    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:40:07.256295 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.256134    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:40:07.258494 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.258468    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 19:40:07.258611 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.258507    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 19:40:07.258611 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.258516    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 19:40:07.258611 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.258555    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 19:40:07.258611 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.258509    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4fnrz\""
Jul 27 19:40:07.258611 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.258509    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-klph9\""
Jul 27 19:40:07.270460 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.270434    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tktlt_967ad4b2-9b0f-4654-be1b-8bcceee7f7c3/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:07.280086 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280067    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bbc6b75-0747-42df-a099-8246656515e1-additional-service-ca\") pod \"cluster-proxy-proxy-agent-c67cbbd4c-bzkw9\" (UID: \"6bbc6b75-0747-42df-a099-8246656515e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"
Jul 27 19:40:07.280232 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280097    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2343471b-c979-4422-a9d6-f30610099419-image-registry-private-configuration\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.280232 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280137    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/225f0ebd-cdb1-44c5-b705-19e759e4dd61-cert\") pod \"ingress-canary-tct2c\" (UID: \"225f0ebd-cdb1-44c5-b705-19e759e4dd61\") " pod="openshift-ingress-canary/ingress-canary-tct2c"
Jul 27 19:40:07.280232 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280160    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/aa006e25-4d61-4397-b366-87d4c776afae-klusterlet-config\") pod \"klusterlet-addon-workmgr-68dc8dbbdb-vrb2j\" (UID: \"aa006e25-4d61-4397-b366-87d4c776afae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68dc8dbbdb-vrb2j"
Jul 27 19:40:07.280232 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280186    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6bbc6b75-0747-42df-a099-8246656515e1-ca\") pod \"cluster-proxy-proxy-agent-c67cbbd4c-bzkw9\" (UID: \"6bbc6b75-0747-42df-a099-8246656515e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"
Jul 27 19:40:07.280232 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280210    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2343471b-c979-4422-a9d6-f30610099419-registry-certificates\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.280232 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280231    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66b96198-308c-49c0-bb97-bb8210624656-tmp-dir\") pod \"dns-default-mcmp4\" (UID: \"66b96198-308c-49c0-bb97-bb8210624656\") " pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:07.280526 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280265    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa006e25-4d61-4397-b366-87d4c776afae-tmp\") pod \"klusterlet-addon-workmgr-68dc8dbbdb-vrb2j\" (UID: \"aa006e25-4d61-4397-b366-87d4c776afae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68dc8dbbdb-vrb2j"
Jul 27 19:40:07.280526 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280290    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnr59\" (UniqueName: \"kubernetes.io/projected/6bbc6b75-0747-42df-a099-8246656515e1-kube-api-access-dnr59\") pod \"cluster-proxy-proxy-agent-c67cbbd4c-bzkw9\" (UID: \"6bbc6b75-0747-42df-a099-8246656515e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"
Jul 27 19:40:07.280526 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280357    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zxxv\" (UniqueName: \"kubernetes.io/projected/bc53579a-3dbd-4fa8-84e5-0407a1f00e5b-kube-api-access-6zxxv\") pod \"managed-serviceaccount-addon-agent-5c846f44cd-b9dq2\" (UID: \"bc53579a-3dbd-4fa8-84e5-0407a1f00e5b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c846f44cd-b9dq2"
Jul 27 19:40:07.280526 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280390    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slgvn\" (UniqueName: \"kubernetes.io/projected/225f0ebd-cdb1-44c5-b705-19e759e4dd61-kube-api-access-slgvn\") pod \"ingress-canary-tct2c\" (UID: \"225f0ebd-cdb1-44c5-b705-19e759e4dd61\") " pod="openshift-ingress-canary/ingress-canary-tct2c"
Jul 27 19:40:07.280526 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280414    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bc53579a-3dbd-4fa8-84e5-0407a1f00e5b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c846f44cd-b9dq2\" (UID: \"bc53579a-3dbd-4fa8-84e5-0407a1f00e5b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c846f44cd-b9dq2"
Jul 27 19:40:07.280526 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280438    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf2z5\" (UniqueName: \"kubernetes.io/projected/aa006e25-4d61-4397-b366-87d4c776afae-kube-api-access-sf2z5\") pod \"klusterlet-addon-workmgr-68dc8dbbdb-vrb2j\" (UID: \"aa006e25-4d61-4397-b366-87d4c776afae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68dc8dbbdb-vrb2j"
Jul 27 19:40:07.280526 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280467    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6bbc6b75-0747-42df-a099-8246656515e1-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-c67cbbd4c-bzkw9\" (UID: \"6bbc6b75-0747-42df-a099-8246656515e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"
Jul 27 19:40:07.280526 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280492    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2343471b-c979-4422-a9d6-f30610099419-installation-pull-secrets\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.280526 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280517    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6bbc6b75-0747-42df-a099-8246656515e1-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-c67cbbd4c-bzkw9\" (UID: \"6bbc6b75-0747-42df-a099-8246656515e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"
Jul 27 19:40:07.280993 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280542    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd9dk\" (UniqueName: \"kubernetes.io/projected/66b96198-308c-49c0-bb97-bb8210624656-kube-api-access-zd9dk\") pod \"dns-default-mcmp4\" (UID: \"66b96198-308c-49c0-bb97-bb8210624656\") " pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:07.280993 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280577    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6bbc6b75-0747-42df-a099-8246656515e1-hub\") pod \"cluster-proxy-proxy-agent-c67cbbd4c-bzkw9\" (UID: \"6bbc6b75-0747-42df-a099-8246656515e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"
Jul 27 19:40:07.280993 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280601    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.280993 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280625    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2343471b-c979-4422-a9d6-f30610099419-ca-trust-extracted\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.280993 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280651    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4ll8\" (UniqueName: \"kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-kube-api-access-q4ll8\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.280993 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280676    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66b96198-308c-49c0-bb97-bb8210624656-metrics-tls\") pod \"dns-default-mcmp4\" (UID: \"66b96198-308c-49c0-bb97-bb8210624656\") " pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:07.280993 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280706    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2343471b-c979-4422-a9d6-f30610099419-trusted-ca\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.280993 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280733    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-bound-sa-token\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.280993 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280760    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/66b96198-308c-49c0-bb97-bb8210624656-config-volume\") pod \"dns-default-mcmp4\" (UID: \"66b96198-308c-49c0-bb97-bb8210624656\") " pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:07.280993 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.280820    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bbc6b75-0747-42df-a099-8246656515e1-additional-service-ca\") pod \"cluster-proxy-proxy-agent-c67cbbd4c-bzkw9\" (UID: \"6bbc6b75-0747-42df-a099-8246656515e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"
Jul 27 19:40:07.281468 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.281160    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2343471b-c979-4422-a9d6-f30610099419-ca-trust-extracted\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.281468 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.281362    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/66b96198-308c-49c0-bb97-bb8210624656-config-volume\") pod \"dns-default-mcmp4\" (UID: \"66b96198-308c-49c0-bb97-bb8210624656\") " pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:07.281891 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.281849    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2343471b-c979-4422-a9d6-f30610099419-registry-certificates\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.282478 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:07.282202    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 19:40:07.282478 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:07.282221    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66df67d8d5-nx4mf: secret "image-registry-tls" not found
Jul 27 19:40:07.282478 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.282250    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2343471b-c979-4422-a9d6-f30610099419-trusted-ca\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.282478 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:07.282288    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls podName:2343471b-c979-4422-a9d6-f30610099419 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:07.782268292 +0000 UTC m=+36.011510469 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls") pod "image-registry-66df67d8d5-nx4mf" (UID: "2343471b-c979-4422-a9d6-f30610099419") : secret "image-registry-tls" not found
Jul 27 19:40:07.282478 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:07.282345    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 19:40:07.282478 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:07.282402    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66b96198-308c-49c0-bb97-bb8210624656-metrics-tls podName:66b96198-308c-49c0-bb97-bb8210624656 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:07.782384942 +0000 UTC m=+36.011627133 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66b96198-308c-49c0-bb97-bb8210624656-metrics-tls") pod "dns-default-mcmp4" (UID: "66b96198-308c-49c0-bb97-bb8210624656") : secret "dns-default-metrics-tls" not found
Jul 27 19:40:07.283505 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.283486    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66b96198-308c-49c0-bb97-bb8210624656-tmp-dir\") pod \"dns-default-mcmp4\" (UID: \"66b96198-308c-49c0-bb97-bb8210624656\") " pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:07.283668 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.283593    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa006e25-4d61-4397-b366-87d4c776afae-tmp\") pod \"klusterlet-addon-workmgr-68dc8dbbdb-vrb2j\" (UID: \"aa006e25-4d61-4397-b366-87d4c776afae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68dc8dbbdb-vrb2j"
Jul 27 19:40:07.285878 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.285831    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2343471b-c979-4422-a9d6-f30610099419-installation-pull-secrets\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.286005 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.285939    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6bbc6b75-0747-42df-a099-8246656515e1-hub\") pod \"cluster-proxy-proxy-agent-c67cbbd4c-bzkw9\" (UID: \"6bbc6b75-0747-42df-a099-8246656515e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"
Jul 27 19:40:07.286300 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.286282    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/225f0ebd-cdb1-44c5-b705-19e759e4dd61-cert\") pod \"ingress-canary-tct2c\" (UID: \"225f0ebd-cdb1-44c5-b705-19e759e4dd61\") " pod="openshift-ingress-canary/ingress-canary-tct2c"
Jul 27 19:40:07.286383 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.286294    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bc53579a-3dbd-4fa8-84e5-0407a1f00e5b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c846f44cd-b9dq2\" (UID: \"bc53579a-3dbd-4fa8-84e5-0407a1f00e5b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c846f44cd-b9dq2"
Jul 27 19:40:07.286521 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.286505    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/aa006e25-4d61-4397-b366-87d4c776afae-klusterlet-config\") pod \"klusterlet-addon-workmgr-68dc8dbbdb-vrb2j\" (UID: \"aa006e25-4d61-4397-b366-87d4c776afae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68dc8dbbdb-vrb2j"
Jul 27 19:40:07.286999 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.286957    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2343471b-c979-4422-a9d6-f30610099419-image-registry-private-configuration\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.287491 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.287468    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6bbc6b75-0747-42df-a099-8246656515e1-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-c67cbbd4c-bzkw9\" (UID: \"6bbc6b75-0747-42df-a099-8246656515e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"
Jul 27 19:40:07.287578 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.287509    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6bbc6b75-0747-42df-a099-8246656515e1-ca\") pod \"cluster-proxy-proxy-agent-c67cbbd4c-bzkw9\" (UID: \"6bbc6b75-0747-42df-a099-8246656515e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"
Jul 27 19:40:07.290296 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.290238    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slgvn\" (UniqueName: \"kubernetes.io/projected/225f0ebd-cdb1-44c5-b705-19e759e4dd61-kube-api-access-slgvn\") pod \"ingress-canary-tct2c\" (UID: \"225f0ebd-cdb1-44c5-b705-19e759e4dd61\") " pod="openshift-ingress-canary/ingress-canary-tct2c"
Jul 27 19:40:07.291105 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.290586    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf2z5\" (UniqueName: \"kubernetes.io/projected/aa006e25-4d61-4397-b366-87d4c776afae-kube-api-access-sf2z5\") pod \"klusterlet-addon-workmgr-68dc8dbbdb-vrb2j\" (UID: \"aa006e25-4d61-4397-b366-87d4c776afae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68dc8dbbdb-vrb2j"
Jul 27 19:40:07.291105 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.290959    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4ll8\" (UniqueName: \"kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-kube-api-access-q4ll8\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.291769 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.291751    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd9dk\" (UniqueName: \"kubernetes.io/projected/66b96198-308c-49c0-bb97-bb8210624656-kube-api-access-zd9dk\") pod \"dns-default-mcmp4\" (UID: \"66b96198-308c-49c0-bb97-bb8210624656\") " pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:07.291935 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.291876    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-bound-sa-token\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.292050 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.292031    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zxxv\" (UniqueName: \"kubernetes.io/projected/bc53579a-3dbd-4fa8-84e5-0407a1f00e5b-kube-api-access-6zxxv\") pod \"managed-serviceaccount-addon-agent-5c846f44cd-b9dq2\" (UID: \"bc53579a-3dbd-4fa8-84e5-0407a1f00e5b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c846f44cd-b9dq2"
Jul 27 19:40:07.292546 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.292525    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnr59\" (UniqueName: \"kubernetes.io/projected/6bbc6b75-0747-42df-a099-8246656515e1-kube-api-access-dnr59\") pod \"cluster-proxy-proxy-agent-c67cbbd4c-bzkw9\" (UID: \"6bbc6b75-0747-42df-a099-8246656515e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"
Jul 27 19:40:07.293390 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.293345    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6bbc6b75-0747-42df-a099-8246656515e1-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-c67cbbd4c-bzkw9\" (UID: \"6bbc6b75-0747-42df-a099-8246656515e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"
Jul 27 19:40:07.373715 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.373646    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tktlt_967ad4b2-9b0f-4654-be1b-8bcceee7f7c3/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:07.414721 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.414677    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68dc8dbbdb-vrb2j"
Jul 27 19:40:07.443813 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.443790    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"
Jul 27 19:40:07.450950 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.450929    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c846f44cd-b9dq2"
Jul 27 19:40:07.475000 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.474980    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tct2c"
Jul 27 19:40:07.784069 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.784029    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:07.784069 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:07.784074    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66b96198-308c-49c0-bb97-bb8210624656-metrics-tls\") pod \"dns-default-mcmp4\" (UID: \"66b96198-308c-49c0-bb97-bb8210624656\") " pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:07.784298 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:07.784208    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 19:40:07.784298 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:07.784231    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66df67d8d5-nx4mf: secret "image-registry-tls" not found
Jul 27 19:40:07.784298 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:07.784260    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 19:40:07.784298 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:07.784299    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls podName:2343471b-c979-4422-a9d6-f30610099419 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:08.784279064 +0000 UTC m=+37.013521243 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls") pod "image-registry-66df67d8d5-nx4mf" (UID: "2343471b-c979-4422-a9d6-f30610099419") : secret "image-registry-tls" not found
Jul 27 19:40:07.784527 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:07.784317    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66b96198-308c-49c0-bb97-bb8210624656-metrics-tls podName:66b96198-308c-49c0-bb97-bb8210624656 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:08.784307496 +0000 UTC m=+37.013549675 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66b96198-308c-49c0-bb97-bb8210624656-metrics-tls") pod "dns-default-mcmp4" (UID: "66b96198-308c-49c0-bb97-bb8210624656") : secret "dns-default-metrics-tls" not found
Jul 27 19:40:08.594824 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:08.594659    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tct2c"]
Jul 27 19:40:08.599159 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:40:08.599135    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod225f0ebd_cdb1_44c5_b705_19e759e4dd61.slice/crio-7718924d605ea917d091713919820115a92b1ef93beccbea5e64efd392221d01 WatchSource:0}: Error finding container 7718924d605ea917d091713919820115a92b1ef93beccbea5e64efd392221d01: Status 404 returned error can't find the container with id 7718924d605ea917d091713919820115a92b1ef93beccbea5e64efd392221d01
Jul 27 19:40:08.606059 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:08.604208    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c846f44cd-b9dq2"]
Jul 27 19:40:08.610170 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:08.606834    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"]
Jul 27 19:40:08.610170 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:08.610080    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-68dc8dbbdb-vrb2j"]
Jul 27 19:40:08.611228 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:40:08.611209    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc53579a_3dbd_4fa8_84e5_0407a1f00e5b.slice/crio-728739c3334f776eba45520a25150db15035df1a093d5e2aec3a4f45ba21bef3 WatchSource:0}: Error finding container 728739c3334f776eba45520a25150db15035df1a093d5e2aec3a4f45ba21bef3: Status 404 returned error can't find the container with id 728739c3334f776eba45520a25150db15035df1a093d5e2aec3a4f45ba21bef3
Jul 27 19:40:08.613212 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:40:08.612908    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6bbc6b75_0747_42df_a099_8246656515e1.slice/crio-624649bf32f55c9fa80e97c043e6b5dab22fcfcca80dcea848ff23d04957fa5f WatchSource:0}: Error finding container 624649bf32f55c9fa80e97c043e6b5dab22fcfcca80dcea848ff23d04957fa5f: Status 404 returned error can't find the container with id 624649bf32f55c9fa80e97c043e6b5dab22fcfcca80dcea848ff23d04957fa5f
Jul 27 19:40:08.613418 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:40:08.613391    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa006e25_4d61_4397_b366_87d4c776afae.slice/crio-8afd6fe619ed2db6c5724eb8ba7d38cf8895beedb23a3bb23ec9421fc7beb6ad WatchSource:0}: Error finding container 8afd6fe619ed2db6c5724eb8ba7d38cf8895beedb23a3bb23ec9421fc7beb6ad: Status 404 returned error can't find the container with id 8afd6fe619ed2db6c5724eb8ba7d38cf8895beedb23a3bb23ec9421fc7beb6ad
Jul 27 19:40:08.794349 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:08.794323    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:08.794475 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:08.794363    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66b96198-308c-49c0-bb97-bb8210624656-metrics-tls\") pod \"dns-default-mcmp4\" (UID: \"66b96198-308c-49c0-bb97-bb8210624656\") " pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:08.794532 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:08.794477    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 19:40:08.794532 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:08.794484    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 19:40:08.794532 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:08.794500    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66df67d8d5-nx4mf: secret "image-registry-tls" not found
Jul 27 19:40:08.794652 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:08.794548    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66b96198-308c-49c0-bb97-bb8210624656-metrics-tls podName:66b96198-308c-49c0-bb97-bb8210624656 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:10.794527804 +0000 UTC m=+39.023770000 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66b96198-308c-49c0-bb97-bb8210624656-metrics-tls") pod "dns-default-mcmp4" (UID: "66b96198-308c-49c0-bb97-bb8210624656") : secret "dns-default-metrics-tls" not found
Jul 27 19:40:08.794652 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:08.794569    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls podName:2343471b-c979-4422-a9d6-f30610099419 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:10.794557271 +0000 UTC m=+39.023799453 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls") pod "image-registry-66df67d8d5-nx4mf" (UID: "2343471b-c979-4422-a9d6-f30610099419") : secret "image-registry-tls" not found
Jul 27 19:40:09.281472 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:09.281427    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tktlt_967ad4b2-9b0f-4654-be1b-8bcceee7f7c3/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:09.381780 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:09.381680    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tktlt_967ad4b2-9b0f-4654-be1b-8bcceee7f7c3/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:09.556242 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:09.556165    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c846f44cd-b9dq2" event={"ID":"bc53579a-3dbd-4fa8-84e5-0407a1f00e5b","Type":"ContainerStarted","Data":"728739c3334f776eba45520a25150db15035df1a093d5e2aec3a4f45ba21bef3"}
Jul 27 19:40:09.557892 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:09.557818    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9" event={"ID":"6bbc6b75-0747-42df-a099-8246656515e1","Type":"ContainerStarted","Data":"624649bf32f55c9fa80e97c043e6b5dab22fcfcca80dcea848ff23d04957fa5f"}
Jul 27 19:40:09.561267 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:09.561240    2579 generic.go:356] "Generic (PLEG): container finished" podID="a76e8a73-6dac-43e0-92ec-6269ccd13593" containerID="1b87be741aaa103e9b710f8ecd4e1a7f74c0a2514773b03b1b05014aca752ca7" exitCode=0
Jul 27 19:40:09.561583 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:09.561562    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lwcg2" event={"ID":"a76e8a73-6dac-43e0-92ec-6269ccd13593","Type":"ContainerDied","Data":"1b87be741aaa103e9b710f8ecd4e1a7f74c0a2514773b03b1b05014aca752ca7"}
Jul 27 19:40:09.566528 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:09.566508    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68dc8dbbdb-vrb2j" event={"ID":"aa006e25-4d61-4397-b366-87d4c776afae","Type":"ContainerStarted","Data":"8afd6fe619ed2db6c5724eb8ba7d38cf8895beedb23a3bb23ec9421fc7beb6ad"}
Jul 27 19:40:09.568107 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:09.568082    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tct2c" event={"ID":"225f0ebd-cdb1-44c5-b705-19e759e4dd61","Type":"ContainerStarted","Data":"7718924d605ea917d091713919820115a92b1ef93beccbea5e64efd392221d01"}
Jul 27 19:40:10.326920 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:10.326803    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tktlt_967ad4b2-9b0f-4654-be1b-8bcceee7f7c3/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:10.390426 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:10.390366    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tktlt_967ad4b2-9b0f-4654-be1b-8bcceee7f7c3/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:10.577362 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:10.577274    2579 generic.go:356] "Generic (PLEG): container finished" podID="a76e8a73-6dac-43e0-92ec-6269ccd13593" containerID="8c7ba4449e6828554aedad8c91f720f8e28bb6d43fcd0cc8f090e150c1abfeb1" exitCode=0
Jul 27 19:40:10.577362 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:10.577327    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lwcg2" event={"ID":"a76e8a73-6dac-43e0-92ec-6269ccd13593","Type":"ContainerDied","Data":"8c7ba4449e6828554aedad8c91f720f8e28bb6d43fcd0cc8f090e150c1abfeb1"}
Jul 27 19:40:10.813966 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:10.813938    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:10.814092 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:10.813981    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66b96198-308c-49c0-bb97-bb8210624656-metrics-tls\") pod \"dns-default-mcmp4\" (UID: \"66b96198-308c-49c0-bb97-bb8210624656\") " pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:10.814151 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:10.814094    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 19:40:10.814151 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:10.814097    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 19:40:10.814151 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:10.814119    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66df67d8d5-nx4mf: secret "image-registry-tls" not found
Jul 27 19:40:10.814289 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:10.814159    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66b96198-308c-49c0-bb97-bb8210624656-metrics-tls podName:66b96198-308c-49c0-bb97-bb8210624656 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:14.814125759 +0000 UTC m=+43.043367956 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66b96198-308c-49c0-bb97-bb8210624656-metrics-tls") pod "dns-default-mcmp4" (UID: "66b96198-308c-49c0-bb97-bb8210624656") : secret "dns-default-metrics-tls" not found
Jul 27 19:40:10.814289 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:10.814177    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls podName:2343471b-c979-4422-a9d6-f30610099419 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:14.814167322 +0000 UTC m=+43.043409500 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls") pod "image-registry-66df67d8d5-nx4mf" (UID: "2343471b-c979-4422-a9d6-f30610099419") : secret "image-registry-tls" not found
Jul 27 19:40:11.244425 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:11.244203    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tktlt_967ad4b2-9b0f-4654-be1b-8bcceee7f7c3/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:11.589696 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:11.589661    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lwcg2" event={"ID":"a76e8a73-6dac-43e0-92ec-6269ccd13593","Type":"ContainerStarted","Data":"5c4660193092450375e9f47d952e4254d11ba70577489e71b1bd8202bc991261"}
Jul 27 19:40:11.614808 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:11.614755    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lwcg2" podStartSLOduration=9.202894814 podStartE2EDuration="38.614740879s" totalImagesPullingTime="29.411846065s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:39:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:40:11.614106487 +0000 UTC m=+39.843348684" watchObservedRunningTime="2026-07-27 19:40:11.614740879 +0000 UTC m=+39.843983070"
Jul 27 19:40:11.737469 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:11.737294    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tktlt_967ad4b2-9b0f-4654-be1b-8bcceee7f7c3/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:11.803270 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:11.803199    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tktlt_967ad4b2-9b0f-4654-be1b-8bcceee7f7c3/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:14.848979 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:14.848945    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:14.849404 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:14.848988    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66b96198-308c-49c0-bb97-bb8210624656-metrics-tls\") pod \"dns-default-mcmp4\" (UID: \"66b96198-308c-49c0-bb97-bb8210624656\") " pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:14.849404 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:14.849112    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 19:40:14.849404 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:14.849129    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66df67d8d5-nx4mf: secret "image-registry-tls" not found
Jul 27 19:40:14.849404 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:14.849183    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls podName:2343471b-c979-4422-a9d6-f30610099419 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:22.84916266 +0000 UTC m=+51.078404858 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls") pod "image-registry-66df67d8d5-nx4mf" (UID: "2343471b-c979-4422-a9d6-f30610099419") : secret "image-registry-tls" not found
Jul 27 19:40:14.849404 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:14.849182    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 19:40:14.849404 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:14.849247    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66b96198-308c-49c0-bb97-bb8210624656-metrics-tls podName:66b96198-308c-49c0-bb97-bb8210624656 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:22.849227467 +0000 UTC m=+51.078469659 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66b96198-308c-49c0-bb97-bb8210624656-metrics-tls") pod "dns-default-mcmp4" (UID: "66b96198-308c-49c0-bb97-bb8210624656") : secret "dns-default-metrics-tls" not found
Jul 27 19:40:16.603779 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:16.603692    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tct2c" event={"ID":"225f0ebd-cdb1-44c5-b705-19e759e4dd61","Type":"ContainerStarted","Data":"f1493881b9c587ae449bd1576b2b428fa27900c4cd7bb1f7c27b77961dfdb9e3"}
Jul 27 19:40:16.605621 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:16.605594    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9" event={"ID":"6bbc6b75-0747-42df-a099-8246656515e1","Type":"ContainerStarted","Data":"8f0f3313d9e61d2c09392e74f5a4a0f5f206dbd7a1664b79cdaa76379c9a866c"}
Jul 27 19:40:16.605718 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:16.605633    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9" event={"ID":"6bbc6b75-0747-42df-a099-8246656515e1","Type":"ContainerStarted","Data":"4c348a2671004b3072bf61b659fcadfc6ed21dbd12e169a26c55d867444a5725"}
Jul 27 19:40:16.606967 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:16.606942    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68dc8dbbdb-vrb2j" event={"ID":"aa006e25-4d61-4397-b366-87d4c776afae","Type":"ContainerStarted","Data":"891bf15befc8fb3966ae49f8346f63917a8528a647bb0295a31e97263c1135fe"}
Jul 27 19:40:16.607160 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:16.607145    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68dc8dbbdb-vrb2j"
Jul 27 19:40:16.608522 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:16.608505    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68dc8dbbdb-vrb2j"
Jul 27 19:40:16.628266 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:16.628228    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tct2c" podStartSLOduration=1.889247704 podStartE2EDuration="9.628214954s" totalImagesPullingTime="7.73896725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:40:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:40:16.627484338 +0000 UTC m=+44.856726535" watchObservedRunningTime="2026-07-27 19:40:16.628214954 +0000 UTC m=+44.857457151"
Jul 27 19:40:16.656331 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:16.656216    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68dc8dbbdb-vrb2j" podStartSLOduration=35.930071349 podStartE2EDuration="43.656199088s" totalImagesPullingTime="7.726127739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:40:16.655875546 +0000 UTC m=+44.885117742" watchObservedRunningTime="2026-07-27 19:40:16.656199088 +0000 UTC m=+44.885441289"
Jul 27 19:40:17.384377 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.384340    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tktlt_967ad4b2-9b0f-4654-be1b-8bcceee7f7c3/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:17.440786 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.440754    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tktlt_967ad4b2-9b0f-4654-be1b-8bcceee7f7c3/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:17.612251 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.612217    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c846f44cd-b9dq2" event={"ID":"bc53579a-3dbd-4fa8-84e5-0407a1f00e5b","Type":"ContainerStarted","Data":"22e9b46d0d32dbca5c26876206d3d58c4fc2981d1098692aad0498c80c440f38"}
Jul 27 19:40:17.614229 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.614202    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9" event={"ID":"6bbc6b75-0747-42df-a099-8246656515e1","Type":"ContainerStarted","Data":"4dfe8c7062904c17fb30eaa7d9c9a29f7e562e5e56aebb81d03844a20f544d56"}
Jul 27 19:40:17.633909 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.633850    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c846f44cd-b9dq2" podStartSLOduration=36.526547882 podStartE2EDuration="44.633835782s" totalImagesPullingTime="8.1072879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:40:17.632954201 +0000 UTC m=+45.862196400" watchObservedRunningTime="2026-07-27 19:40:17.633835782 +0000 UTC m=+45.863077979"
Jul 27 19:40:17.658987 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.658903    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9" podStartSLOduration=36.932086673 podStartE2EDuration="44.658887649s" totalImagesPullingTime="7.726800976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:40:17.657439716 +0000 UTC m=+45.886681913" watchObservedRunningTime="2026-07-27 19:40:17.658887649 +0000 UTC m=+45.888129849"
Jul 27 19:40:17.670403 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.670380    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bpst8"]
Jul 27 19:40:17.691965 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.691945    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-xfztm"]
Jul 27 19:40:17.692089 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.692070    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bpst8"
Jul 27 19:40:17.694385 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.694366    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 27 19:40:17.694533 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.694517    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-76cfl\""
Jul 27 19:40:17.694646 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.694630    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:40:17.694868 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.694844    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 27 19:40:17.695078 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.695066    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 27 19:40:17.715988 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.715965    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6tml7"]
Jul 27 19:40:17.716101 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.716087    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-xfztm"
Jul 27 19:40:17.719278 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.718523    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 27 19:40:17.719278 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.718758    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 27 19:40:17.719278 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.718899    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:40:17.719278 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.719153    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 27 19:40:17.719278 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.719153    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-gm8hw\""
Jul 27 19:40:17.725468 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.725448    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 27 19:40:17.740048 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.740028    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-5tg5r"]
Jul 27 19:40:17.740158 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.740145    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6tml7"
Jul 27 19:40:17.742089 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.742070    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 27 19:40:17.742175 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.742073    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:40:17.742175 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.742117    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 27 19:40:17.742424 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.742411    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6xcrb\""
Jul 27 19:40:17.758267 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.758250    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bpst8"]
Jul 27 19:40:17.758338 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.758276    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-5tg5r"]
Jul 27 19:40:17.758338 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.758289    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-xfztm"]
Jul 27 19:40:17.758338 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.758301    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6tml7"]
Jul 27 19:40:17.758473 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.758346    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-5tg5r"
Jul 27 19:40:17.760073 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.760001    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-cmngv\""
Jul 27 19:40:17.761133 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.761116    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx29v"]
Jul 27 19:40:17.769994 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.769976    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e814129b-3b96-4c75-ba5b-480e39d6d71e-trusted-ca\") pod \"console-operator-584dcb6b54-xfztm\" (UID: \"e814129b-3b96-4c75-ba5b-480e39d6d71e\") " pod="openshift-console-operator/console-operator-584dcb6b54-xfztm"
Jul 27 19:40:17.770075 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.770008    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zb5f\" (UniqueName: \"kubernetes.io/projected/c91d5e40-730e-4dfb-814a-f95776e4b5e9-kube-api-access-4zb5f\") pod \"service-ca-operator-5d6c75f84f-bpst8\" (UID: \"c91d5e40-730e-4dfb-814a-f95776e4b5e9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bpst8"
Jul 27 19:40:17.770075 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.770044    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e814129b-3b96-4c75-ba5b-480e39d6d71e-config\") pod \"console-operator-584dcb6b54-xfztm\" (UID: \"e814129b-3b96-4c75-ba5b-480e39d6d71e\") " pod="openshift-console-operator/console-operator-584dcb6b54-xfztm"
Jul 27 19:40:17.770284 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.770174    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8298c2f2-f22a-41cf-8202-ba1c367add65-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-6tml7\" (UID: \"8298c2f2-f22a-41cf-8202-ba1c367add65\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6tml7"
Jul 27 19:40:17.770284 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.770208    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c91d5e40-730e-4dfb-814a-f95776e4b5e9-serving-cert\") pod \"service-ca-operator-5d6c75f84f-bpst8\" (UID: \"c91d5e40-730e-4dfb-814a-f95776e4b5e9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bpst8"
Jul 27 19:40:17.770284 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.770248    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vvfr\" (UniqueName: \"kubernetes.io/projected/e814129b-3b96-4c75-ba5b-480e39d6d71e-kube-api-access-4vvfr\") pod \"console-operator-584dcb6b54-xfztm\" (UID: \"e814129b-3b96-4c75-ba5b-480e39d6d71e\") " pod="openshift-console-operator/console-operator-584dcb6b54-xfztm"
Jul 27 19:40:17.770284 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.770274    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c91d5e40-730e-4dfb-814a-f95776e4b5e9-config\") pod \"service-ca-operator-5d6c75f84f-bpst8\" (UID: \"c91d5e40-730e-4dfb-814a-f95776e4b5e9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bpst8"
Jul 27 19:40:17.770516 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.770305    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e814129b-3b96-4c75-ba5b-480e39d6d71e-serving-cert\") pod \"console-operator-584dcb6b54-xfztm\" (UID: \"e814129b-3b96-4c75-ba5b-480e39d6d71e\") " pod="openshift-console-operator/console-operator-584dcb6b54-xfztm"
Jul 27 19:40:17.770516 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.770339    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7gzd\" (UniqueName: \"kubernetes.io/projected/8298c2f2-f22a-41cf-8202-ba1c367add65-kube-api-access-j7gzd\") pod \"cluster-samples-operator-6748489656-6tml7\" (UID: \"8298c2f2-f22a-41cf-8202-ba1c367add65\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6tml7"
Jul 27 19:40:17.782124 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.782106    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp"]
Jul 27 19:40:17.782233 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.782219    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx29v"
Jul 27 19:40:17.783998 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.783975    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 27 19:40:17.784081 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.784031    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:40:17.784333 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.784316    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-lbnks\""
Jul 27 19:40:17.803337 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.803318    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx29v"]
Jul 27 19:40:17.803421 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.803350    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl"]
Jul 27 19:40:17.803463 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.803431    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp"
Jul 27 19:40:17.805402 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.805380    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 27 19:40:17.805570 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.805555    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 27 19:40:17.805618 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.805598    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 27 19:40:17.806184 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.806164    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 27 19:40:17.806273 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.806167    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-vzkmz\""
Jul 27 19:40:17.839342 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.839326    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-n9pwk"]
Jul 27 19:40:17.839455 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.839442    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl"
Jul 27 19:40:17.841543 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.841524    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:40:17.841543 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.841531    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-5gdzb\""
Jul 27 19:40:17.841699 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.841529    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 27 19:40:17.841699 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.841530    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 27 19:40:17.841699 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.841531    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 27 19:40:17.861509 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.861485    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp"]
Jul 27 19:40:17.861600 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.861519    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl"]
Jul 27 19:40:17.861600 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.861538    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-n9pwk"]
Jul 27 19:40:17.861713 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.861609    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-n9pwk"
Jul 27 19:40:17.863539 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.863521    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 27 19:40:17.863768 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.863750    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 27 19:40:17.864080 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.864066    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 27 19:40:17.864164 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.864131    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 27 19:40:17.864226 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.864174    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-pfqh7\""
Jul 27 19:40:17.864484 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.864469    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5f5b9f5469-b26r4"]
Jul 27 19:40:17.871563 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.871539    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 27 19:40:17.872128 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.872105    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5wz8\" (UniqueName: \"kubernetes.io/projected/bd0782d2-c168-41d3-bc16-bf48f1453f20-kube-api-access-w5wz8\") pod \"volume-data-source-validator-549c6d54cf-hx29v\" (UID: \"bd0782d2-c168-41d3-bc16-bf48f1453f20\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx29v"
Jul 27 19:40:17.872215 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.872187    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8298c2f2-f22a-41cf-8202-ba1c367add65-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-6tml7\" (UID: \"8298c2f2-f22a-41cf-8202-ba1c367add65\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6tml7"
Jul 27 19:40:17.872291 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.872220    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c91d5e40-730e-4dfb-814a-f95776e4b5e9-serving-cert\") pod \"service-ca-operator-5d6c75f84f-bpst8\" (UID: \"c91d5e40-730e-4dfb-814a-f95776e4b5e9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bpst8"
Jul 27 19:40:17.872291 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.872267    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vvfr\" (UniqueName: \"kubernetes.io/projected/e814129b-3b96-4c75-ba5b-480e39d6d71e-kube-api-access-4vvfr\") pod \"console-operator-584dcb6b54-xfztm\" (UID: \"e814129b-3b96-4c75-ba5b-480e39d6d71e\") " pod="openshift-console-operator/console-operator-584dcb6b54-xfztm"
Jul 27 19:40:17.872384 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.872294    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0a45576e-6f99-4ff2-bdbe-934a603e6756-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kx5wp\" (UID: \"0a45576e-6f99-4ff2-bdbe-934a603e6756\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp"
Jul 27 19:40:17.872384 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.872330    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c91d5e40-730e-4dfb-814a-f95776e4b5e9-config\") pod \"service-ca-operator-5d6c75f84f-bpst8\" (UID: \"c91d5e40-730e-4dfb-814a-f95776e4b5e9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bpst8"
Jul 27 19:40:17.872384 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.872364    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e814129b-3b96-4c75-ba5b-480e39d6d71e-serving-cert\") pod \"console-operator-584dcb6b54-xfztm\" (UID: \"e814129b-3b96-4c75-ba5b-480e39d6d71e\") " pod="openshift-console-operator/console-operator-584dcb6b54-xfztm"
Jul 27 19:40:17.872525 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.872408    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7gzd\" (UniqueName: \"kubernetes.io/projected/8298c2f2-f22a-41cf-8202-ba1c367add65-kube-api-access-j7gzd\") pod \"cluster-samples-operator-6748489656-6tml7\" (UID: \"8298c2f2-f22a-41cf-8202-ba1c367add65\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6tml7"
Jul 27 19:40:17.872525 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.872443    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b1f4f700-7ce1-4669-a383-793dfbcf4f2e-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl\" (UID: \"b1f4f700-7ce1-4669-a383-793dfbcf4f2e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl"
Jul 27 19:40:17.872525 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.872474    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s7dx\" (UniqueName: \"kubernetes.io/projected/49b1c33d-0c79-4aa6-ba6a-d63ca54cea64-kube-api-access-8s7dx\") pod \"network-check-source-7d88cb55d8-5tg5r\" (UID: \"49b1c33d-0c79-4aa6-ba6a-d63ca54cea64\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-5tg5r"
Jul 27 19:40:17.872525 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.872506    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e814129b-3b96-4c75-ba5b-480e39d6d71e-trusted-ca\") pod \"console-operator-584dcb6b54-xfztm\" (UID: \"e814129b-3b96-4c75-ba5b-480e39d6d71e\") " pod="openshift-console-operator/console-operator-584dcb6b54-xfztm"
Jul 27 19:40:17.872713 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.872541    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0a45576e-6f99-4ff2-bdbe-934a603e6756-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-kx5wp\" (UID: \"0a45576e-6f99-4ff2-bdbe-934a603e6756\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp"
Jul 27 19:40:17.872713 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.872580    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zb5f\" (UniqueName: \"kubernetes.io/projected/c91d5e40-730e-4dfb-814a-f95776e4b5e9-kube-api-access-4zb5f\") pod \"service-ca-operator-5d6c75f84f-bpst8\" (UID: \"c91d5e40-730e-4dfb-814a-f95776e4b5e9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bpst8"
Jul 27 19:40:17.872713 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.872612    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b1f4f700-7ce1-4669-a383-793dfbcf4f2e-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl\" (UID: \"b1f4f700-7ce1-4669-a383-793dfbcf4f2e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl"
Jul 27 19:40:17.872713 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.872664    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e814129b-3b96-4c75-ba5b-480e39d6d71e-config\") pod \"console-operator-584dcb6b54-xfztm\" (UID: \"e814129b-3b96-4c75-ba5b-480e39d6d71e\") " pod="openshift-console-operator/console-operator-584dcb6b54-xfztm"
Jul 27 19:40:17.872713 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.872698    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p2cv\" (UniqueName: \"kubernetes.io/projected/0a45576e-6f99-4ff2-bdbe-934a603e6756-kube-api-access-4p2cv\") pod \"cluster-monitoring-operator-548b489cb8-kx5wp\" (UID: \"0a45576e-6f99-4ff2-bdbe-934a603e6756\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp"
Jul 27 19:40:17.872954 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.872733    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sjz2\" (UniqueName: \"kubernetes.io/projected/b1f4f700-7ce1-4669-a383-793dfbcf4f2e-kube-api-access-7sjz2\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl\" (UID: \"b1f4f700-7ce1-4669-a383-793dfbcf4f2e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl"
Jul 27 19:40:17.873003 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:17.872954    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 19:40:17.873046 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:17.873016    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8298c2f2-f22a-41cf-8202-ba1c367add65-samples-operator-tls podName:8298c2f2-f22a-41cf-8202-ba1c367add65 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:18.372998037 +0000 UTC m=+46.602240226 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8298c2f2-f22a-41cf-8202-ba1c367add65-samples-operator-tls") pod "cluster-samples-operator-6748489656-6tml7" (UID: "8298c2f2-f22a-41cf-8202-ba1c367add65") : secret "samples-operator-tls" not found
Jul 27 19:40:17.874119 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.874089    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c91d5e40-730e-4dfb-814a-f95776e4b5e9-config\") pod \"service-ca-operator-5d6c75f84f-bpst8\" (UID: \"c91d5e40-730e-4dfb-814a-f95776e4b5e9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bpst8"
Jul 27 19:40:17.874267 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.874205    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e814129b-3b96-4c75-ba5b-480e39d6d71e-config\") pod \"console-operator-584dcb6b54-xfztm\" (UID: \"e814129b-3b96-4c75-ba5b-480e39d6d71e\") " pod="openshift-console-operator/console-operator-584dcb6b54-xfztm"
Jul 27 19:40:17.874342 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.874325    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e814129b-3b96-4c75-ba5b-480e39d6d71e-trusted-ca\") pod \"console-operator-584dcb6b54-xfztm\" (UID: \"e814129b-3b96-4c75-ba5b-480e39d6d71e\") " pod="openshift-console-operator/console-operator-584dcb6b54-xfztm"
Jul 27 19:40:17.877481 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.877461    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c91d5e40-730e-4dfb-814a-f95776e4b5e9-serving-cert\") pod \"service-ca-operator-5d6c75f84f-bpst8\" (UID: \"c91d5e40-730e-4dfb-814a-f95776e4b5e9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bpst8"
Jul 27 19:40:17.877552 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.877492    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e814129b-3b96-4c75-ba5b-480e39d6d71e-serving-cert\") pod \"console-operator-584dcb6b54-xfztm\" (UID: \"e814129b-3b96-4c75-ba5b-480e39d6d71e\") " pod="openshift-console-operator/console-operator-584dcb6b54-xfztm"
Jul 27 19:40:17.883626 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.883604    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5f5b9f5469-b26r4"]
Jul 27 19:40:17.883727 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.883695    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:17.886103 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.886083    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 27 19:40:17.886196 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.886146    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tmsww\""
Jul 27 19:40:17.886312 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.886296    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 27 19:40:17.886419 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.886402    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 27 19:40:17.886495 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.886481    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 27 19:40:17.886655 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.886626    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 27 19:40:17.886838 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.886821    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 27 19:40:17.887249 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.887222    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7gzd\" (UniqueName: \"kubernetes.io/projected/8298c2f2-f22a-41cf-8202-ba1c367add65-kube-api-access-j7gzd\") pod \"cluster-samples-operator-6748489656-6tml7\" (UID: \"8298c2f2-f22a-41cf-8202-ba1c367add65\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6tml7"
Jul 27 19:40:17.887470 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.887452    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zb5f\" (UniqueName: \"kubernetes.io/projected/c91d5e40-730e-4dfb-814a-f95776e4b5e9-kube-api-access-4zb5f\") pod \"service-ca-operator-5d6c75f84f-bpst8\" (UID: \"c91d5e40-730e-4dfb-814a-f95776e4b5e9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bpst8"
Jul 27 19:40:17.888029 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.888009    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vvfr\" (UniqueName: \"kubernetes.io/projected/e814129b-3b96-4c75-ba5b-480e39d6d71e-kube-api-access-4vvfr\") pod \"console-operator-584dcb6b54-xfztm\" (UID: \"e814129b-3b96-4c75-ba5b-480e39d6d71e\") " pod="openshift-console-operator/console-operator-584dcb6b54-xfztm"
Jul 27 19:40:17.973434 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.973415    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6c35228-a7c8-44a3-9b48-63c27bb72b25-service-ca-bundle\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:17.973527 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.973443    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-default-certificate\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:17.973527 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.973464    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-metrics-certs\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:17.973527 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.973516    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/096c978c-0cb6-4ff6-8fba-93b2113f24a4-serving-cert\") pod \"insights-operator-5885fdb759-n9pwk\" (UID: \"096c978c-0cb6-4ff6-8fba-93b2113f24a4\") " pod="openshift-insights/insights-operator-5885fdb759-n9pwk"
Jul 27 19:40:17.973678 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.973535    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vfkj\" (UniqueName: \"kubernetes.io/projected/096c978c-0cb6-4ff6-8fba-93b2113f24a4-kube-api-access-6vfkj\") pod \"insights-operator-5885fdb759-n9pwk\" (UID: \"096c978c-0cb6-4ff6-8fba-93b2113f24a4\") " pod="openshift-insights/insights-operator-5885fdb759-n9pwk"
Jul 27 19:40:17.973678 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.973557    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b1f4f700-7ce1-4669-a383-793dfbcf4f2e-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl\" (UID: \"b1f4f700-7ce1-4669-a383-793dfbcf4f2e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl"
Jul 27 19:40:17.973678 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.973574    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s7dx\" (UniqueName: \"kubernetes.io/projected/49b1c33d-0c79-4aa6-ba6a-d63ca54cea64-kube-api-access-8s7dx\") pod \"network-check-source-7d88cb55d8-5tg5r\" (UID: \"49b1c33d-0c79-4aa6-ba6a-d63ca54cea64\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-5tg5r"
Jul 27 19:40:17.973678 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.973589    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/096c978c-0cb6-4ff6-8fba-93b2113f24a4-tmp\") pod \"insights-operator-5885fdb759-n9pwk\" (UID: \"096c978c-0cb6-4ff6-8fba-93b2113f24a4\") " pod="openshift-insights/insights-operator-5885fdb759-n9pwk"
Jul 27 19:40:17.973678 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.973603    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-stats-auth\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:17.973678 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.973667    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0a45576e-6f99-4ff2-bdbe-934a603e6756-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-kx5wp\" (UID: \"0a45576e-6f99-4ff2-bdbe-934a603e6756\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp"
Jul 27 19:40:17.973994 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.973684    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/096c978c-0cb6-4ff6-8fba-93b2113f24a4-snapshots\") pod \"insights-operator-5885fdb759-n9pwk\" (UID: \"096c978c-0cb6-4ff6-8fba-93b2113f24a4\") " pod="openshift-insights/insights-operator-5885fdb759-n9pwk"
Jul 27 19:40:17.973994 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.973705    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9msp\" (UniqueName: \"kubernetes.io/projected/c6c35228-a7c8-44a3-9b48-63c27bb72b25-kube-api-access-j9msp\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:17.973994 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.973733    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b1f4f700-7ce1-4669-a383-793dfbcf4f2e-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl\" (UID: \"b1f4f700-7ce1-4669-a383-793dfbcf4f2e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl"
Jul 27 19:40:17.973994 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.973795    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p2cv\" (UniqueName: \"kubernetes.io/projected/0a45576e-6f99-4ff2-bdbe-934a603e6756-kube-api-access-4p2cv\") pod \"cluster-monitoring-operator-548b489cb8-kx5wp\" (UID: \"0a45576e-6f99-4ff2-bdbe-934a603e6756\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp"
Jul 27 19:40:17.973994 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.973834    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sjz2\" (UniqueName: \"kubernetes.io/projected/b1f4f700-7ce1-4669-a383-793dfbcf4f2e-kube-api-access-7sjz2\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl\" (UID: \"b1f4f700-7ce1-4669-a383-793dfbcf4f2e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl"
Jul 27 19:40:17.973994 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.973896    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/096c978c-0cb6-4ff6-8fba-93b2113f24a4-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-n9pwk\" (UID: \"096c978c-0cb6-4ff6-8fba-93b2113f24a4\") " pod="openshift-insights/insights-operator-5885fdb759-n9pwk"
Jul 27 19:40:17.973994 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.973931    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5wz8\" (UniqueName: \"kubernetes.io/projected/bd0782d2-c168-41d3-bc16-bf48f1453f20-kube-api-access-w5wz8\") pod \"volume-data-source-validator-549c6d54cf-hx29v\" (UID: \"bd0782d2-c168-41d3-bc16-bf48f1453f20\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx29v"
Jul 27 19:40:17.973994 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.973959    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/096c978c-0cb6-4ff6-8fba-93b2113f24a4-service-ca-bundle\") pod \"insights-operator-5885fdb759-n9pwk\" (UID: \"096c978c-0cb6-4ff6-8fba-93b2113f24a4\") " pod="openshift-insights/insights-operator-5885fdb759-n9pwk"
Jul 27 19:40:17.974541 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.974054    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b1f4f700-7ce1-4669-a383-793dfbcf4f2e-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl\" (UID: \"b1f4f700-7ce1-4669-a383-793dfbcf4f2e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl"
Jul 27 19:40:17.974541 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.974085    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0a45576e-6f99-4ff2-bdbe-934a603e6756-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kx5wp\" (UID: \"0a45576e-6f99-4ff2-bdbe-934a603e6756\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp"
Jul 27 19:40:17.974541 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:17.974182    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 19:40:17.974541 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:17.974232    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a45576e-6f99-4ff2-bdbe-934a603e6756-cluster-monitoring-operator-tls podName:0a45576e-6f99-4ff2-bdbe-934a603e6756 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:18.474215768 +0000 UTC m=+46.703457947 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0a45576e-6f99-4ff2-bdbe-934a603e6756-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kx5wp" (UID: "0a45576e-6f99-4ff2-bdbe-934a603e6756") : secret "cluster-monitoring-operator-tls" not found
Jul 27 19:40:17.974541 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.974368    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0a45576e-6f99-4ff2-bdbe-934a603e6756-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-kx5wp\" (UID: \"0a45576e-6f99-4ff2-bdbe-934a603e6756\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp"
Jul 27 19:40:17.975805 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.975786    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b1f4f700-7ce1-4669-a383-793dfbcf4f2e-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl\" (UID: \"b1f4f700-7ce1-4669-a383-793dfbcf4f2e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl"
Jul 27 19:40:17.983227 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.983209    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s7dx\" (UniqueName: \"kubernetes.io/projected/49b1c33d-0c79-4aa6-ba6a-d63ca54cea64-kube-api-access-8s7dx\") pod \"network-check-source-7d88cb55d8-5tg5r\" (UID: \"49b1c33d-0c79-4aa6-ba6a-d63ca54cea64\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-5tg5r"
Jul 27 19:40:17.984512 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.984494    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sjz2\" (UniqueName: \"kubernetes.io/projected/b1f4f700-7ce1-4669-a383-793dfbcf4f2e-kube-api-access-7sjz2\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl\" (UID: \"b1f4f700-7ce1-4669-a383-793dfbcf4f2e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl"
Jul 27 19:40:17.984790 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.984766    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p2cv\" (UniqueName: \"kubernetes.io/projected/0a45576e-6f99-4ff2-bdbe-934a603e6756-kube-api-access-4p2cv\") pod \"cluster-monitoring-operator-548b489cb8-kx5wp\" (UID: \"0a45576e-6f99-4ff2-bdbe-934a603e6756\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp"
Jul 27 19:40:17.984839 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:17.984795    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5wz8\" (UniqueName: \"kubernetes.io/projected/bd0782d2-c168-41d3-bc16-bf48f1453f20-kube-api-access-w5wz8\") pod \"volume-data-source-validator-549c6d54cf-hx29v\" (UID: \"bd0782d2-c168-41d3-bc16-bf48f1453f20\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx29v"
Jul 27 19:40:18.001581 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.001568    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bpst8"
Jul 27 19:40:18.025687 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.025669    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-xfztm"
Jul 27 19:40:18.036898 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.036851    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tktlt_967ad4b2-9b0f-4654-be1b-8bcceee7f7c3/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:18.067843 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.067818    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-5tg5r"
Jul 27 19:40:18.075196 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.075174    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/096c978c-0cb6-4ff6-8fba-93b2113f24a4-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-n9pwk\" (UID: \"096c978c-0cb6-4ff6-8fba-93b2113f24a4\") " pod="openshift-insights/insights-operator-5885fdb759-n9pwk"
Jul 27 19:40:18.075264 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.075215    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/096c978c-0cb6-4ff6-8fba-93b2113f24a4-service-ca-bundle\") pod \"insights-operator-5885fdb759-n9pwk\" (UID: \"096c978c-0cb6-4ff6-8fba-93b2113f24a4\") " pod="openshift-insights/insights-operator-5885fdb759-n9pwk"
Jul 27 19:40:18.075322 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.075293    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6c35228-a7c8-44a3-9b48-63c27bb72b25-service-ca-bundle\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:18.075370 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.075326    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-default-certificate\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:18.075423 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.075365    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-metrics-certs\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:18.075423 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.075404    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/096c978c-0cb6-4ff6-8fba-93b2113f24a4-serving-cert\") pod \"insights-operator-5885fdb759-n9pwk\" (UID: \"096c978c-0cb6-4ff6-8fba-93b2113f24a4\") " pod="openshift-insights/insights-operator-5885fdb759-n9pwk"
Jul 27 19:40:18.075517 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.075429    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vfkj\" (UniqueName: \"kubernetes.io/projected/096c978c-0cb6-4ff6-8fba-93b2113f24a4-kube-api-access-6vfkj\") pod \"insights-operator-5885fdb759-n9pwk\" (UID: \"096c978c-0cb6-4ff6-8fba-93b2113f24a4\") " pod="openshift-insights/insights-operator-5885fdb759-n9pwk"
Jul 27 19:40:18.075517 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.075459    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/096c978c-0cb6-4ff6-8fba-93b2113f24a4-tmp\") pod \"insights-operator-5885fdb759-n9pwk\" (UID: \"096c978c-0cb6-4ff6-8fba-93b2113f24a4\") " pod="openshift-insights/insights-operator-5885fdb759-n9pwk"
Jul 27 19:40:18.075517 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.075489    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-stats-auth\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:18.075664 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:18.075529    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c6c35228-a7c8-44a3-9b48-63c27bb72b25-service-ca-bundle podName:c6c35228-a7c8-44a3-9b48-63c27bb72b25 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:18.575508596 +0000 UTC m=+46.804750787 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c6c35228-a7c8-44a3-9b48-63c27bb72b25-service-ca-bundle") pod "router-default-5f5b9f5469-b26r4" (UID: "c6c35228-a7c8-44a3-9b48-63c27bb72b25") : configmap references non-existent config key: service-ca.crt
Jul 27 19:40:18.075664 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.075559    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/096c978c-0cb6-4ff6-8fba-93b2113f24a4-snapshots\") pod \"insights-operator-5885fdb759-n9pwk\" (UID: \"096c978c-0cb6-4ff6-8fba-93b2113f24a4\") " pod="openshift-insights/insights-operator-5885fdb759-n9pwk"
Jul 27 19:40:18.075664 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.075590    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9msp\" (UniqueName: \"kubernetes.io/projected/c6c35228-a7c8-44a3-9b48-63c27bb72b25-kube-api-access-j9msp\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:18.075664 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:18.075631    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 19:40:18.075892 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:18.075698    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-metrics-certs podName:c6c35228-a7c8-44a3-9b48-63c27bb72b25 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:18.57568163 +0000 UTC m=+46.804923840 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-metrics-certs") pod "router-default-5f5b9f5469-b26r4" (UID: "c6c35228-a7c8-44a3-9b48-63c27bb72b25") : secret "router-metrics-certs-default" not found
Jul 27 19:40:18.076367 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.076339    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/096c978c-0cb6-4ff6-8fba-93b2113f24a4-snapshots\") pod \"insights-operator-5885fdb759-n9pwk\" (UID: \"096c978c-0cb6-4ff6-8fba-93b2113f24a4\") " pod="openshift-insights/insights-operator-5885fdb759-n9pwk"
Jul 27 19:40:18.076457 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.076386    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/096c978c-0cb6-4ff6-8fba-93b2113f24a4-tmp\") pod \"insights-operator-5885fdb759-n9pwk\" (UID: \"096c978c-0cb6-4ff6-8fba-93b2113f24a4\") " pod="openshift-insights/insights-operator-5885fdb759-n9pwk"
Jul 27 19:40:18.076765 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.076745    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/096c978c-0cb6-4ff6-8fba-93b2113f24a4-service-ca-bundle\") pod \"insights-operator-5885fdb759-n9pwk\" (UID: \"096c978c-0cb6-4ff6-8fba-93b2113f24a4\") " pod="openshift-insights/insights-operator-5885fdb759-n9pwk"
Jul 27 19:40:18.076882 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.076834    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/096c978c-0cb6-4ff6-8fba-93b2113f24a4-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-n9pwk\" (UID: \"096c978c-0cb6-4ff6-8fba-93b2113f24a4\") " pod="openshift-insights/insights-operator-5885fdb759-n9pwk"
Jul 27 19:40:18.078417 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.078371    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/096c978c-0cb6-4ff6-8fba-93b2113f24a4-serving-cert\") pod \"insights-operator-5885fdb759-n9pwk\" (UID: \"096c978c-0cb6-4ff6-8fba-93b2113f24a4\") " pod="openshift-insights/insights-operator-5885fdb759-n9pwk"
Jul 27 19:40:18.078799 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.078736    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-default-certificate\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:18.080317 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.080276    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-stats-auth\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:18.084031 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.084013    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vfkj\" (UniqueName: \"kubernetes.io/projected/096c978c-0cb6-4ff6-8fba-93b2113f24a4-kube-api-access-6vfkj\") pod \"insights-operator-5885fdb759-n9pwk\" (UID: \"096c978c-0cb6-4ff6-8fba-93b2113f24a4\") " pod="openshift-insights/insights-operator-5885fdb759-n9pwk"
Jul 27 19:40:18.084724 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.084704    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9msp\" (UniqueName: \"kubernetes.io/projected/c6c35228-a7c8-44a3-9b48-63c27bb72b25-kube-api-access-j9msp\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:18.091116 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.090574    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx29v"
Jul 27 19:40:18.112473 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.112375    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tktlt_967ad4b2-9b0f-4654-be1b-8bcceee7f7c3/konnectivity-agent/0.log" limit=1024
Jul 27 19:40:18.147795 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.147762    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl"
Jul 27 19:40:18.170402 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.170370    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-n9pwk"
Jul 27 19:40:18.180940 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.180148    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bpst8"]
Jul 27 19:40:18.187883 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:40:18.185317    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc91d5e40_730e_4dfb_814a_f95776e4b5e9.slice/crio-5ccf0c1f8f7a90aa47ef37af6d04fb8a0c9c669c3de99becc7db5a7e0b02f8f3 WatchSource:0}: Error finding container 5ccf0c1f8f7a90aa47ef37af6d04fb8a0c9c669c3de99becc7db5a7e0b02f8f3: Status 404 returned error can't find the container with id 5ccf0c1f8f7a90aa47ef37af6d04fb8a0c9c669c3de99becc7db5a7e0b02f8f3
Jul 27 19:40:18.222278 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.222192    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-xfztm"]
Jul 27 19:40:18.229822 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:40:18.229786    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode814129b_3b96_4c75_ba5b_480e39d6d71e.slice/crio-31b18dc4932c0f61b465acb7f3f1875960011939384749325bf3b550517d5fb7 WatchSource:0}: Error finding container 31b18dc4932c0f61b465acb7f3f1875960011939384749325bf3b550517d5fb7: Status 404 returned error can't find the container with id 31b18dc4932c0f61b465acb7f3f1875960011939384749325bf3b550517d5fb7
Jul 27 19:40:18.292201 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.290030    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-5tg5r"]
Jul 27 19:40:18.293074 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:40:18.292986    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49b1c33d_0c79_4aa6_ba6a_d63ca54cea64.slice/crio-f5b42e1b1943b36face65501413630ca14fc4dd67080d852dc3ce5b0eeccc700 WatchSource:0}: Error finding container f5b42e1b1943b36face65501413630ca14fc4dd67080d852dc3ce5b0eeccc700: Status 404 returned error can't find the container with id f5b42e1b1943b36face65501413630ca14fc4dd67080d852dc3ce5b0eeccc700
Jul 27 19:40:18.303280 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.303240    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx29v"]
Jul 27 19:40:18.306379 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:40:18.306343    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd0782d2_c168_41d3_bc16_bf48f1453f20.slice/crio-e2130309fdc7e569cea9bda5bb7a1d14361592764e54a757ac1489d147d6fb14 WatchSource:0}: Error finding container e2130309fdc7e569cea9bda5bb7a1d14361592764e54a757ac1489d147d6fb14: Status 404 returned error can't find the container with id e2130309fdc7e569cea9bda5bb7a1d14361592764e54a757ac1489d147d6fb14
Jul 27 19:40:18.335942 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.335848    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl"]
Jul 27 19:40:18.338557 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:40:18.338534    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1f4f700_7ce1_4669_a383_793dfbcf4f2e.slice/crio-14e8c8c6b04ebefd4a90c7c3ff3cf7c714e171ccc923d4c7e7991f427168a25d WatchSource:0}: Error finding container 14e8c8c6b04ebefd4a90c7c3ff3cf7c714e171ccc923d4c7e7991f427168a25d: Status 404 returned error can't find the container with id 14e8c8c6b04ebefd4a90c7c3ff3cf7c714e171ccc923d4c7e7991f427168a25d
Jul 27 19:40:18.357565 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.357542    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-n9pwk"]
Jul 27 19:40:18.360596 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:40:18.360570    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod096c978c_0cb6_4ff6_8fba_93b2113f24a4.slice/crio-7a4c8cadb15718d2cb0aac4ea493372aaa96c18ef328a32c1fc54313de64dd31 WatchSource:0}: Error finding container 7a4c8cadb15718d2cb0aac4ea493372aaa96c18ef328a32c1fc54313de64dd31: Status 404 returned error can't find the container with id 7a4c8cadb15718d2cb0aac4ea493372aaa96c18ef328a32c1fc54313de64dd31
Jul 27 19:40:18.378958 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.378933    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8298c2f2-f22a-41cf-8202-ba1c367add65-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-6tml7\" (UID: \"8298c2f2-f22a-41cf-8202-ba1c367add65\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6tml7"
Jul 27 19:40:18.379083 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:18.379068    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 19:40:18.379135 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:18.379126    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8298c2f2-f22a-41cf-8202-ba1c367add65-samples-operator-tls podName:8298c2f2-f22a-41cf-8202-ba1c367add65 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:19.379108126 +0000 UTC m=+47.608350305 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8298c2f2-f22a-41cf-8202-ba1c367add65-samples-operator-tls") pod "cluster-samples-operator-6748489656-6tml7" (UID: "8298c2f2-f22a-41cf-8202-ba1c367add65") : secret "samples-operator-tls" not found
Jul 27 19:40:18.480139 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.480082    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0a45576e-6f99-4ff2-bdbe-934a603e6756-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kx5wp\" (UID: \"0a45576e-6f99-4ff2-bdbe-934a603e6756\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp"
Jul 27 19:40:18.480239 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:18.480223    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 19:40:18.480286 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:18.480277    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a45576e-6f99-4ff2-bdbe-934a603e6756-cluster-monitoring-operator-tls podName:0a45576e-6f99-4ff2-bdbe-934a603e6756 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:19.480262838 +0000 UTC m=+47.709505036 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0a45576e-6f99-4ff2-bdbe-934a603e6756-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kx5wp" (UID: "0a45576e-6f99-4ff2-bdbe-934a603e6756") : secret "cluster-monitoring-operator-tls" not found
Jul 27 19:40:18.580896 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.580873    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-metrics-certs\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:18.581005 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.580993    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6c35228-a7c8-44a3-9b48-63c27bb72b25-service-ca-bundle\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:18.581051 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:18.581002    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 19:40:18.581051 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:18.581048    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-metrics-certs podName:c6c35228-a7c8-44a3-9b48-63c27bb72b25 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:19.581033615 +0000 UTC m=+47.810275791 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-metrics-certs") pod "router-default-5f5b9f5469-b26r4" (UID: "c6c35228-a7c8-44a3-9b48-63c27bb72b25") : secret "router-metrics-certs-default" not found
Jul 27 19:40:18.581133 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:18.581078    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c6c35228-a7c8-44a3-9b48-63c27bb72b25-service-ca-bundle podName:c6c35228-a7c8-44a3-9b48-63c27bb72b25 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:19.58106737 +0000 UTC m=+47.810309569 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c6c35228-a7c8-44a3-9b48-63c27bb72b25-service-ca-bundle") pod "router-default-5f5b9f5469-b26r4" (UID: "c6c35228-a7c8-44a3-9b48-63c27bb72b25") : configmap references non-existent config key: service-ca.crt
Jul 27 19:40:18.618982 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.618954    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl" event={"ID":"b1f4f700-7ce1-4669-a383-793dfbcf4f2e","Type":"ContainerStarted","Data":"14e8c8c6b04ebefd4a90c7c3ff3cf7c714e171ccc923d4c7e7991f427168a25d"}
Jul 27 19:40:18.620164 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.620139    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bpst8" event={"ID":"c91d5e40-730e-4dfb-814a-f95776e4b5e9","Type":"ContainerStarted","Data":"5ccf0c1f8f7a90aa47ef37af6d04fb8a0c9c669c3de99becc7db5a7e0b02f8f3"}
Jul 27 19:40:18.621230 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.621209    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-xfztm" event={"ID":"e814129b-3b96-4c75-ba5b-480e39d6d71e","Type":"ContainerStarted","Data":"31b18dc4932c0f61b465acb7f3f1875960011939384749325bf3b550517d5fb7"}
Jul 27 19:40:18.622310 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.622291    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-n9pwk" event={"ID":"096c978c-0cb6-4ff6-8fba-93b2113f24a4","Type":"ContainerStarted","Data":"7a4c8cadb15718d2cb0aac4ea493372aaa96c18ef328a32c1fc54313de64dd31"}
Jul 27 19:40:18.623296 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.623277    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx29v" event={"ID":"bd0782d2-c168-41d3-bc16-bf48f1453f20","Type":"ContainerStarted","Data":"e2130309fdc7e569cea9bda5bb7a1d14361592764e54a757ac1489d147d6fb14"}
Jul 27 19:40:18.624245 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:18.624228    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-5tg5r" event={"ID":"49b1c33d-0c79-4aa6-ba6a-d63ca54cea64","Type":"ContainerStarted","Data":"f5b42e1b1943b36face65501413630ca14fc4dd67080d852dc3ce5b0eeccc700"}
Jul 27 19:40:19.388273 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:19.388217    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8298c2f2-f22a-41cf-8202-ba1c367add65-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-6tml7\" (UID: \"8298c2f2-f22a-41cf-8202-ba1c367add65\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6tml7"
Jul 27 19:40:19.389009 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:19.388527    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 19:40:19.389009 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:19.388610    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8298c2f2-f22a-41cf-8202-ba1c367add65-samples-operator-tls podName:8298c2f2-f22a-41cf-8202-ba1c367add65 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:21.388587584 +0000 UTC m=+49.617829777 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8298c2f2-f22a-41cf-8202-ba1c367add65-samples-operator-tls") pod "cluster-samples-operator-6748489656-6tml7" (UID: "8298c2f2-f22a-41cf-8202-ba1c367add65") : secret "samples-operator-tls" not found
Jul 27 19:40:19.489671 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:19.489628    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0a45576e-6f99-4ff2-bdbe-934a603e6756-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kx5wp\" (UID: \"0a45576e-6f99-4ff2-bdbe-934a603e6756\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp"
Jul 27 19:40:19.489879 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:19.489835    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 19:40:19.489973 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:19.489926    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a45576e-6f99-4ff2-bdbe-934a603e6756-cluster-monitoring-operator-tls podName:0a45576e-6f99-4ff2-bdbe-934a603e6756 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:21.489905903 +0000 UTC m=+49.719148080 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0a45576e-6f99-4ff2-bdbe-934a603e6756-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kx5wp" (UID: "0a45576e-6f99-4ff2-bdbe-934a603e6756") : secret "cluster-monitoring-operator-tls" not found
Jul 27 19:40:19.591013 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:19.590979    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6c35228-a7c8-44a3-9b48-63c27bb72b25-service-ca-bundle\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:19.591163 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:19.591026    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-metrics-certs\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:19.591226 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:19.591188    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 19:40:19.591271 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:19.591252    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-metrics-certs podName:c6c35228-a7c8-44a3-9b48-63c27bb72b25 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:21.591232205 +0000 UTC m=+49.820474382 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-metrics-certs") pod "router-default-5f5b9f5469-b26r4" (UID: "c6c35228-a7c8-44a3-9b48-63c27bb72b25") : secret "router-metrics-certs-default" not found
Jul 27 19:40:19.591661 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:19.591642    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c6c35228-a7c8-44a3-9b48-63c27bb72b25-service-ca-bundle podName:c6c35228-a7c8-44a3-9b48-63c27bb72b25 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:21.591629967 +0000 UTC m=+49.820872142 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c6c35228-a7c8-44a3-9b48-63c27bb72b25-service-ca-bundle") pod "router-default-5f5b9f5469-b26r4" (UID: "c6c35228-a7c8-44a3-9b48-63c27bb72b25") : configmap references non-existent config key: service-ca.crt
Jul 27 19:40:21.407687 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:21.407600    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8298c2f2-f22a-41cf-8202-ba1c367add65-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-6tml7\" (UID: \"8298c2f2-f22a-41cf-8202-ba1c367add65\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6tml7"
Jul 27 19:40:21.408139 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:21.407774    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 19:40:21.408139 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:21.407877    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8298c2f2-f22a-41cf-8202-ba1c367add65-samples-operator-tls podName:8298c2f2-f22a-41cf-8202-ba1c367add65 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:25.407839775 +0000 UTC m=+53.637081970 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8298c2f2-f22a-41cf-8202-ba1c367add65-samples-operator-tls") pod "cluster-samples-operator-6748489656-6tml7" (UID: "8298c2f2-f22a-41cf-8202-ba1c367add65") : secret "samples-operator-tls" not found
Jul 27 19:40:21.508457 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:21.508417    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0a45576e-6f99-4ff2-bdbe-934a603e6756-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kx5wp\" (UID: \"0a45576e-6f99-4ff2-bdbe-934a603e6756\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp"
Jul 27 19:40:21.508623 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:21.508597    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 19:40:21.508699 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:21.508687    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a45576e-6f99-4ff2-bdbe-934a603e6756-cluster-monitoring-operator-tls podName:0a45576e-6f99-4ff2-bdbe-934a603e6756 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:25.508665874 +0000 UTC m=+53.737908075 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0a45576e-6f99-4ff2-bdbe-934a603e6756-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kx5wp" (UID: "0a45576e-6f99-4ff2-bdbe-934a603e6756") : secret "cluster-monitoring-operator-tls" not found
Jul 27 19:40:21.609368 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:21.609334    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6c35228-a7c8-44a3-9b48-63c27bb72b25-service-ca-bundle\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:21.609517 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:21.609395    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-metrics-certs\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:21.609556 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:21.609519    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c6c35228-a7c8-44a3-9b48-63c27bb72b25-service-ca-bundle podName:c6c35228-a7c8-44a3-9b48-63c27bb72b25 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:25.609500644 +0000 UTC m=+53.838742825 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c6c35228-a7c8-44a3-9b48-63c27bb72b25-service-ca-bundle") pod "router-default-5f5b9f5469-b26r4" (UID: "c6c35228-a7c8-44a3-9b48-63c27bb72b25") : configmap references non-existent config key: service-ca.crt
Jul 27 19:40:21.609556 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:21.609520    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 19:40:21.609623 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:21.609561    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-metrics-certs podName:c6c35228-a7c8-44a3-9b48-63c27bb72b25 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:25.609551177 +0000 UTC m=+53.838793353 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-metrics-certs") pod "router-default-5f5b9f5469-b26r4" (UID: "c6c35228-a7c8-44a3-9b48-63c27bb72b25") : secret "router-metrics-certs-default" not found
Jul 27 19:40:22.920254 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:22.920213    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:22.920254 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:22.920256    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66b96198-308c-49c0-bb97-bb8210624656-metrics-tls\") pod \"dns-default-mcmp4\" (UID: \"66b96198-308c-49c0-bb97-bb8210624656\") " pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:22.920745 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:22.920401    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 19:40:22.920745 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:22.920404    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 19:40:22.920745 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:22.920422    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66df67d8d5-nx4mf: secret "image-registry-tls" not found
Jul 27 19:40:22.920745 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:22.920474    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66b96198-308c-49c0-bb97-bb8210624656-metrics-tls podName:66b96198-308c-49c0-bb97-bb8210624656 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:38.920456457 +0000 UTC m=+67.149698654 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66b96198-308c-49c0-bb97-bb8210624656-metrics-tls") pod "dns-default-mcmp4" (UID: "66b96198-308c-49c0-bb97-bb8210624656") : secret "dns-default-metrics-tls" not found
Jul 27 19:40:22.920745 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:22.920489    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls podName:2343471b-c979-4422-a9d6-f30610099419 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:38.920483009 +0000 UTC m=+67.149725186 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls") pod "image-registry-66df67d8d5-nx4mf" (UID: "2343471b-c979-4422-a9d6-f30610099419") : secret "image-registry-tls" not found
Jul 27 19:40:25.446135 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:25.446096    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8298c2f2-f22a-41cf-8202-ba1c367add65-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-6tml7\" (UID: \"8298c2f2-f22a-41cf-8202-ba1c367add65\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6tml7"
Jul 27 19:40:25.446674 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:25.446311    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 19:40:25.446674 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:25.446375    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8298c2f2-f22a-41cf-8202-ba1c367add65-samples-operator-tls podName:8298c2f2-f22a-41cf-8202-ba1c367add65 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:33.446355449 +0000 UTC m=+61.675597638 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8298c2f2-f22a-41cf-8202-ba1c367add65-samples-operator-tls") pod "cluster-samples-operator-6748489656-6tml7" (UID: "8298c2f2-f22a-41cf-8202-ba1c367add65") : secret "samples-operator-tls" not found
Jul 27 19:40:25.547307 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:25.547261    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0a45576e-6f99-4ff2-bdbe-934a603e6756-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kx5wp\" (UID: \"0a45576e-6f99-4ff2-bdbe-934a603e6756\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp"
Jul 27 19:40:25.547507 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:25.547443    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 19:40:25.547566 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:25.547523    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a45576e-6f99-4ff2-bdbe-934a603e6756-cluster-monitoring-operator-tls podName:0a45576e-6f99-4ff2-bdbe-934a603e6756 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:33.547500098 +0000 UTC m=+61.776742280 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0a45576e-6f99-4ff2-bdbe-934a603e6756-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kx5wp" (UID: "0a45576e-6f99-4ff2-bdbe-934a603e6756") : secret "cluster-monitoring-operator-tls" not found
Jul 27 19:40:25.648332 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:25.648289    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-metrics-certs\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:25.648531 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:25.648473    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 19:40:25.648531 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:25.648488    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6c35228-a7c8-44a3-9b48-63c27bb72b25-service-ca-bundle\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:25.648639 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:25.648558    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-metrics-certs podName:c6c35228-a7c8-44a3-9b48-63c27bb72b25 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:33.648530495 +0000 UTC m=+61.877772694 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-metrics-certs") pod "router-default-5f5b9f5469-b26r4" (UID: "c6c35228-a7c8-44a3-9b48-63c27bb72b25") : secret "router-metrics-certs-default" not found
Jul 27 19:40:25.648639 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:25.648619    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c6c35228-a7c8-44a3-9b48-63c27bb72b25-service-ca-bundle podName:c6c35228-a7c8-44a3-9b48-63c27bb72b25 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:33.648603429 +0000 UTC m=+61.877845624 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c6c35228-a7c8-44a3-9b48-63c27bb72b25-service-ca-bundle") pod "router-default-5f5b9f5469-b26r4" (UID: "c6c35228-a7c8-44a3-9b48-63c27bb72b25") : configmap references non-existent config key: service-ca.crt
Jul 27 19:40:25.660635 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:25.660600    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx29v" event={"ID":"bd0782d2-c168-41d3-bc16-bf48f1453f20","Type":"ContainerStarted","Data":"a679b4cc915638b9d68b71996b8f809790fb6a2388d1eaed86583b481d5b54a9"}
Jul 27 19:40:25.662559 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:25.662530    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-5tg5r" event={"ID":"49b1c33d-0c79-4aa6-ba6a-d63ca54cea64","Type":"ContainerStarted","Data":"e6fc130881cbc8ab31b4e340bb8aaad059544d15dc0ad40777dabcac0e87bda7"}
Jul 27 19:40:25.664493 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:25.664471    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl" event={"ID":"b1f4f700-7ce1-4669-a383-793dfbcf4f2e","Type":"ContainerStarted","Data":"d5dcc9cce099535b6a67b9e8a42466999272c4e35ae8249ab3048deffd510dcf"}
Jul 27 19:40:25.666072 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:25.666050    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bpst8" event={"ID":"c91d5e40-730e-4dfb-814a-f95776e4b5e9","Type":"ContainerStarted","Data":"4734b605cfb8fde9e1c68166d119cf20dedc9e85fe3c3b90937345f3474824e7"}
Jul 27 19:40:25.667623 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:25.667592    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/0.log"
Jul 27 19:40:25.667719 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:25.667635    2579 generic.go:356] "Generic (PLEG): container finished" podID="e814129b-3b96-4c75-ba5b-480e39d6d71e" containerID="d14a777262b6ab049ce53a4c6f8064f993a82a4f873a68ef59a876320bae5bd6" exitCode=255
Jul 27 19:40:25.667829 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:25.667808    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-xfztm" event={"ID":"e814129b-3b96-4c75-ba5b-480e39d6d71e","Type":"ContainerDied","Data":"d14a777262b6ab049ce53a4c6f8064f993a82a4f873a68ef59a876320bae5bd6"}
Jul 27 19:40:25.667952 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:25.667930    2579 scope.go:117] "RemoveContainer" containerID="d14a777262b6ab049ce53a4c6f8064f993a82a4f873a68ef59a876320bae5bd6"
Jul 27 19:40:25.669311 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:25.669291    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-n9pwk" event={"ID":"096c978c-0cb6-4ff6-8fba-93b2113f24a4","Type":"ContainerStarted","Data":"e54f030e7766bfaa3edd114711d8677c5ee9b9c6b38816b6e70c9adef8e4fb27"}
Jul 27 19:40:25.692749 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:25.692703    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hx29v" podStartSLOduration=2.8314024939999998 podStartE2EDuration="8.692687022s" totalImagesPullingTime="5.861284528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:40:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:40:25.675296539 +0000 UTC m=+53.904538738" watchObservedRunningTime="2026-07-27 19:40:25.692687022 +0000 UTC m=+53.921929242"
Jul 27 19:40:25.712232 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:25.712176    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-5tg5r" podStartSLOduration=2.360798993 podStartE2EDuration="8.712157445s" totalImagesPullingTime="6.351358452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:40:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:40:25.711118631 +0000 UTC m=+53.940360828" watchObservedRunningTime="2026-07-27 19:40:25.712157445 +0000 UTC m=+53.941399645"
Jul 27 19:40:25.751831 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:25.751764    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-n9pwk" podStartSLOduration=2.461218864 podStartE2EDuration="8.751743646s" totalImagesPullingTime="6.290524782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:40:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:40:25.74895247 +0000 UTC m=+53.978194672" watchObservedRunningTime="2026-07-27 19:40:25.751743646 +0000 UTC m=+53.980985837"
Jul 27 19:40:25.753784 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:25.753746    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl" podStartSLOduration=2.448819575 podStartE2EDuration="8.753728935s" totalImagesPullingTime="6.30490936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:40:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:40:25.729324827 +0000 UTC m=+53.958567026" watchObservedRunningTime="2026-07-27 19:40:25.753728935 +0000 UTC m=+53.982971138"
Jul 27 19:40:25.766218 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:25.766169    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bpst8" podStartSLOduration=2.306869449 podStartE2EDuration="8.766151375s" totalImagesPullingTime="6.459281926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:40:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:40:25.764634611 +0000 UTC m=+53.993876811" watchObservedRunningTime="2026-07-27 19:40:25.766151375 +0000 UTC m=+53.995393580"
Jul 27 19:40:26.567019 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.566986    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lppjn"]
Jul 27 19:40:26.570008 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.569981    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:26.571787 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.571763    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 27 19:40:26.572129 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.572107    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 27 19:40:26.572232 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.572215    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-csj86\""
Jul 27 19:40:26.582469 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.582445    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lppjn"]
Jul 27 19:40:26.656294 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.656255    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/722f2327-d04b-4470-aea1-be7c383a497a-data-volume\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:26.656294 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.656294    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/722f2327-d04b-4470-aea1-be7c383a497a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:26.656470 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.656313    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xflj\" (UniqueName: \"kubernetes.io/projected/722f2327-d04b-4470-aea1-be7c383a497a-kube-api-access-6xflj\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:26.656508 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.656461    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/722f2327-d04b-4470-aea1-be7c383a497a-crio-socket\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:26.656555 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.656509    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/722f2327-d04b-4470-aea1-be7c383a497a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:26.675956 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.675932    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 19:40:26.676351 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.676335    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/0.log"
Jul 27 19:40:26.676428 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.676369    2579 generic.go:356] "Generic (PLEG): container finished" podID="e814129b-3b96-4c75-ba5b-480e39d6d71e" containerID="0a198de49cbf5e9a418830e0e1d6a7f8209740ee4c1fc35b74f66daa8ef88e06" exitCode=255
Jul 27 19:40:26.676496 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.676451    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-xfztm" event={"ID":"e814129b-3b96-4c75-ba5b-480e39d6d71e","Type":"ContainerDied","Data":"0a198de49cbf5e9a418830e0e1d6a7f8209740ee4c1fc35b74f66daa8ef88e06"}
Jul 27 19:40:26.676538 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.676500    2579 scope.go:117] "RemoveContainer" containerID="d14a777262b6ab049ce53a4c6f8064f993a82a4f873a68ef59a876320bae5bd6"
Jul 27 19:40:26.676749 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.676730    2579 scope.go:117] "RemoveContainer" containerID="0a198de49cbf5e9a418830e0e1d6a7f8209740ee4c1fc35b74f66daa8ef88e06"
Jul 27 19:40:26.676976 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:26.676951    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-xfztm_openshift-console-operator(e814129b-3b96-4c75-ba5b-480e39d6d71e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-xfztm" podUID="e814129b-3b96-4c75-ba5b-480e39d6d71e"
Jul 27 19:40:26.757263 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.757225    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/722f2327-d04b-4470-aea1-be7c383a497a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:26.757463 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.757437    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/722f2327-d04b-4470-aea1-be7c383a497a-data-volume\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:26.757552 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:26.757533    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 19:40:26.757631 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.757606    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/722f2327-d04b-4470-aea1-be7c383a497a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:26.757707 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.757657    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xflj\" (UniqueName: \"kubernetes.io/projected/722f2327-d04b-4470-aea1-be7c383a497a-kube-api-access-6xflj\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:26.757707 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:26.757671    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 19:40:26.757707 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:26.757697    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/722f2327-d04b-4470-aea1-be7c383a497a-insights-runtime-extractor-tls podName:722f2327-d04b-4470-aea1-be7c383a497a nodeName:}" failed. No retries permitted until 2026-07-27 19:40:27.257675007 +0000 UTC m=+55.486917198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/722f2327-d04b-4470-aea1-be7c383a497a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lppjn" (UID: "722f2327-d04b-4470-aea1-be7c383a497a") : secret "insights-runtime-extractor-tls" not found
Jul 27 19:40:26.757878 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.757752    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/722f2327-d04b-4470-aea1-be7c383a497a-data-volume\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:26.757878 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:26.757762    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/722f2327-d04b-4470-aea1-be7c383a497a-kube-rbac-proxy-cm podName:722f2327-d04b-4470-aea1-be7c383a497a nodeName:}" failed. No retries permitted until 2026-07-27 19:40:27.25773975 +0000 UTC m=+55.486981977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/722f2327-d04b-4470-aea1-be7c383a497a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lppjn" (UID: "722f2327-d04b-4470-aea1-be7c383a497a") : configmap "kube-rbac-proxy" not found
Jul 27 19:40:26.758050 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.758030    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/722f2327-d04b-4470-aea1-be7c383a497a-crio-socket\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:26.758201 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.758176    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/722f2327-d04b-4470-aea1-be7c383a497a-crio-socket\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:26.770278 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:26.770252    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xflj\" (UniqueName: \"kubernetes.io/projected/722f2327-d04b-4470-aea1-be7c383a497a-kube-api-access-6xflj\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:27.261692 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:27.261657    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/722f2327-d04b-4470-aea1-be7c383a497a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:27.261888 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:27.261733    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/722f2327-d04b-4470-aea1-be7c383a497a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:27.261888 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:27.261805    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 19:40:27.261888 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:27.261812    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 19:40:27.261999 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:27.261901    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/722f2327-d04b-4470-aea1-be7c383a497a-insights-runtime-extractor-tls podName:722f2327-d04b-4470-aea1-be7c383a497a nodeName:}" failed. No retries permitted until 2026-07-27 19:40:28.261883003 +0000 UTC m=+56.491125194 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/722f2327-d04b-4470-aea1-be7c383a497a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lppjn" (UID: "722f2327-d04b-4470-aea1-be7c383a497a") : secret "insights-runtime-extractor-tls" not found
Jul 27 19:40:27.261999 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:27.261919    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/722f2327-d04b-4470-aea1-be7c383a497a-kube-rbac-proxy-cm podName:722f2327-d04b-4470-aea1-be7c383a497a nodeName:}" failed. No retries permitted until 2026-07-27 19:40:28.261909545 +0000 UTC m=+56.491151722 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/722f2327-d04b-4470-aea1-be7c383a497a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lppjn" (UID: "722f2327-d04b-4470-aea1-be7c383a497a") : configmap "kube-rbac-proxy" not found
Jul 27 19:40:27.687208 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:27.687182    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 19:40:27.687617 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:27.687485    2579 scope.go:117] "RemoveContainer" containerID="0a198de49cbf5e9a418830e0e1d6a7f8209740ee4c1fc35b74f66daa8ef88e06"
Jul 27 19:40:27.687656 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:27.687628    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-xfztm_openshift-console-operator(e814129b-3b96-4c75-ba5b-480e39d6d71e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-xfztm" podUID="e814129b-3b96-4c75-ba5b-480e39d6d71e"
Jul 27 19:40:28.026723 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:28.026648    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-xfztm"
Jul 27 19:40:28.026949 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:28.026729    2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-xfztm"
Jul 27 19:40:28.271047 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:28.271014    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/722f2327-d04b-4470-aea1-be7c383a497a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:28.271201 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:28.271077    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/722f2327-d04b-4470-aea1-be7c383a497a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:28.271201 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:28.271170    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 19:40:28.271201 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:28.271192    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 19:40:28.271300 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:28.271244    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/722f2327-d04b-4470-aea1-be7c383a497a-insights-runtime-extractor-tls podName:722f2327-d04b-4470-aea1-be7c383a497a nodeName:}" failed. No retries permitted until 2026-07-27 19:40:30.271224209 +0000 UTC m=+58.500466399 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/722f2327-d04b-4470-aea1-be7c383a497a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lppjn" (UID: "722f2327-d04b-4470-aea1-be7c383a497a") : secret "insights-runtime-extractor-tls" not found
Jul 27 19:40:28.271300 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:28.271260    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/722f2327-d04b-4470-aea1-be7c383a497a-kube-rbac-proxy-cm podName:722f2327-d04b-4470-aea1-be7c383a497a nodeName:}" failed. No retries permitted until 2026-07-27 19:40:30.271252675 +0000 UTC m=+58.500494850 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/722f2327-d04b-4470-aea1-be7c383a497a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lppjn" (UID: "722f2327-d04b-4470-aea1-be7c383a497a") : configmap "kube-rbac-proxy" not found
Jul 27 19:40:28.690712 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:28.690686    2579 scope.go:117] "RemoveContainer" containerID="0a198de49cbf5e9a418830e0e1d6a7f8209740ee4c1fc35b74f66daa8ef88e06"
Jul 27 19:40:28.691117 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:28.690830    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-xfztm_openshift-console-operator(e814129b-3b96-4c75-ba5b-480e39d6d71e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-xfztm" podUID="e814129b-3b96-4c75-ba5b-480e39d6d71e"
Jul 27 19:40:29.172661 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:29.172638    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-grpbk_73b8094b-b342-4acd-9323-a5e71e0c3dfb/dns-node-resolver/0.log"
Jul 27 19:40:29.971771 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:29.971746    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m2pzk_91f1914a-7ad4-45a2-949f-75375e34df1f/node-ca/0.log"
Jul 27 19:40:30.287404 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:30.287328    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/722f2327-d04b-4470-aea1-be7c383a497a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:30.287404 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:30.287387    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/722f2327-d04b-4470-aea1-be7c383a497a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:30.287580 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:30.287456    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 19:40:30.287580 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:30.287465    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 19:40:30.287580 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:30.287517    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/722f2327-d04b-4470-aea1-be7c383a497a-kube-rbac-proxy-cm podName:722f2327-d04b-4470-aea1-be7c383a497a nodeName:}" failed. No retries permitted until 2026-07-27 19:40:34.287500708 +0000 UTC m=+62.516742905 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/722f2327-d04b-4470-aea1-be7c383a497a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lppjn" (UID: "722f2327-d04b-4470-aea1-be7c383a497a") : configmap "kube-rbac-proxy" not found
Jul 27 19:40:30.287580 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:30.287530    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/722f2327-d04b-4470-aea1-be7c383a497a-insights-runtime-extractor-tls podName:722f2327-d04b-4470-aea1-be7c383a497a nodeName:}" failed. No retries permitted until 2026-07-27 19:40:34.287524401 +0000 UTC m=+62.516766577 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/722f2327-d04b-4470-aea1-be7c383a497a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lppjn" (UID: "722f2327-d04b-4470-aea1-be7c383a497a") : secret "insights-runtime-extractor-tls" not found
Jul 27 19:40:30.771238 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:30.771207    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tct2c_225f0ebd-cdb1-44c5-b705-19e759e4dd61/serve-healthcheck-canary/0.log"
Jul 27 19:40:31.372450 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:31.372416    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl_b1f4f700-7ce1-4669-a383-793dfbcf4f2e/kube-storage-version-migrator-operator/0.log"
Jul 27 19:40:32.549153 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:32.549123    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nwhgp"
Jul 27 19:40:33.516623 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:33.516592    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8298c2f2-f22a-41cf-8202-ba1c367add65-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-6tml7\" (UID: \"8298c2f2-f22a-41cf-8202-ba1c367add65\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6tml7"
Jul 27 19:40:33.516909 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:33.516738    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 19:40:33.516909 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:33.516811    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8298c2f2-f22a-41cf-8202-ba1c367add65-samples-operator-tls podName:8298c2f2-f22a-41cf-8202-ba1c367add65 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:49.516795542 +0000 UTC m=+77.746037722 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8298c2f2-f22a-41cf-8202-ba1c367add65-samples-operator-tls") pod "cluster-samples-operator-6748489656-6tml7" (UID: "8298c2f2-f22a-41cf-8202-ba1c367add65") : secret "samples-operator-tls" not found
Jul 27 19:40:33.617091 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:33.617056    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0a45576e-6f99-4ff2-bdbe-934a603e6756-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kx5wp\" (UID: \"0a45576e-6f99-4ff2-bdbe-934a603e6756\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp"
Jul 27 19:40:33.617453 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:33.617204    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 19:40:33.617453 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:33.617267    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a45576e-6f99-4ff2-bdbe-934a603e6756-cluster-monitoring-operator-tls podName:0a45576e-6f99-4ff2-bdbe-934a603e6756 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:49.617252414 +0000 UTC m=+77.846494604 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0a45576e-6f99-4ff2-bdbe-934a603e6756-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kx5wp" (UID: "0a45576e-6f99-4ff2-bdbe-934a603e6756") : secret "cluster-monitoring-operator-tls" not found
Jul 27 19:40:33.718307 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:33.718278    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-metrics-certs\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:33.718452 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:33.718404    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6c35228-a7c8-44a3-9b48-63c27bb72b25-service-ca-bundle\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:33.718452 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:33.718413    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 19:40:33.718537 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:33.718471    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-metrics-certs podName:c6c35228-a7c8-44a3-9b48-63c27bb72b25 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:49.718451291 +0000 UTC m=+77.947693486 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-metrics-certs") pod "router-default-5f5b9f5469-b26r4" (UID: "c6c35228-a7c8-44a3-9b48-63c27bb72b25") : secret "router-metrics-certs-default" not found
Jul 27 19:40:33.718537 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:33.718492    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c6c35228-a7c8-44a3-9b48-63c27bb72b25-service-ca-bundle podName:c6c35228-a7c8-44a3-9b48-63c27bb72b25 nodeName:}" failed. No retries permitted until 2026-07-27 19:40:49.718480384 +0000 UTC m=+77.947722585 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c6c35228-a7c8-44a3-9b48-63c27bb72b25-service-ca-bundle") pod "router-default-5f5b9f5469-b26r4" (UID: "c6c35228-a7c8-44a3-9b48-63c27bb72b25") : configmap references non-existent config key: service-ca.crt
Jul 27 19:40:34.324135 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:34.324100    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/722f2327-d04b-4470-aea1-be7c383a497a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:34.324334 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:34.324161    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/722f2327-d04b-4470-aea1-be7c383a497a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:34.324334 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:34.324229    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 19:40:34.324334 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:34.324277    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/722f2327-d04b-4470-aea1-be7c383a497a-kube-rbac-proxy-cm podName:722f2327-d04b-4470-aea1-be7c383a497a nodeName:}" failed. No retries permitted until 2026-07-27 19:40:42.324264375 +0000 UTC m=+70.553506551 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/722f2327-d04b-4470-aea1-be7c383a497a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lppjn" (UID: "722f2327-d04b-4470-aea1-be7c383a497a") : configmap "kube-rbac-proxy" not found
Jul 27 19:40:34.324334 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:34.324278    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 19:40:34.324499 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:34.324338    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/722f2327-d04b-4470-aea1-be7c383a497a-insights-runtime-extractor-tls podName:722f2327-d04b-4470-aea1-be7c383a497a nodeName:}" failed. No retries permitted until 2026-07-27 19:40:42.324320962 +0000 UTC m=+70.553563140 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/722f2327-d04b-4470-aea1-be7c383a497a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lppjn" (UID: "722f2327-d04b-4470-aea1-be7c383a497a") : secret "insights-runtime-extractor-tls" not found
Jul 27 19:40:38.058127 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.058087    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs\") pod \"network-metrics-daemon-sdk4x\" (UID: \"74f89d7b-f93b-4dc3-9e18-41cf466d90bd\") " pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:40:38.058592 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.058142    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret\") pod \"global-pull-secret-syncer-7gt55\" (UID: \"6407e54e-d37a-4344-91d3-4494c81246c4\") " pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:40:38.060410 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.060387    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 19:40:38.060553 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.060446    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 19:40:38.069075 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:38.069054    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 27 19:40:38.069137 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:38.069120    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs podName:74f89d7b-f93b-4dc3-9e18-41cf466d90bd nodeName:}" failed. No retries permitted until 2026-07-27 19:41:42.069104469 +0000 UTC m=+130.298346658 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs") pod "network-metrics-daemon-sdk4x" (UID: "74f89d7b-f93b-4dc3-9e18-41cf466d90bd") : secret "metrics-daemon-secret" not found
Jul 27 19:40:38.071275 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.071256    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6407e54e-d37a-4344-91d3-4494c81246c4-original-pull-secret\") pod \"global-pull-secret-syncer-7gt55\" (UID: \"6407e54e-d37a-4344-91d3-4494c81246c4\") " pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:40:38.175361 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.175327    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gt55"
Jul 27 19:40:38.259874 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.259830    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8ck5\" (UniqueName: \"kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5\") pod \"network-check-target-fpbzl\" (UID: \"f9f7d637-863f-4a7f-8327-ce46c5076d4d\") " pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:40:38.262491 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.262461    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8ck5\" (UniqueName: \"kubernetes.io/projected/f9f7d637-863f-4a7f-8327-ce46c5076d4d-kube-api-access-p8ck5\") pod \"network-check-target-fpbzl\" (UID: \"f9f7d637-863f-4a7f-8327-ce46c5076d4d\") " pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:40:38.322269 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.322246    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7gt55"]
Jul 27 19:40:38.324573 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:40:38.324547    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6407e54e_d37a_4344_91d3_4494c81246c4.slice/crio-47d2bbe95be88397d34c40039769b32832929b72c19cb4cb9386e741d09fa3bf WatchSource:0}: Error finding container 47d2bbe95be88397d34c40039769b32832929b72c19cb4cb9386e741d09fa3bf: Status 404 returned error can't find the container with id 47d2bbe95be88397d34c40039769b32832929b72c19cb4cb9386e741d09fa3bf
Jul 27 19:40:38.485878 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.485835    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-klph9\""
Jul 27 19:40:38.493316 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.493293    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:40:38.616006 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.615932    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fpbzl"]
Jul 27 19:40:38.618940 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:40:38.618915    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9f7d637_863f_4a7f_8327_ce46c5076d4d.slice/crio-92ee14c71506c4ef3b31d0ac6c1f87d213dddb6c5976fe028a87fe6be57afdf7 WatchSource:0}: Error finding container 92ee14c71506c4ef3b31d0ac6c1f87d213dddb6c5976fe028a87fe6be57afdf7: Status 404 returned error can't find the container with id 92ee14c71506c4ef3b31d0ac6c1f87d213dddb6c5976fe028a87fe6be57afdf7
Jul 27 19:40:38.724479 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.724451    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fpbzl" event={"ID":"f9f7d637-863f-4a7f-8327-ce46c5076d4d","Type":"ContainerStarted","Data":"7aff9767eb0f9feea06b7025e240f330baa1812dd5527ff13d00989d806e3048"}
Jul 27 19:40:38.724596 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.724502    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fpbzl" event={"ID":"f9f7d637-863f-4a7f-8327-ce46c5076d4d","Type":"ContainerStarted","Data":"92ee14c71506c4ef3b31d0ac6c1f87d213dddb6c5976fe028a87fe6be57afdf7"}
Jul 27 19:40:38.724596 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.724567    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:40:38.725673 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.725653    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7gt55" event={"ID":"6407e54e-d37a-4344-91d3-4494c81246c4","Type":"ContainerStarted","Data":"47d2bbe95be88397d34c40039769b32832929b72c19cb4cb9386e741d09fa3bf"}
Jul 27 19:40:38.743706 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.743669    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fpbzl" podStartSLOduration=65.743656634 podStartE2EDuration="1m5.743656634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:39:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:40:38.743121794 +0000 UTC m=+66.972363992" watchObservedRunningTime="2026-07-27 19:40:38.743656634 +0000 UTC m=+66.972898877"
Jul 27 19:40:38.966908 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.966877    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:38.966908 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.966910    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66b96198-308c-49c0-bb97-bb8210624656-metrics-tls\") pod \"dns-default-mcmp4\" (UID: \"66b96198-308c-49c0-bb97-bb8210624656\") " pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:38.969489 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.969460    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66b96198-308c-49c0-bb97-bb8210624656-metrics-tls\") pod \"dns-default-mcmp4\" (UID: \"66b96198-308c-49c0-bb97-bb8210624656\") " pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:38.969594 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.969518    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls\") pod \"image-registry-66df67d8d5-nx4mf\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") " pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:38.983547 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.983523    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tnv29\""
Jul 27 19:40:38.992209 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:38.992186    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:39.144916 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:39.144885    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mcmp4"]
Jul 27 19:40:39.151271 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:40:39.151241    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66b96198_308c_49c0_bb97_bb8210624656.slice/crio-110704b84f57c187f32f396b14e3a28e287ab37b8c214674102287aa126082e0 WatchSource:0}: Error finding container 110704b84f57c187f32f396b14e3a28e287ab37b8c214674102287aa126082e0: Status 404 returned error can't find the container with id 110704b84f57c187f32f396b14e3a28e287ab37b8c214674102287aa126082e0
Jul 27 19:40:39.271683 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:39.271606    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4lj5s\""
Jul 27 19:40:39.279685 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:39.279656    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:39.418371 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:39.418339    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66df67d8d5-nx4mf"]
Jul 27 19:40:39.421718 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:40:39.421691    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2343471b_c979_4422_a9d6_f30610099419.slice/crio-56e32f396b0a73753d449434781668d30ceb9a035d93a8d7b4231d38a1267219 WatchSource:0}: Error finding container 56e32f396b0a73753d449434781668d30ceb9a035d93a8d7b4231d38a1267219: Status 404 returned error can't find the container with id 56e32f396b0a73753d449434781668d30ceb9a035d93a8d7b4231d38a1267219
Jul 27 19:40:39.731104 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:39.731066    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mcmp4" event={"ID":"66b96198-308c-49c0-bb97-bb8210624656","Type":"ContainerStarted","Data":"110704b84f57c187f32f396b14e3a28e287ab37b8c214674102287aa126082e0"}
Jul 27 19:40:39.734424 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:39.734400    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf" event={"ID":"2343471b-c979-4422-a9d6-f30610099419","Type":"ContainerStarted","Data":"e7b70c91d07a60cdd9fa870b4f8ec0f2b7c6cb8fe66575bd57a251aea204c838"}
Jul 27 19:40:39.734546 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:39.734435    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf" event={"ID":"2343471b-c979-4422-a9d6-f30610099419","Type":"ContainerStarted","Data":"56e32f396b0a73753d449434781668d30ceb9a035d93a8d7b4231d38a1267219"}
Jul 27 19:40:39.734546 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:39.734461    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:40:39.754063 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:39.754022    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf" podStartSLOduration=44.754007872 podStartE2EDuration="44.754007872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:39:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:40:39.753812962 +0000 UTC m=+67.983055160" watchObservedRunningTime="2026-07-27 19:40:39.754007872 +0000 UTC m=+67.983250069"
Jul 27 19:40:40.256652 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:40.256620    2579 scope.go:117] "RemoveContainer" containerID="0a198de49cbf5e9a418830e0e1d6a7f8209740ee4c1fc35b74f66daa8ef88e06"
Jul 27 19:40:40.742277 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:40.742235    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 19:40:40.742927 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:40.742842    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-xfztm" event={"ID":"e814129b-3b96-4c75-ba5b-480e39d6d71e","Type":"ContainerStarted","Data":"74cd8d437a077edd5f59eb7f43dba0609b212086a034ad7fb97265d88bb78360"}
Jul 27 19:40:40.743262 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:40.743243    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-xfztm"
Jul 27 19:40:40.761980 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:40.760283    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-xfztm" podStartSLOduration=17.344094628 podStartE2EDuration="23.760266055s" totalImagesPullingTime="6.416171427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:40:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:40:40.758981497 +0000 UTC m=+68.988223696" watchObservedRunningTime="2026-07-27 19:40:40.760266055 +0000 UTC m=+68.989508257"
Jul 27 19:40:41.408234 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:41.408207    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-xfztm"
Jul 27 19:40:41.749208 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:41.749166    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mcmp4" event={"ID":"66b96198-308c-49c0-bb97-bb8210624656","Type":"ContainerStarted","Data":"be103e99405ea7bf5a728a74cd08725cd26a548ee38432248abd25224c3922c2"}
Jul 27 19:40:41.749363 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:41.749228    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mcmp4" event={"ID":"66b96198-308c-49c0-bb97-bb8210624656","Type":"ContainerStarted","Data":"3696881d70ddaba710d9c8545c874596c2796fd28790e165f977875981d41d8a"}
Jul 27 19:40:41.766657 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:41.766611    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mcmp4" podStartSLOduration=32.754279962 podStartE2EDuration="34.766595295s" totalImagesPullingTime="2.012315333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:40:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:40:41.765122058 +0000 UTC m=+69.994364268" watchObservedRunningTime="2026-07-27 19:40:41.766595295 +0000 UTC m=+69.995837492"
Jul 27 19:40:42.402975 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:42.402941    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/722f2327-d04b-4470-aea1-be7c383a497a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:42.403159 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:42.403011    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/722f2327-d04b-4470-aea1-be7c383a497a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:42.403159 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:42.403061    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 19:40:42.403159 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:40:42.403135    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/722f2327-d04b-4470-aea1-be7c383a497a-insights-runtime-extractor-tls podName:722f2327-d04b-4470-aea1-be7c383a497a nodeName:}" failed. No retries permitted until 2026-07-27 19:40:58.403116226 +0000 UTC m=+86.632358422 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/722f2327-d04b-4470-aea1-be7c383a497a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lppjn" (UID: "722f2327-d04b-4470-aea1-be7c383a497a") : secret "insights-runtime-extractor-tls" not found
Jul 27 19:40:42.403566 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:42.403546    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/722f2327-d04b-4470-aea1-be7c383a497a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:42.753928 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:42.753897    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:43.757966 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:43.757919    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7gt55" event={"ID":"6407e54e-d37a-4344-91d3-4494c81246c4","Type":"ContainerStarted","Data":"2fe1c1c22633309939144310521d889d8d175a4e451889fc7197c4c35f183951"}
Jul 27 19:40:43.772794 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:43.772748    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7gt55" podStartSLOduration=66.273772879 podStartE2EDuration="1m10.772733377s" totalImagesPullingTime="4.498960498s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:40:43.772179607 +0000 UTC m=+72.001421804" watchObservedRunningTime="2026-07-27 19:40:43.772733377 +0000 UTC m=+72.001975574"
Jul 27 19:40:49.561551 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:49.561510    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8298c2f2-f22a-41cf-8202-ba1c367add65-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-6tml7\" (UID: \"8298c2f2-f22a-41cf-8202-ba1c367add65\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6tml7"
Jul 27 19:40:49.564014 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:49.563992    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8298c2f2-f22a-41cf-8202-ba1c367add65-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-6tml7\" (UID: \"8298c2f2-f22a-41cf-8202-ba1c367add65\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6tml7"
Jul 27 19:40:49.662610 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:49.662576    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0a45576e-6f99-4ff2-bdbe-934a603e6756-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kx5wp\" (UID: \"0a45576e-6f99-4ff2-bdbe-934a603e6756\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp"
Jul 27 19:40:49.665035 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:49.665011    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0a45576e-6f99-4ff2-bdbe-934a603e6756-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kx5wp\" (UID: \"0a45576e-6f99-4ff2-bdbe-934a603e6756\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp"
Jul 27 19:40:49.763217 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:49.763185    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6c35228-a7c8-44a3-9b48-63c27bb72b25-service-ca-bundle\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:49.763340 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:49.763233    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-metrics-certs\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:49.763898 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:49.763847    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6c35228-a7c8-44a3-9b48-63c27bb72b25-service-ca-bundle\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:49.765543 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:49.765526    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6c35228-a7c8-44a3-9b48-63c27bb72b25-metrics-certs\") pod \"router-default-5f5b9f5469-b26r4\" (UID: \"c6c35228-a7c8-44a3-9b48-63c27bb72b25\") " pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:49.849487 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:49.849426    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6xcrb\""
Jul 27 19:40:49.857912 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:49.857893    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6tml7"
Jul 27 19:40:49.913735 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:49.913715    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-vzkmz\""
Jul 27 19:40:49.922341 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:49.922322    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp"
Jul 27 19:40:50.005369 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:50.005134    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tmsww\""
Jul 27 19:40:50.013712 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:50.013690    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:50.035680 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:50.035656    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6tml7"]
Jul 27 19:40:50.082248 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:50.082163    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp"]
Jul 27 19:40:50.085484 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:40:50.085459    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a45576e_6f99_4ff2_bdbe_934a603e6756.slice/crio-33ef5237ad389bafcb7761bbd5fc3c528d6a0ebb4d58525d27b42b4f6ed4abe8 WatchSource:0}: Error finding container 33ef5237ad389bafcb7761bbd5fc3c528d6a0ebb4d58525d27b42b4f6ed4abe8: Status 404 returned error can't find the container with id 33ef5237ad389bafcb7761bbd5fc3c528d6a0ebb4d58525d27b42b4f6ed4abe8
Jul 27 19:40:50.173507 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:50.173480    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5f5b9f5469-b26r4"]
Jul 27 19:40:50.177510 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:40:50.177484    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6c35228_a7c8_44a3_9b48_63c27bb72b25.slice/crio-9f277f954d095c7e29564976744334720de618b829f20aca31e0657fe29dc6e3 WatchSource:0}: Error finding container 9f277f954d095c7e29564976744334720de618b829f20aca31e0657fe29dc6e3: Status 404 returned error can't find the container with id 9f277f954d095c7e29564976744334720de618b829f20aca31e0657fe29dc6e3
Jul 27 19:40:50.783710 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:50.783667    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp" event={"ID":"0a45576e-6f99-4ff2-bdbe-934a603e6756","Type":"ContainerStarted","Data":"33ef5237ad389bafcb7761bbd5fc3c528d6a0ebb4d58525d27b42b4f6ed4abe8"}
Jul 27 19:40:50.785511 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:50.785481    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5f5b9f5469-b26r4" event={"ID":"c6c35228-a7c8-44a3-9b48-63c27bb72b25","Type":"ContainerStarted","Data":"7c87c5de83c2366c2ec18a1c1a83adf955bde026316789fd8d871e223bd9bc9b"}
Jul 27 19:40:50.785668 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:50.785528    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5f5b9f5469-b26r4" event={"ID":"c6c35228-a7c8-44a3-9b48-63c27bb72b25","Type":"ContainerStarted","Data":"9f277f954d095c7e29564976744334720de618b829f20aca31e0657fe29dc6e3"}
Jul 27 19:40:50.786849 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:50.786825    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6tml7" event={"ID":"8298c2f2-f22a-41cf-8202-ba1c367add65","Type":"ContainerStarted","Data":"d359ead9810b99250b8ec3c12274d330f1a78114671c1c30e2948008bb85aa3b"}
Jul 27 19:40:51.014355 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:51.014317    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:51.017381 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:51.017358    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:51.038204 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:51.038122    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5f5b9f5469-b26r4" podStartSLOduration=34.038104241 podStartE2EDuration="34.038104241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:40:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:40:50.812626643 +0000 UTC m=+79.041868840" watchObservedRunningTime="2026-07-27 19:40:51.038104241 +0000 UTC m=+79.267346442"
Jul 27 19:40:51.791487 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:51.791449    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:51.792876 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:51.792836    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5f5b9f5469-b26r4"
Jul 27 19:40:52.760124 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:52.760092    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mcmp4"
Jul 27 19:40:53.799295 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:53.799258    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp" event={"ID":"0a45576e-6f99-4ff2-bdbe-934a603e6756","Type":"ContainerStarted","Data":"a589b4b8ce89e7f06f83644211b717b711fd1d213bd733806bbaf5e91c211bf1"}
Jul 27 19:40:53.801162 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:53.801136    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6tml7" event={"ID":"8298c2f2-f22a-41cf-8202-ba1c367add65","Type":"ContainerStarted","Data":"eb5a9c611e4fd87eb968808fd43e86d745b3184ec1f89d8dec518e4bbffd8f21"}
Jul 27 19:40:53.801298 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:53.801189    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6tml7" event={"ID":"8298c2f2-f22a-41cf-8202-ba1c367add65","Type":"ContainerStarted","Data":"1bbd704c1699143710432e4399b19ee4928ba92c3fa55569e4cd5da78d4cd19a"}
Jul 27 19:40:53.814232 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:53.814182    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kx5wp" podStartSLOduration=33.879804707 podStartE2EDuration="36.814168723s" totalImagesPullingTime="2.934364016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:40:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:40:53.813918515 +0000 UTC m=+82.043160710" watchObservedRunningTime="2026-07-27 19:40:53.814168723 +0000 UTC m=+82.043410914"
Jul 27 19:40:53.831820 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:53.831771    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-6tml7" podStartSLOduration=33.927551301 podStartE2EDuration="36.831754342s" totalImagesPullingTime="2.904203041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:40:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:40:53.830062326 +0000 UTC m=+82.059304524" watchObservedRunningTime="2026-07-27 19:40:53.831754342 +0000 UTC m=+82.060996541"
Jul 27 19:40:58.432113 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:58.432075    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/722f2327-d04b-4470-aea1-be7c383a497a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:58.434623 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:58.434598    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/722f2327-d04b-4470-aea1-be7c383a497a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lppjn\" (UID: \"722f2327-d04b-4470-aea1-be7c383a497a\") " pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:58.683795 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:58.683728    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-csj86\""
Jul 27 19:40:58.691814 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:58.691793    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lppjn"
Jul 27 19:40:58.821687 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:58.821583    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lppjn"]
Jul 27 19:40:58.824179 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:40:58.824150    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod722f2327_d04b_4470_aea1_be7c383a497a.slice/crio-090ea2903db86758405f5ebe102555874e461ae68f55e028ef328274e907f58f WatchSource:0}: Error finding container 090ea2903db86758405f5ebe102555874e461ae68f55e028ef328274e907f58f: Status 404 returned error can't find the container with id 090ea2903db86758405f5ebe102555874e461ae68f55e028ef328274e907f58f
Jul 27 19:40:59.286519 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:59.286476    2579 patch_prober.go:28] interesting pod/image-registry-66df67d8d5-nx4mf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 27 19:40:59.286751 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:59.286557    2579 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf" podUID="2343471b-c979-4422-a9d6-f30610099419" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 19:40:59.824667 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:40:59.824628    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lppjn" event={"ID":"722f2327-d04b-4470-aea1-be7c383a497a","Type":"ContainerStarted","Data":"090ea2903db86758405f5ebe102555874e461ae68f55e028ef328274e907f58f"}
Jul 27 19:41:00.748330 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:00.748299    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:41:03.841838 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:03.841802    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lppjn" event={"ID":"722f2327-d04b-4470-aea1-be7c383a497a","Type":"ContainerStarted","Data":"adf56984b7bf32cfe2ce779849c2205c4e297930d7c0780a6f6fc03523dd0007"}
Jul 27 19:41:07.860504 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:07.860464    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lppjn" event={"ID":"722f2327-d04b-4470-aea1-be7c383a497a","Type":"ContainerStarted","Data":"995ac2178a6b15cd742b4dd4b2dbd182aa096141c536ed10312cd6b240485298"}
Jul 27 19:41:09.737115 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:09.737083    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fpbzl"
Jul 27 19:41:10.874172 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:10.874141    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lppjn" event={"ID":"722f2327-d04b-4470-aea1-be7c383a497a","Type":"ContainerStarted","Data":"75d99cb8a3216f2e502791ecc9c61c5568d0e7ea47c7a68ef4919dd7cef5e3c1"}
Jul 27 19:41:10.894160 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:10.894071    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lppjn" podStartSLOduration=33.208999291 podStartE2EDuration="44.894056427s" totalImagesPullingTime="11.685057136s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:40:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:41:10.892937576 +0000 UTC m=+99.122179774" watchObservedRunningTime="2026-07-27 19:41:10.894056427 +0000 UTC m=+99.123298626"
Jul 27 19:41:11.116099 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.116064    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-j4lvl"]
Jul 27 19:41:11.119709 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.119690    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.125075 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.125055    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 27 19:41:11.126127 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.126106    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 27 19:41:11.126570 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.126554    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 27 19:41:11.128258 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.128240    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rvkzp\""
Jul 27 19:41:11.129368 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.129355    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 27 19:41:11.242770 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.242738    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/74bf592e-5185-4b72-92f7-a516401d267f-root\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.242770 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.242769    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74bf592e-5185-4b72-92f7-a516401d267f-node-exporter-tls\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.242988 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.242788    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/74bf592e-5185-4b72-92f7-a516401d267f-node-exporter-wtmp\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.242988 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.242817    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7ldm\" (UniqueName: \"kubernetes.io/projected/74bf592e-5185-4b72-92f7-a516401d267f-kube-api-access-n7ldm\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.242988 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.242851    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74bf592e-5185-4b72-92f7-a516401d267f-metrics-client-ca\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.242988 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.242926    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/74bf592e-5185-4b72-92f7-a516401d267f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.242988 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.242980    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/74bf592e-5185-4b72-92f7-a516401d267f-node-exporter-accelerators-collector-config\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.243163 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.243004    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74bf592e-5185-4b72-92f7-a516401d267f-sys\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.243163 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.243031    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/74bf592e-5185-4b72-92f7-a516401d267f-node-exporter-textfile\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.343413 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.343379    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/74bf592e-5185-4b72-92f7-a516401d267f-node-exporter-accelerators-collector-config\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.343413 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.343416    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74bf592e-5185-4b72-92f7-a516401d267f-sys\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.343612 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.343435    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/74bf592e-5185-4b72-92f7-a516401d267f-node-exporter-textfile\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.343612 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.343475    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/74bf592e-5185-4b72-92f7-a516401d267f-root\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.343612 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.343490    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74bf592e-5185-4b72-92f7-a516401d267f-node-exporter-tls\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.343612 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.343504    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/74bf592e-5185-4b72-92f7-a516401d267f-node-exporter-wtmp\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.343612 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.343499    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74bf592e-5185-4b72-92f7-a516401d267f-sys\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.343612 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.343521    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7ldm\" (UniqueName: \"kubernetes.io/projected/74bf592e-5185-4b72-92f7-a516401d267f-kube-api-access-n7ldm\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.343612 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.343577    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/74bf592e-5185-4b72-92f7-a516401d267f-root\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.343612 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.343589    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74bf592e-5185-4b72-92f7-a516401d267f-metrics-client-ca\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.344003 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.343626    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/74bf592e-5185-4b72-92f7-a516401d267f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.344003 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.343659    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/74bf592e-5185-4b72-92f7-a516401d267f-node-exporter-wtmp\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.344003 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.343975    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/74bf592e-5185-4b72-92f7-a516401d267f-node-exporter-textfile\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.344107 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.344094    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/74bf592e-5185-4b72-92f7-a516401d267f-node-exporter-accelerators-collector-config\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.344440 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.344423    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74bf592e-5185-4b72-92f7-a516401d267f-metrics-client-ca\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.345971 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.345943    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/74bf592e-5185-4b72-92f7-a516401d267f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.346134 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.346116    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74bf592e-5185-4b72-92f7-a516401d267f-node-exporter-tls\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.351341 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.351319    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7ldm\" (UniqueName: \"kubernetes.io/projected/74bf592e-5185-4b72-92f7-a516401d267f-kube-api-access-n7ldm\") pod \"node-exporter-j4lvl\" (UID: \"74bf592e-5185-4b72-92f7-a516401d267f\") " pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.428434 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.428407    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-j4lvl"
Jul 27 19:41:11.441732 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:41:11.441707    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74bf592e_5185_4b72_92f7_a516401d267f.slice/crio-579ed443c26431f324dc145d31b99f6bdbca3b75cbf8794c6a580d4f91fb632f WatchSource:0}: Error finding container 579ed443c26431f324dc145d31b99f6bdbca3b75cbf8794c6a580d4f91fb632f: Status 404 returned error can't find the container with id 579ed443c26431f324dc145d31b99f6bdbca3b75cbf8794c6a580d4f91fb632f
Jul 27 19:41:11.880066 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:11.879982    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j4lvl" event={"ID":"74bf592e-5185-4b72-92f7-a516401d267f","Type":"ContainerStarted","Data":"579ed443c26431f324dc145d31b99f6bdbca3b75cbf8794c6a580d4f91fb632f"}
Jul 27 19:41:12.885187 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:12.885112    2579 generic.go:356] "Generic (PLEG): container finished" podID="74bf592e-5185-4b72-92f7-a516401d267f" containerID="222de59b81b57c39b8d6fc388afce149867ba139fb9d1d6207e84853f562d12e" exitCode=0
Jul 27 19:41:12.885531 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:12.885194    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j4lvl" event={"ID":"74bf592e-5185-4b72-92f7-a516401d267f","Type":"ContainerDied","Data":"222de59b81b57c39b8d6fc388afce149867ba139fb9d1d6207e84853f562d12e"}
Jul 27 19:41:13.890493 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:13.890451    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j4lvl" event={"ID":"74bf592e-5185-4b72-92f7-a516401d267f","Type":"ContainerStarted","Data":"ef48c9664592822d4f0675fcac0d8e6c3cbe8c7124f45f8cadc34dbb3e894dd6"}
Jul 27 19:41:13.890885 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:13.890516    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j4lvl" event={"ID":"74bf592e-5185-4b72-92f7-a516401d267f","Type":"ContainerStarted","Data":"89c6f4af057b93c3b98f55b7ec3a81fee2f1ebd3b837db415f9deaf7c3a0bb03"}
Jul 27 19:41:13.911986 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:13.911942    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-j4lvl" podStartSLOduration=1.846889573 podStartE2EDuration="2.911926411s" totalImagesPullingTime="995.386586ms" totalInitContainerRuntime="69.650252ms" isStatefulPod=true podCreationTimestamp="2026-07-27 19:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:41:13.911172183 +0000 UTC m=+102.140414382" watchObservedRunningTime="2026-07-27 19:41:13.911926411 +0000 UTC m=+102.141168609"
Jul 27 19:41:16.414147 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:16.414116    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66df67d8d5-nx4mf"]
Jul 27 19:41:27.445106 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:27.445067    2579 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9" podUID="6bbc6b75-0747-42df-a099-8246656515e1" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 27 19:41:35.979951 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:35.979918    2579 generic.go:356] "Generic (PLEG): container finished" podID="c91d5e40-730e-4dfb-814a-f95776e4b5e9" containerID="4734b605cfb8fde9e1c68166d119cf20dedc9e85fe3c3b90937345f3474824e7" exitCode=0
Jul 27 19:41:35.980372 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:35.979958    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bpst8" event={"ID":"c91d5e40-730e-4dfb-814a-f95776e4b5e9","Type":"ContainerDied","Data":"4734b605cfb8fde9e1c68166d119cf20dedc9e85fe3c3b90937345f3474824e7"}
Jul 27 19:41:35.980372 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:35.980225    2579 scope.go:117] "RemoveContainer" containerID="4734b605cfb8fde9e1c68166d119cf20dedc9e85fe3c3b90937345f3474824e7"
Jul 27 19:41:36.984972 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:36.984937    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bpst8" event={"ID":"c91d5e40-730e-4dfb-814a-f95776e4b5e9","Type":"ContainerStarted","Data":"975f75651ed9b3916468b8ef28c5d59315c1db883515873b670e0de231d2e05e"}
Jul 27 19:41:37.445851 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:37.445813    2579 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9" podUID="6bbc6b75-0747-42df-a099-8246656515e1" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 27 19:41:41.433386 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.433338    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf" podUID="2343471b-c979-4422-a9d6-f30610099419" containerName="registry" containerID="cri-o://e7b70c91d07a60cdd9fa870b4f8ec0f2b7c6cb8fe66575bd57a251aea204c838" gracePeriod=30
Jul 27 19:41:41.683754 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.683696    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:41:41.781123 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.781098    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4ll8\" (UniqueName: \"kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-kube-api-access-q4ll8\") pod \"2343471b-c979-4422-a9d6-f30610099419\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") "
Jul 27 19:41:41.781261 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.781129    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2343471b-c979-4422-a9d6-f30610099419-image-registry-private-configuration\") pod \"2343471b-c979-4422-a9d6-f30610099419\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") "
Jul 27 19:41:41.781261 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.781152    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2343471b-c979-4422-a9d6-f30610099419-ca-trust-extracted\") pod \"2343471b-c979-4422-a9d6-f30610099419\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") "
Jul 27 19:41:41.781261 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.781176    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2343471b-c979-4422-a9d6-f30610099419-trusted-ca\") pod \"2343471b-c979-4422-a9d6-f30610099419\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") "
Jul 27 19:41:41.781261 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.781206    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2343471b-c979-4422-a9d6-f30610099419-registry-certificates\") pod \"2343471b-c979-4422-a9d6-f30610099419\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") "
Jul 27 19:41:41.781261 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.781240    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-bound-sa-token\") pod \"2343471b-c979-4422-a9d6-f30610099419\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") "
Jul 27 19:41:41.781496 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.781411    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2343471b-c979-4422-a9d6-f30610099419-installation-pull-secrets\") pod \"2343471b-c979-4422-a9d6-f30610099419\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") "
Jul 27 19:41:41.781496 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.781473    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls\") pod \"2343471b-c979-4422-a9d6-f30610099419\" (UID: \"2343471b-c979-4422-a9d6-f30610099419\") "
Jul 27 19:41:41.781872 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.781693    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2343471b-c979-4422-a9d6-f30610099419-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2343471b-c979-4422-a9d6-f30610099419" (UID: "2343471b-c979-4422-a9d6-f30610099419"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:41:41.781872 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.781687    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2343471b-c979-4422-a9d6-f30610099419-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2343471b-c979-4422-a9d6-f30610099419" (UID: "2343471b-c979-4422-a9d6-f30610099419"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:41:41.783719 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.783691    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-kube-api-access-q4ll8" (OuterVolumeSpecName: "kube-api-access-q4ll8") pod "2343471b-c979-4422-a9d6-f30610099419" (UID: "2343471b-c979-4422-a9d6-f30610099419"). InnerVolumeSpecName "kube-api-access-q4ll8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:41:41.783928 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.783905    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2343471b-c979-4422-a9d6-f30610099419-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2343471b-c979-4422-a9d6-f30610099419" (UID: "2343471b-c979-4422-a9d6-f30610099419"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:41:41.784151 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.784128    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2343471b-c979-4422-a9d6-f30610099419" (UID: "2343471b-c979-4422-a9d6-f30610099419"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:41:41.784268 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.784242    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2343471b-c979-4422-a9d6-f30610099419-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2343471b-c979-4422-a9d6-f30610099419" (UID: "2343471b-c979-4422-a9d6-f30610099419"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:41:41.784330 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.784314    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2343471b-c979-4422-a9d6-f30610099419" (UID: "2343471b-c979-4422-a9d6-f30610099419"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:41:41.790545 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.790522    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2343471b-c979-4422-a9d6-f30610099419-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2343471b-c979-4422-a9d6-f30610099419" (UID: "2343471b-c979-4422-a9d6-f30610099419"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:41:41.882721 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.882703    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4ll8\" (UniqueName: \"kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-kube-api-access-q4ll8\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 19:41:41.882721 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.882722    2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2343471b-c979-4422-a9d6-f30610099419-image-registry-private-configuration\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 19:41:41.882843 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.882732    2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2343471b-c979-4422-a9d6-f30610099419-ca-trust-extracted\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 19:41:41.882843 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.882741    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2343471b-c979-4422-a9d6-f30610099419-trusted-ca\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 19:41:41.882843 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.882749    2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2343471b-c979-4422-a9d6-f30610099419-registry-certificates\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 19:41:41.882843 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.882757    2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-bound-sa-token\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 19:41:41.882843 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.882765    2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2343471b-c979-4422-a9d6-f30610099419-installation-pull-secrets\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 19:41:41.882843 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:41.882773    2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2343471b-c979-4422-a9d6-f30610099419-registry-tls\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 19:41:42.003506 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:42.003446    2579 generic.go:356] "Generic (PLEG): container finished" podID="2343471b-c979-4422-a9d6-f30610099419" containerID="e7b70c91d07a60cdd9fa870b4f8ec0f2b7c6cb8fe66575bd57a251aea204c838" exitCode=0
Jul 27 19:41:42.003506 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:42.003498    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf"
Jul 27 19:41:42.003614 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:42.003529    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf" event={"ID":"2343471b-c979-4422-a9d6-f30610099419","Type":"ContainerDied","Data":"e7b70c91d07a60cdd9fa870b4f8ec0f2b7c6cb8fe66575bd57a251aea204c838"}
Jul 27 19:41:42.003614 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:42.003579    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66df67d8d5-nx4mf" event={"ID":"2343471b-c979-4422-a9d6-f30610099419","Type":"ContainerDied","Data":"56e32f396b0a73753d449434781668d30ceb9a035d93a8d7b4231d38a1267219"}
Jul 27 19:41:42.003614 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:42.003600    2579 scope.go:117] "RemoveContainer" containerID="e7b70c91d07a60cdd9fa870b4f8ec0f2b7c6cb8fe66575bd57a251aea204c838"
Jul 27 19:41:42.012001 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:42.011983    2579 scope.go:117] "RemoveContainer" containerID="e7b70c91d07a60cdd9fa870b4f8ec0f2b7c6cb8fe66575bd57a251aea204c838"
Jul 27 19:41:42.012253 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:41:42.012234    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7b70c91d07a60cdd9fa870b4f8ec0f2b7c6cb8fe66575bd57a251aea204c838\": container with ID starting with e7b70c91d07a60cdd9fa870b4f8ec0f2b7c6cb8fe66575bd57a251aea204c838 not found: ID does not exist" containerID="e7b70c91d07a60cdd9fa870b4f8ec0f2b7c6cb8fe66575bd57a251aea204c838"
Jul 27 19:41:42.012306 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:42.012260    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7b70c91d07a60cdd9fa870b4f8ec0f2b7c6cb8fe66575bd57a251aea204c838"} err="failed to get container status \"e7b70c91d07a60cdd9fa870b4f8ec0f2b7c6cb8fe66575bd57a251aea204c838\": rpc error: code = NotFound desc = could not find container \"e7b70c91d07a60cdd9fa870b4f8ec0f2b7c6cb8fe66575bd57a251aea204c838\": container with ID starting with e7b70c91d07a60cdd9fa870b4f8ec0f2b7c6cb8fe66575bd57a251aea204c838 not found: ID does not exist"
Jul 27 19:41:42.022335 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:42.022316    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66df67d8d5-nx4mf"]
Jul 27 19:41:42.026166 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:42.026147    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-66df67d8d5-nx4mf"]
Jul 27 19:41:42.084584 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:42.084558    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs\") pod \"network-metrics-daemon-sdk4x\" (UID: \"74f89d7b-f93b-4dc3-9e18-41cf466d90bd\") " pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:41:42.086891 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:42.086872    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74f89d7b-f93b-4dc3-9e18-41cf466d90bd-metrics-certs\") pod \"network-metrics-daemon-sdk4x\" (UID: \"74f89d7b-f93b-4dc3-9e18-41cf466d90bd\") " pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:41:42.266417 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:42.264213    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2343471b-c979-4422-a9d6-f30610099419" path="/var/lib/kubelet/pods/2343471b-c979-4422-a9d6-f30610099419/volumes"
Jul 27 19:41:42.370215 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:42.370190    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4fnrz\""
Jul 27 19:41:42.377993 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:42.377975    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdk4x"
Jul 27 19:41:42.508019 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:42.507730    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sdk4x"]
Jul 27 19:41:42.510451 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:41:42.510421    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74f89d7b_f93b_4dc3_9e18_41cf466d90bd.slice/crio-ecfdfc866c2821dbae81232ae411026af8d2e8709698cc0b291d73403668e223 WatchSource:0}: Error finding container ecfdfc866c2821dbae81232ae411026af8d2e8709698cc0b291d73403668e223: Status 404 returned error can't find the container with id ecfdfc866c2821dbae81232ae411026af8d2e8709698cc0b291d73403668e223
Jul 27 19:41:43.009197 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:43.009145    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sdk4x" event={"ID":"74f89d7b-f93b-4dc3-9e18-41cf466d90bd","Type":"ContainerStarted","Data":"ecfdfc866c2821dbae81232ae411026af8d2e8709698cc0b291d73403668e223"}
Jul 27 19:41:44.016152 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:44.016118    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sdk4x" event={"ID":"74f89d7b-f93b-4dc3-9e18-41cf466d90bd","Type":"ContainerStarted","Data":"ca3f6a5c188f4f861e4e558a0b8df0719afb598aa271a1a0a376d7320655f01d"}
Jul 27 19:41:44.016555 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:44.016174    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sdk4x" event={"ID":"74f89d7b-f93b-4dc3-9e18-41cf466d90bd","Type":"ContainerStarted","Data":"577b07a1a5963b5f15234db03ce1ef021dd13c52575ec026730ce08202ff6f7d"}
Jul 27 19:41:44.032180 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:44.032132    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sdk4x" podStartSLOduration=130.045979637 podStartE2EDuration="2m11.032114971s" totalImagesPullingTime="986.135334ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:41:44.031703096 +0000 UTC m=+132.260945294" watchObservedRunningTime="2026-07-27 19:41:44.032114971 +0000 UTC m=+132.261357169"
Jul 27 19:41:46.025031 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:46.025002    2579 generic.go:356] "Generic (PLEG): container finished" podID="b1f4f700-7ce1-4669-a383-793dfbcf4f2e" containerID="d5dcc9cce099535b6a67b9e8a42466999272c4e35ae8249ab3048deffd510dcf" exitCode=0
Jul 27 19:41:46.025373 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:46.025079    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl" event={"ID":"b1f4f700-7ce1-4669-a383-793dfbcf4f2e","Type":"ContainerDied","Data":"d5dcc9cce099535b6a67b9e8a42466999272c4e35ae8249ab3048deffd510dcf"}
Jul 27 19:41:46.025373 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:46.025369    2579 scope.go:117] "RemoveContainer" containerID="d5dcc9cce099535b6a67b9e8a42466999272c4e35ae8249ab3048deffd510dcf"
Jul 27 19:41:47.031129 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:47.031095    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl" event={"ID":"b1f4f700-7ce1-4669-a383-793dfbcf4f2e","Type":"ContainerStarted","Data":"ee5566253ac0ab1ff2409972fc14257c4e8ace288eda768e71aa740f945ea387"}
Jul 27 19:41:47.444724 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:47.444687    2579 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9" podUID="6bbc6b75-0747-42df-a099-8246656515e1" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 27 19:41:47.444905 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:47.444755    2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9"
Jul 27 19:41:47.445242 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:47.445224    2579 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"4dfe8c7062904c17fb30eaa7d9c9a29f7e562e5e56aebb81d03844a20f544d56"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 27 19:41:47.445279 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:47.445263    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9" podUID="6bbc6b75-0747-42df-a099-8246656515e1" containerName="service-proxy" containerID="cri-o://4dfe8c7062904c17fb30eaa7d9c9a29f7e562e5e56aebb81d03844a20f544d56" gracePeriod=30
Jul 27 19:41:48.037405 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:48.037371    2579 generic.go:356] "Generic (PLEG): container finished" podID="6bbc6b75-0747-42df-a099-8246656515e1" containerID="4dfe8c7062904c17fb30eaa7d9c9a29f7e562e5e56aebb81d03844a20f544d56" exitCode=2
Jul 27 19:41:48.037760 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:48.037437    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9" event={"ID":"6bbc6b75-0747-42df-a099-8246656515e1","Type":"ContainerDied","Data":"4dfe8c7062904c17fb30eaa7d9c9a29f7e562e5e56aebb81d03844a20f544d56"}
Jul 27 19:41:48.037760 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:41:48.037468    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c67cbbd4c-bzkw9" event={"ID":"6bbc6b75-0747-42df-a099-8246656515e1","Type":"ContainerStarted","Data":"890968d601020ec7915fbc848c645c717f4de0a249a299947b20b65f137d0f09"}
Jul 27 19:42:02.089914 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:42:02.089880    2579 generic.go:356] "Generic (PLEG): container finished" podID="096c978c-0cb6-4ff6-8fba-93b2113f24a4" containerID="e54f030e7766bfaa3edd114711d8677c5ee9b9c6b38816b6e70c9adef8e4fb27" exitCode=0
Jul 27 19:42:02.090434 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:42:02.089950    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-n9pwk" event={"ID":"096c978c-0cb6-4ff6-8fba-93b2113f24a4","Type":"ContainerDied","Data":"e54f030e7766bfaa3edd114711d8677c5ee9b9c6b38816b6e70c9adef8e4fb27"}
Jul 27 19:42:02.090434 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:42:02.090364    2579 scope.go:117] "RemoveContainer" containerID="e54f030e7766bfaa3edd114711d8677c5ee9b9c6b38816b6e70c9adef8e4fb27"
Jul 27 19:42:03.095774 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:42:03.095741    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-n9pwk" event={"ID":"096c978c-0cb6-4ff6-8fba-93b2113f24a4","Type":"ContainerStarted","Data":"3338c4f9c6faab7b0537d95558adf7b4c4de2e0d5bb0d655ce37b5894ddf0063"}
Jul 27 19:44:20.285577 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.285545    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"]
Jul 27 19:44:20.285998 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.285799    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2343471b-c979-4422-a9d6-f30610099419" containerName="registry"
Jul 27 19:44:20.285998 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.285809    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2343471b-c979-4422-a9d6-f30610099419" containerName="registry"
Jul 27 19:44:20.285998 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.285891    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2343471b-c979-4422-a9d6-f30610099419" containerName="registry"
Jul 27 19:44:20.288685 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.288667    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"
Jul 27 19:44:20.295287 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.294926    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 27 19:44:20.295287 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.295150    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 27 19:44:20.295287 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.295065    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 27 19:44:20.295287 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.295133    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 27 19:44:20.295824 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.295808    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6clqf\""
Jul 27 19:44:20.314128 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.314105    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"]
Jul 27 19:44:20.392760 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.392731    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9mdn\" (UniqueName: \"kubernetes.io/projected/482626a0-f93e-4282-9c33-ee3ea818e5d1-kube-api-access-n9mdn\") pod \"opendatahub-operator-controller-manager-7df5588f67-9f6vv\" (UID: \"482626a0-f93e-4282-9c33-ee3ea818e5d1\") " pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"
Jul 27 19:44:20.392929 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.392766    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/482626a0-f93e-4282-9c33-ee3ea818e5d1-webhook-cert\") pod \"opendatahub-operator-controller-manager-7df5588f67-9f6vv\" (UID: \"482626a0-f93e-4282-9c33-ee3ea818e5d1\") " pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"
Jul 27 19:44:20.392929 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.392793    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/482626a0-f93e-4282-9c33-ee3ea818e5d1-tmp\") pod \"opendatahub-operator-controller-manager-7df5588f67-9f6vv\" (UID: \"482626a0-f93e-4282-9c33-ee3ea818e5d1\") " pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"
Jul 27 19:44:20.392929 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.392818    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/482626a0-f93e-4282-9c33-ee3ea818e5d1-manifests\") pod \"opendatahub-operator-controller-manager-7df5588f67-9f6vv\" (UID: \"482626a0-f93e-4282-9c33-ee3ea818e5d1\") " pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"
Jul 27 19:44:20.392929 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.392905    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/482626a0-f93e-4282-9c33-ee3ea818e5d1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7df5588f67-9f6vv\" (UID: \"482626a0-f93e-4282-9c33-ee3ea818e5d1\") " pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"
Jul 27 19:44:20.493877 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.493818    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/482626a0-f93e-4282-9c33-ee3ea818e5d1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7df5588f67-9f6vv\" (UID: \"482626a0-f93e-4282-9c33-ee3ea818e5d1\") " pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"
Jul 27 19:44:20.494042 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.493915    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9mdn\" (UniqueName: \"kubernetes.io/projected/482626a0-f93e-4282-9c33-ee3ea818e5d1-kube-api-access-n9mdn\") pod \"opendatahub-operator-controller-manager-7df5588f67-9f6vv\" (UID: \"482626a0-f93e-4282-9c33-ee3ea818e5d1\") " pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"
Jul 27 19:44:20.494042 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.493945    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/482626a0-f93e-4282-9c33-ee3ea818e5d1-webhook-cert\") pod \"opendatahub-operator-controller-manager-7df5588f67-9f6vv\" (UID: \"482626a0-f93e-4282-9c33-ee3ea818e5d1\") " pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"
Jul 27 19:44:20.494042 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.493983    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/482626a0-f93e-4282-9c33-ee3ea818e5d1-tmp\") pod \"opendatahub-operator-controller-manager-7df5588f67-9f6vv\" (UID: \"482626a0-f93e-4282-9c33-ee3ea818e5d1\") " pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"
Jul 27 19:44:20.494042 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.494009    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/482626a0-f93e-4282-9c33-ee3ea818e5d1-manifests\") pod \"opendatahub-operator-controller-manager-7df5588f67-9f6vv\" (UID: \"482626a0-f93e-4282-9c33-ee3ea818e5d1\") " pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"
Jul 27 19:44:20.494435 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.494411    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/482626a0-f93e-4282-9c33-ee3ea818e5d1-manifests\") pod \"opendatahub-operator-controller-manager-7df5588f67-9f6vv\" (UID: \"482626a0-f93e-4282-9c33-ee3ea818e5d1\") " pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"
Jul 27 19:44:20.496413 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.496382    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/482626a0-f93e-4282-9c33-ee3ea818e5d1-tmp\") pod \"opendatahub-operator-controller-manager-7df5588f67-9f6vv\" (UID: \"482626a0-f93e-4282-9c33-ee3ea818e5d1\") " pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"
Jul 27 19:44:20.496602 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.496580    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/482626a0-f93e-4282-9c33-ee3ea818e5d1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7df5588f67-9f6vv\" (UID: \"482626a0-f93e-4282-9c33-ee3ea818e5d1\") " pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"
Jul 27 19:44:20.496602 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.496595    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/482626a0-f93e-4282-9c33-ee3ea818e5d1-webhook-cert\") pod \"opendatahub-operator-controller-manager-7df5588f67-9f6vv\" (UID: \"482626a0-f93e-4282-9c33-ee3ea818e5d1\") " pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"
Jul 27 19:44:20.504899 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.504879    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9mdn\" (UniqueName: \"kubernetes.io/projected/482626a0-f93e-4282-9c33-ee3ea818e5d1-kube-api-access-n9mdn\") pod \"opendatahub-operator-controller-manager-7df5588f67-9f6vv\" (UID: \"482626a0-f93e-4282-9c33-ee3ea818e5d1\") " pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"
Jul 27 19:44:20.599531 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.599446    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"
Jul 27 19:44:20.749254 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:20.749224    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"]
Jul 27 19:44:20.752958 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:44:20.752928    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod482626a0_f93e_4282_9c33_ee3ea818e5d1.slice/crio-a98b1501a84a6f4212d609f19d7f26423bc44c9cb12f1d275b744522a67a49d8 WatchSource:0}: Error finding container a98b1501a84a6f4212d609f19d7f26423bc44c9cb12f1d275b744522a67a49d8: Status 404 returned error can't find the container with id a98b1501a84a6f4212d609f19d7f26423bc44c9cb12f1d275b744522a67a49d8
Jul 27 19:44:21.610451 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:21.610413    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv" event={"ID":"482626a0-f93e-4282-9c33-ee3ea818e5d1","Type":"ContainerStarted","Data":"a98b1501a84a6f4212d609f19d7f26423bc44c9cb12f1d275b744522a67a49d8"}
Jul 27 19:44:24.625209 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:24.625171    2579 generic.go:356] "Generic (PLEG): container finished" podID="482626a0-f93e-4282-9c33-ee3ea818e5d1" containerID="28a3b06415baf56c3ed46d321d6796d093ea0061c0f0d6e5edff0a2b5b10dcdb" exitCode=0
Jul 27 19:44:24.625541 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:24.625208    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv" event={"ID":"482626a0-f93e-4282-9c33-ee3ea818e5d1","Type":"ContainerDied","Data":"28a3b06415baf56c3ed46d321d6796d093ea0061c0f0d6e5edff0a2b5b10dcdb"}
Jul 27 19:44:25.631354 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:25.631321    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv" event={"ID":"482626a0-f93e-4282-9c33-ee3ea818e5d1","Type":"ContainerStarted","Data":"fbad136e191606272a8c5092e016b6dcc892e74f01f110caab3af1fc8825feab"}
Jul 27 19:44:25.631785 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:25.631383    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"
Jul 27 19:44:25.654263 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:25.654220    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv" podStartSLOduration=1.4768083220000001 podStartE2EDuration="5.654204425s" totalImagesPullingTime="3.264600052s" totalInitContainerRuntime="912.796051ms" isStatefulPod=false podCreationTimestamp="2026-07-27 19:44:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:44:25.652921608 +0000 UTC m=+293.882163806" watchObservedRunningTime="2026-07-27 19:44:25.654204425 +0000 UTC m=+293.883446623"
Jul 27 19:44:32.194196 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:32.194165    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 19:44:32.197204 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:32.197182    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 19:44:32.207160 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:32.207140    2579 kubelet.go:1648] "Image garbage collection succeeded"
Jul 27 19:44:36.636937 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:36.636909    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7df5588f67-9f6vv"
Jul 27 19:44:39.694667 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:39.694630    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5fd5d97dbc-7xtj6"]
Jul 27 19:44:39.698588 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:39.698568    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5fd5d97dbc-7xtj6"
Jul 27 19:44:39.703581 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:39.703162    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 27 19:44:39.707641 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:39.707620    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 27 19:44:39.707990 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:39.707969    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-zq4br\""
Jul 27 19:44:39.732752 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:39.732721    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6f11af4-1ae4-472e-9383-774dbb37ab62-tmp\") pod \"kube-auth-proxy-5fd5d97dbc-7xtj6\" (UID: \"c6f11af4-1ae4-472e-9383-774dbb37ab62\") " pod="openshift-ingress/kube-auth-proxy-5fd5d97dbc-7xtj6"
Jul 27 19:44:39.732752 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:39.732754    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6f11af4-1ae4-472e-9383-774dbb37ab62-tls-certs\") pod \"kube-auth-proxy-5fd5d97dbc-7xtj6\" (UID: \"c6f11af4-1ae4-472e-9383-774dbb37ab62\") " pod="openshift-ingress/kube-auth-proxy-5fd5d97dbc-7xtj6"
Jul 27 19:44:39.732942 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:39.732869    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jm5q\" (UniqueName: \"kubernetes.io/projected/c6f11af4-1ae4-472e-9383-774dbb37ab62-kube-api-access-7jm5q\") pod \"kube-auth-proxy-5fd5d97dbc-7xtj6\" (UID: \"c6f11af4-1ae4-472e-9383-774dbb37ab62\") " pod="openshift-ingress/kube-auth-proxy-5fd5d97dbc-7xtj6"
Jul 27 19:44:39.764307 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:39.764283    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5fd5d97dbc-7xtj6"]
Jul 27 19:44:39.833639 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:39.833607    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6f11af4-1ae4-472e-9383-774dbb37ab62-tmp\") pod \"kube-auth-proxy-5fd5d97dbc-7xtj6\" (UID: \"c6f11af4-1ae4-472e-9383-774dbb37ab62\") " pod="openshift-ingress/kube-auth-proxy-5fd5d97dbc-7xtj6"
Jul 27 19:44:39.833639 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:39.833642    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6f11af4-1ae4-472e-9383-774dbb37ab62-tls-certs\") pod \"kube-auth-proxy-5fd5d97dbc-7xtj6\" (UID: \"c6f11af4-1ae4-472e-9383-774dbb37ab62\") " pod="openshift-ingress/kube-auth-proxy-5fd5d97dbc-7xtj6"
Jul 27 19:44:39.833845 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:39.833689    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jm5q\" (UniqueName: \"kubernetes.io/projected/c6f11af4-1ae4-472e-9383-774dbb37ab62-kube-api-access-7jm5q\") pod \"kube-auth-proxy-5fd5d97dbc-7xtj6\" (UID: \"c6f11af4-1ae4-472e-9383-774dbb37ab62\") " pod="openshift-ingress/kube-auth-proxy-5fd5d97dbc-7xtj6"
Jul 27 19:44:39.836134 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:39.836107    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6f11af4-1ae4-472e-9383-774dbb37ab62-tmp\") pod \"kube-auth-proxy-5fd5d97dbc-7xtj6\" (UID: \"c6f11af4-1ae4-472e-9383-774dbb37ab62\") " pod="openshift-ingress/kube-auth-proxy-5fd5d97dbc-7xtj6"
Jul 27 19:44:39.836465 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:39.836446    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6f11af4-1ae4-472e-9383-774dbb37ab62-tls-certs\") pod \"kube-auth-proxy-5fd5d97dbc-7xtj6\" (UID: \"c6f11af4-1ae4-472e-9383-774dbb37ab62\") " pod="openshift-ingress/kube-auth-proxy-5fd5d97dbc-7xtj6"
Jul 27 19:44:39.852336 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:39.852312    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jm5q\" (UniqueName: \"kubernetes.io/projected/c6f11af4-1ae4-472e-9383-774dbb37ab62-kube-api-access-7jm5q\") pod \"kube-auth-proxy-5fd5d97dbc-7xtj6\" (UID: \"c6f11af4-1ae4-472e-9383-774dbb37ab62\") " pod="openshift-ingress/kube-auth-proxy-5fd5d97dbc-7xtj6"
Jul 27 19:44:40.009046 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:40.008961    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5fd5d97dbc-7xtj6"
Jul 27 19:44:40.156503 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:40.156479    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5fd5d97dbc-7xtj6"]
Jul 27 19:44:40.158548 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:44:40.158506    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6f11af4_1ae4_472e_9383_774dbb37ab62.slice/crio-945a9285946732b09cf8fb9926059e69c54e08ada9a9bb6a2ed0e45b80578c9f WatchSource:0}: Error finding container 945a9285946732b09cf8fb9926059e69c54e08ada9a9bb6a2ed0e45b80578c9f: Status 404 returned error can't find the container with id 945a9285946732b09cf8fb9926059e69c54e08ada9a9bb6a2ed0e45b80578c9f
Jul 27 19:44:40.163022 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:40.161681    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 19:44:40.691412 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:40.691374    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5fd5d97dbc-7xtj6" event={"ID":"c6f11af4-1ae4-472e-9383-774dbb37ab62","Type":"ContainerStarted","Data":"945a9285946732b09cf8fb9926059e69c54e08ada9a9bb6a2ed0e45b80578c9f"}
Jul 27 19:44:43.707140 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:43.707099    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5fd5d97dbc-7xtj6" event={"ID":"c6f11af4-1ae4-472e-9383-774dbb37ab62","Type":"ContainerStarted","Data":"2e4fb0d38a24a406885635abe50e4cd490b7709f5d242323c64173301ee2b5e3"}
Jul 27 19:44:43.723744 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:43.723695    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5fd5d97dbc-7xtj6" podStartSLOduration=2.084227645 podStartE2EDuration="4.723681701s" totalImagesPullingTime="2.639454056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:44:43.723237984 +0000 UTC m=+311.952480205" watchObservedRunningTime="2026-07-27 19:44:43.723681701 +0000 UTC m=+311.952923899"
Jul 27 19:44:47.984039 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:47.983999    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8689f9464-g8plw"]
Jul 27 19:44:47.988455 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:47.988432    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8689f9464-g8plw"
Jul 27 19:44:47.990748 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:47.990702    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:44:47.990958 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:47.990938    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 27 19:44:47.991337 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:47.991322    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 27 19:44:47.991415 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:47.991350    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 27 19:44:47.991467 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:47.991419    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-wl6p8\""
Jul 27 19:44:47.991547 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:47.991535    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 27 19:44:48.013638 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:48.013608    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8689f9464-g8plw"]
Jul 27 19:44:48.104163 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:48.104134    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e5c5f39c-feaa-4254-b244-5defedc33034-manager-config\") pod \"lws-controller-manager-8689f9464-g8plw\" (UID: \"e5c5f39c-feaa-4254-b244-5defedc33034\") " pod="openshift-lws-operator/lws-controller-manager-8689f9464-g8plw"
Jul 27 19:44:48.104163 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:48.104162    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e5c5f39c-feaa-4254-b244-5defedc33034-metrics-cert\") pod \"lws-controller-manager-8689f9464-g8plw\" (UID: \"e5c5f39c-feaa-4254-b244-5defedc33034\") " pod="openshift-lws-operator/lws-controller-manager-8689f9464-g8plw"
Jul 27 19:44:48.104348 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:48.104189    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5c5f39c-feaa-4254-b244-5defedc33034-cert\") pod \"lws-controller-manager-8689f9464-g8plw\" (UID: \"e5c5f39c-feaa-4254-b244-5defedc33034\") " pod="openshift-lws-operator/lws-controller-manager-8689f9464-g8plw"
Jul 27 19:44:48.104348 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:48.104278    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz4cg\" (UniqueName: \"kubernetes.io/projected/e5c5f39c-feaa-4254-b244-5defedc33034-kube-api-access-vz4cg\") pod \"lws-controller-manager-8689f9464-g8plw\" (UID: \"e5c5f39c-feaa-4254-b244-5defedc33034\") " pod="openshift-lws-operator/lws-controller-manager-8689f9464-g8plw"
Jul 27 19:44:48.204718 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:48.204683    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz4cg\" (UniqueName: \"kubernetes.io/projected/e5c5f39c-feaa-4254-b244-5defedc33034-kube-api-access-vz4cg\") pod \"lws-controller-manager-8689f9464-g8plw\" (UID: \"e5c5f39c-feaa-4254-b244-5defedc33034\") " pod="openshift-lws-operator/lws-controller-manager-8689f9464-g8plw"
Jul 27 19:44:48.204837 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:48.204732    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e5c5f39c-feaa-4254-b244-5defedc33034-manager-config\") pod \"lws-controller-manager-8689f9464-g8plw\" (UID: \"e5c5f39c-feaa-4254-b244-5defedc33034\") " pod="openshift-lws-operator/lws-controller-manager-8689f9464-g8plw"
Jul 27 19:44:48.204837 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:48.204746    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e5c5f39c-feaa-4254-b244-5defedc33034-metrics-cert\") pod \"lws-controller-manager-8689f9464-g8plw\" (UID: \"e5c5f39c-feaa-4254-b244-5defedc33034\") " pod="openshift-lws-operator/lws-controller-manager-8689f9464-g8plw"
Jul 27 19:44:48.204837 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:48.204778    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5c5f39c-feaa-4254-b244-5defedc33034-cert\") pod \"lws-controller-manager-8689f9464-g8plw\" (UID: \"e5c5f39c-feaa-4254-b244-5defedc33034\") " pod="openshift-lws-operator/lws-controller-manager-8689f9464-g8plw"
Jul 27 19:44:48.205328 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:48.205308    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e5c5f39c-feaa-4254-b244-5defedc33034-manager-config\") pod \"lws-controller-manager-8689f9464-g8plw\" (UID: \"e5c5f39c-feaa-4254-b244-5defedc33034\") " pod="openshift-lws-operator/lws-controller-manager-8689f9464-g8plw"
Jul 27 19:44:48.207397 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:48.207377    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e5c5f39c-feaa-4254-b244-5defedc33034-metrics-cert\") pod \"lws-controller-manager-8689f9464-g8plw\" (UID: \"e5c5f39c-feaa-4254-b244-5defedc33034\") " pod="openshift-lws-operator/lws-controller-manager-8689f9464-g8plw"
Jul 27 19:44:48.207471 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:48.207406    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5c5f39c-feaa-4254-b244-5defedc33034-cert\") pod \"lws-controller-manager-8689f9464-g8plw\" (UID: \"e5c5f39c-feaa-4254-b244-5defedc33034\") " pod="openshift-lws-operator/lws-controller-manager-8689f9464-g8plw"
Jul 27 19:44:48.213265 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:48.213244    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz4cg\" (UniqueName: \"kubernetes.io/projected/e5c5f39c-feaa-4254-b244-5defedc33034-kube-api-access-vz4cg\") pod \"lws-controller-manager-8689f9464-g8plw\" (UID: \"e5c5f39c-feaa-4254-b244-5defedc33034\") " pod="openshift-lws-operator/lws-controller-manager-8689f9464-g8plw"
Jul 27 19:44:48.299554 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:48.299507    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8689f9464-g8plw"
Jul 27 19:44:48.441850 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:48.441828    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8689f9464-g8plw"]
Jul 27 19:44:48.444143 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:44:48.444120    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5c5f39c_feaa_4254_b244_5defedc33034.slice/crio-d780f5ef9b9937b6b8dc595e3454e1757ac1611f0e9e69b45f0606d5d9cd36c4 WatchSource:0}: Error finding container d780f5ef9b9937b6b8dc595e3454e1757ac1611f0e9e69b45f0606d5d9cd36c4: Status 404 returned error can't find the container with id d780f5ef9b9937b6b8dc595e3454e1757ac1611f0e9e69b45f0606d5d9cd36c4
Jul 27 19:44:48.729897 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:48.729845    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8689f9464-g8plw" event={"ID":"e5c5f39c-feaa-4254-b244-5defedc33034","Type":"ContainerStarted","Data":"d780f5ef9b9937b6b8dc595e3454e1757ac1611f0e9e69b45f0606d5d9cd36c4"}
Jul 27 19:44:51.751172 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:51.751140    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8689f9464-g8plw" event={"ID":"e5c5f39c-feaa-4254-b244-5defedc33034","Type":"ContainerStarted","Data":"e6d9604e527ff1f4f442fc0047e7aa4903a5f4028b6d127c6f1b520efab0690f"}
Jul 27 19:44:51.751529 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:51.751280    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8689f9464-g8plw"
Jul 27 19:44:51.770388 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:44:51.770347    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8689f9464-g8plw" podStartSLOduration=2.324102575 podStartE2EDuration="4.770333086s" totalImagesPullingTime="2.446230511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:44:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:44:51.76896233 +0000 UTC m=+319.998204529" watchObservedRunningTime="2026-07-27 19:44:51.770333086 +0000 UTC m=+319.999575284"
Jul 27 19:45:02.757924 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:45:02.757891    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8689f9464-g8plw"
Jul 27 19:47:15.963318 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:15.963275    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-ktgrn"]
Jul 27 19:47:15.966534 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:15.966514    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ktgrn"
Jul 27 19:47:15.968514 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:15.968496    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 27 19:47:15.968921 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:15.968897    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-shc69\""
Jul 27 19:47:15.969025 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:15.968935    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 27 19:47:15.971005 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:15.970987    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-ktgrn"]
Jul 27 19:47:16.066131 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:16.066109    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-bx6gs"]
Jul 27 19:47:16.069183 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:16.069165    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bx6gs"
Jul 27 19:47:16.076763 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:16.076744    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bx6gs"]
Jul 27 19:47:16.080400 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:16.080380    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcgsp\" (UniqueName: \"kubernetes.io/projected/a9c36c78-ce25-45c8-adf2-d13b894892e5-kube-api-access-dcgsp\") pod \"authorino-58cd955675-ktgrn\" (UID: \"a9c36c78-ce25-45c8-adf2-d13b894892e5\") " pod="kuadrant-system/authorino-58cd955675-ktgrn"
Jul 27 19:47:16.181279 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:16.181240    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkc4k\" (UniqueName: \"kubernetes.io/projected/526f507e-57ec-4c71-926a-217b99e6d6fa-kube-api-access-fkc4k\") pod \"authorino-75d556bd77-bx6gs\" (UID: \"526f507e-57ec-4c71-926a-217b99e6d6fa\") " pod="kuadrant-system/authorino-75d556bd77-bx6gs"
Jul 27 19:47:16.181388 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:16.181297    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcgsp\" (UniqueName: \"kubernetes.io/projected/a9c36c78-ce25-45c8-adf2-d13b894892e5-kube-api-access-dcgsp\") pod \"authorino-58cd955675-ktgrn\" (UID: \"a9c36c78-ce25-45c8-adf2-d13b894892e5\") " pod="kuadrant-system/authorino-58cd955675-ktgrn"
Jul 27 19:47:16.189698 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:16.189674    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcgsp\" (UniqueName: \"kubernetes.io/projected/a9c36c78-ce25-45c8-adf2-d13b894892e5-kube-api-access-dcgsp\") pod \"authorino-58cd955675-ktgrn\" (UID: \"a9c36c78-ce25-45c8-adf2-d13b894892e5\") " pod="kuadrant-system/authorino-58cd955675-ktgrn"
Jul 27 19:47:16.275969 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:16.275916    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ktgrn"
Jul 27 19:47:16.281777 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:16.281757    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkc4k\" (UniqueName: \"kubernetes.io/projected/526f507e-57ec-4c71-926a-217b99e6d6fa-kube-api-access-fkc4k\") pod \"authorino-75d556bd77-bx6gs\" (UID: \"526f507e-57ec-4c71-926a-217b99e6d6fa\") " pod="kuadrant-system/authorino-75d556bd77-bx6gs"
Jul 27 19:47:16.289652 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:16.289623    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkc4k\" (UniqueName: \"kubernetes.io/projected/526f507e-57ec-4c71-926a-217b99e6d6fa-kube-api-access-fkc4k\") pod \"authorino-75d556bd77-bx6gs\" (UID: \"526f507e-57ec-4c71-926a-217b99e6d6fa\") " pod="kuadrant-system/authorino-75d556bd77-bx6gs"
Jul 27 19:47:16.377577 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:16.377547    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bx6gs"
Jul 27 19:47:16.404621 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:16.404590    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-ktgrn"]
Jul 27 19:47:16.407972 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:47:16.407934    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9c36c78_ce25_45c8_adf2_d13b894892e5.slice/crio-1aee76f53e95aa5bae37892199b2c426c50c1fc7a2418fdb62fb97060bccacb7 WatchSource:0}: Error finding container 1aee76f53e95aa5bae37892199b2c426c50c1fc7a2418fdb62fb97060bccacb7: Status 404 returned error can't find the container with id 1aee76f53e95aa5bae37892199b2c426c50c1fc7a2418fdb62fb97060bccacb7
Jul 27 19:47:16.505062 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:16.505036    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bx6gs"]
Jul 27 19:47:16.508631 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:47:16.508608    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod526f507e_57ec_4c71_926a_217b99e6d6fa.slice/crio-066b585f1f157da4a544bf5bdb8f5de9df2a8560acda1aca790af40fe211fe56 WatchSource:0}: Error finding container 066b585f1f157da4a544bf5bdb8f5de9df2a8560acda1aca790af40fe211fe56: Status 404 returned error can't find the container with id 066b585f1f157da4a544bf5bdb8f5de9df2a8560acda1aca790af40fe211fe56
Jul 27 19:47:17.347123 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:17.347022    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ktgrn" event={"ID":"a9c36c78-ce25-45c8-adf2-d13b894892e5","Type":"ContainerStarted","Data":"1aee76f53e95aa5bae37892199b2c426c50c1fc7a2418fdb62fb97060bccacb7"}
Jul 27 19:47:17.348587 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:17.348514    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bx6gs" event={"ID":"526f507e-57ec-4c71-926a-217b99e6d6fa","Type":"ContainerStarted","Data":"066b585f1f157da4a544bf5bdb8f5de9df2a8560acda1aca790af40fe211fe56"}
Jul 27 19:47:20.363392 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:20.363354    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bx6gs" event={"ID":"526f507e-57ec-4c71-926a-217b99e6d6fa","Type":"ContainerStarted","Data":"4e9658cd3a1672b1a9b7a1b0c7b60ef1c168e54cfb130b1c03508d01b393c133"}
Jul 27 19:47:20.364930 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:20.364899    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ktgrn" event={"ID":"a9c36c78-ce25-45c8-adf2-d13b894892e5","Type":"ContainerStarted","Data":"dfbbbd32e0e674f14182cfb9e3f950f0949771333702bee4e56c75ffd8848fad"}
Jul 27 19:47:20.379368 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:20.379318    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-bx6gs" podStartSLOduration=1.030045318 podStartE2EDuration="4.379299776s" totalImagesPullingTime="3.349254458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:47:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:47:20.378724238 +0000 UTC m=+468.607966437" watchObservedRunningTime="2026-07-27 19:47:20.379299776 +0000 UTC m=+468.608541975"
Jul 27 19:47:20.394482 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:20.394430    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-ktgrn" podStartSLOduration=1.9545011410000002 podStartE2EDuration="5.394416392s" totalImagesPullingTime="3.439915251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:47:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:47:20.392952315 +0000 UTC m=+468.622194527" watchObservedRunningTime="2026-07-27 19:47:20.394416392 +0000 UTC m=+468.623658592"
Jul 27 19:47:20.418101 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:20.418079    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-ktgrn"]
Jul 27 19:47:22.372916 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:22.372880    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-ktgrn" podUID="a9c36c78-ce25-45c8-adf2-d13b894892e5" containerName="authorino" containerID="cri-o://dfbbbd32e0e674f14182cfb9e3f950f0949771333702bee4e56c75ffd8848fad" gracePeriod=30
Jul 27 19:47:22.610119 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:22.610093    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ktgrn"
Jul 27 19:47:22.732462 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:22.732435    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcgsp\" (UniqueName: \"kubernetes.io/projected/a9c36c78-ce25-45c8-adf2-d13b894892e5-kube-api-access-dcgsp\") pod \"a9c36c78-ce25-45c8-adf2-d13b894892e5\" (UID: \"a9c36c78-ce25-45c8-adf2-d13b894892e5\") "
Jul 27 19:47:22.734660 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:22.734638    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9c36c78-ce25-45c8-adf2-d13b894892e5-kube-api-access-dcgsp" (OuterVolumeSpecName: "kube-api-access-dcgsp") pod "a9c36c78-ce25-45c8-adf2-d13b894892e5" (UID: "a9c36c78-ce25-45c8-adf2-d13b894892e5"). InnerVolumeSpecName "kube-api-access-dcgsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:47:22.833372 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:22.833340    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcgsp\" (UniqueName: \"kubernetes.io/projected/a9c36c78-ce25-45c8-adf2-d13b894892e5-kube-api-access-dcgsp\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 19:47:23.379387 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:23.379353    2579 generic.go:356] "Generic (PLEG): container finished" podID="a9c36c78-ce25-45c8-adf2-d13b894892e5" containerID="dfbbbd32e0e674f14182cfb9e3f950f0949771333702bee4e56c75ffd8848fad" exitCode=0
Jul 27 19:47:23.379887 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:23.379397    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ktgrn"
Jul 27 19:47:23.379887 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:23.379429    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ktgrn" event={"ID":"a9c36c78-ce25-45c8-adf2-d13b894892e5","Type":"ContainerDied","Data":"dfbbbd32e0e674f14182cfb9e3f950f0949771333702bee4e56c75ffd8848fad"}
Jul 27 19:47:23.379887 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:23.379458    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ktgrn" event={"ID":"a9c36c78-ce25-45c8-adf2-d13b894892e5","Type":"ContainerDied","Data":"1aee76f53e95aa5bae37892199b2c426c50c1fc7a2418fdb62fb97060bccacb7"}
Jul 27 19:47:23.379887 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:23.379474    2579 scope.go:117] "RemoveContainer" containerID="dfbbbd32e0e674f14182cfb9e3f950f0949771333702bee4e56c75ffd8848fad"
Jul 27 19:47:23.388195 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:23.388178    2579 scope.go:117] "RemoveContainer" containerID="dfbbbd32e0e674f14182cfb9e3f950f0949771333702bee4e56c75ffd8848fad"
Jul 27 19:47:23.388440 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:47:23.388421    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfbbbd32e0e674f14182cfb9e3f950f0949771333702bee4e56c75ffd8848fad\": container with ID starting with dfbbbd32e0e674f14182cfb9e3f950f0949771333702bee4e56c75ffd8848fad not found: ID does not exist" containerID="dfbbbd32e0e674f14182cfb9e3f950f0949771333702bee4e56c75ffd8848fad"
Jul 27 19:47:23.388484 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:23.388450    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfbbbd32e0e674f14182cfb9e3f950f0949771333702bee4e56c75ffd8848fad"} err="failed to get container status \"dfbbbd32e0e674f14182cfb9e3f950f0949771333702bee4e56c75ffd8848fad\": rpc error: code = NotFound desc = could not find container \"dfbbbd32e0e674f14182cfb9e3f950f0949771333702bee4e56c75ffd8848fad\": container with ID starting with dfbbbd32e0e674f14182cfb9e3f950f0949771333702bee4e56c75ffd8848fad not found: ID does not exist"
Jul 27 19:47:23.403004 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:23.402983    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-ktgrn"]
Jul 27 19:47:23.408807 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:23.408788    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-ktgrn"]
Jul 27 19:47:24.261938 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:24.261898    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9c36c78-ce25-45c8-adf2-d13b894892e5" path="/var/lib/kubelet/pods/a9c36c78-ce25-45c8-adf2-d13b894892e5/volumes"
Jul 27 19:47:48.728140 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:48.728106    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fh9kw"]
Jul 27 19:47:48.728735 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:48.728382    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9c36c78-ce25-45c8-adf2-d13b894892e5" containerName="authorino"
Jul 27 19:47:48.728735 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:48.728393    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9c36c78-ce25-45c8-adf2-d13b894892e5" containerName="authorino"
Jul 27 19:47:48.728735 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:48.728463    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9c36c78-ce25-45c8-adf2-d13b894892e5" containerName="authorino"
Jul 27 19:47:48.736084 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:48.736059    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fh9kw"
Jul 27 19:47:48.741913 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:48.741890    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fh9kw"]
Jul 27 19:47:48.828880 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:48.828826    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j2bv\" (UniqueName: \"kubernetes.io/projected/1ce9c823-e054-4a01-bee2-06cd4a6e75e7-kube-api-access-7j2bv\") pod \"authorino-5d4b67c446-fh9kw\" (UID: \"1ce9c823-e054-4a01-bee2-06cd4a6e75e7\") " pod="kuadrant-system/authorino-5d4b67c446-fh9kw"
Jul 27 19:47:48.929697 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:48.929660    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j2bv\" (UniqueName: \"kubernetes.io/projected/1ce9c823-e054-4a01-bee2-06cd4a6e75e7-kube-api-access-7j2bv\") pod \"authorino-5d4b67c446-fh9kw\" (UID: \"1ce9c823-e054-4a01-bee2-06cd4a6e75e7\") " pod="kuadrant-system/authorino-5d4b67c446-fh9kw"
Jul 27 19:47:48.939952 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:48.939921    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j2bv\" (UniqueName: \"kubernetes.io/projected/1ce9c823-e054-4a01-bee2-06cd4a6e75e7-kube-api-access-7j2bv\") pod \"authorino-5d4b67c446-fh9kw\" (UID: \"1ce9c823-e054-4a01-bee2-06cd4a6e75e7\") " pod="kuadrant-system/authorino-5d4b67c446-fh9kw"
Jul 27 19:47:48.940612 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:48.940592    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fh9kw"]
Jul 27 19:47:48.940801 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:48.940791    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fh9kw"
Jul 27 19:47:48.961090 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:48.961067    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6bddf87cdd-zrqts"]
Jul 27 19:47:48.965136 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:48.965118    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bddf87cdd-zrqts"
Jul 27 19:47:48.971149 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:48.971126    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6bddf87cdd-zrqts"]
Jul 27 19:47:49.030350 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:49.030321    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2gpn\" (UniqueName: \"kubernetes.io/projected/93ec60e4-b09b-417b-8518-62d545bd7081-kube-api-access-h2gpn\") pod \"authorino-6bddf87cdd-zrqts\" (UID: \"93ec60e4-b09b-417b-8518-62d545bd7081\") " pod="kuadrant-system/authorino-6bddf87cdd-zrqts"
Jul 27 19:47:49.070906 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:49.070877    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fh9kw"]
Jul 27 19:47:49.074358 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:47:49.074326    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ce9c823_e054_4a01_bee2_06cd4a6e75e7.slice/crio-e22730aecfea9c51688bf72551f664cbc743302606eb393a2ba55a3d09570b9d WatchSource:0}: Error finding container e22730aecfea9c51688bf72551f664cbc743302606eb393a2ba55a3d09570b9d: Status 404 returned error can't find the container with id e22730aecfea9c51688bf72551f664cbc743302606eb393a2ba55a3d09570b9d
Jul 27 19:47:49.131533 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:49.131502    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2gpn\" (UniqueName: \"kubernetes.io/projected/93ec60e4-b09b-417b-8518-62d545bd7081-kube-api-access-h2gpn\") pod \"authorino-6bddf87cdd-zrqts\" (UID: \"93ec60e4-b09b-417b-8518-62d545bd7081\") " pod="kuadrant-system/authorino-6bddf87cdd-zrqts"
Jul 27 19:47:49.140387 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:49.140366    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2gpn\" (UniqueName: \"kubernetes.io/projected/93ec60e4-b09b-417b-8518-62d545bd7081-kube-api-access-h2gpn\") pod \"authorino-6bddf87cdd-zrqts\" (UID: \"93ec60e4-b09b-417b-8518-62d545bd7081\") " pod="kuadrant-system/authorino-6bddf87cdd-zrqts"
Jul 27 19:47:49.216778 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:49.216744    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6bddf87cdd-zrqts"]
Jul 27 19:47:49.217013 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:49.217000    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bddf87cdd-zrqts"
Jul 27 19:47:49.244985 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:49.244924    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-558f77558d-m2wqh"]
Jul 27 19:47:49.248053 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:49.248038    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-558f77558d-m2wqh"
Jul 27 19:47:49.250060 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:49.250042    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 27 19:47:49.257973 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:49.257936    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-558f77558d-m2wqh"]
Jul 27 19:47:49.333506 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:49.333476    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b8d1c8ea-230b-4e83-af4c-c5c22393c5ca-tls-cert\") pod \"authorino-558f77558d-m2wqh\" (UID: \"b8d1c8ea-230b-4e83-af4c-c5c22393c5ca\") " pod="kuadrant-system/authorino-558f77558d-m2wqh"
Jul 27 19:47:49.333646 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:49.333580    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpdxf\" (UniqueName: \"kubernetes.io/projected/b8d1c8ea-230b-4e83-af4c-c5c22393c5ca-kube-api-access-xpdxf\") pod \"authorino-558f77558d-m2wqh\" (UID: \"b8d1c8ea-230b-4e83-af4c-c5c22393c5ca\") " pod="kuadrant-system/authorino-558f77558d-m2wqh"
Jul 27 19:47:49.347915 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:49.347878    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6bddf87cdd-zrqts"]
Jul 27 19:47:49.350508 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:47:49.350480    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93ec60e4_b09b_417b_8518_62d545bd7081.slice/crio-5f5a4969578789c0313e7c1e9391051f864c16fe8a41d80981bdf1262c448169 WatchSource:0}: Error finding container 5f5a4969578789c0313e7c1e9391051f864c16fe8a41d80981bdf1262c448169: Status 404 returned error can't find the container with id 5f5a4969578789c0313e7c1e9391051f864c16fe8a41d80981bdf1262c448169
Jul 27 19:47:49.434333 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:49.434299    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpdxf\" (UniqueName: \"kubernetes.io/projected/b8d1c8ea-230b-4e83-af4c-c5c22393c5ca-kube-api-access-xpdxf\") pod \"authorino-558f77558d-m2wqh\" (UID: \"b8d1c8ea-230b-4e83-af4c-c5c22393c5ca\") " pod="kuadrant-system/authorino-558f77558d-m2wqh"
Jul 27 19:47:49.434501 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:49.434362    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b8d1c8ea-230b-4e83-af4c-c5c22393c5ca-tls-cert\") pod \"authorino-558f77558d-m2wqh\" (UID: \"b8d1c8ea-230b-4e83-af4c-c5c22393c5ca\") " pod="kuadrant-system/authorino-558f77558d-m2wqh"
Jul 27 19:47:49.436913 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:49.436891    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b8d1c8ea-230b-4e83-af4c-c5c22393c5ca-tls-cert\") pod \"authorino-558f77558d-m2wqh\" (UID: \"b8d1c8ea-230b-4e83-af4c-c5c22393c5ca\") " pod="kuadrant-system/authorino-558f77558d-m2wqh"
Jul 27 19:47:49.442129 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:49.442104    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpdxf\" (UniqueName: \"kubernetes.io/projected/b8d1c8ea-230b-4e83-af4c-c5c22393c5ca-kube-api-access-xpdxf\") pod \"authorino-558f77558d-m2wqh\" (UID: \"b8d1c8ea-230b-4e83-af4c-c5c22393c5ca\") " pod="kuadrant-system/authorino-558f77558d-m2wqh"
Jul 27 19:47:49.495785 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:49.495712    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bddf87cdd-zrqts" event={"ID":"93ec60e4-b09b-417b-8518-62d545bd7081","Type":"ContainerStarted","Data":"5f5a4969578789c0313e7c1e9391051f864c16fe8a41d80981bdf1262c448169"}
Jul 27 19:47:49.496939 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:49.496917    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fh9kw" event={"ID":"1ce9c823-e054-4a01-bee2-06cd4a6e75e7","Type":"ContainerStarted","Data":"e22730aecfea9c51688bf72551f664cbc743302606eb393a2ba55a3d09570b9d"}
Jul 27 19:47:49.558253 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:49.558215    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-558f77558d-m2wqh"
Jul 27 19:47:49.688968 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:49.688936    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-558f77558d-m2wqh"]
Jul 27 19:47:49.692832 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:47:49.692808    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8d1c8ea_230b_4e83_af4c_c5c22393c5ca.slice/crio-708afc4551362c890957e7d2348b562ef8a1f7d6236788e5b8d97dbbcf2f8175 WatchSource:0}: Error finding container 708afc4551362c890957e7d2348b562ef8a1f7d6236788e5b8d97dbbcf2f8175: Status 404 returned error can't find the container with id 708afc4551362c890957e7d2348b562ef8a1f7d6236788e5b8d97dbbcf2f8175
Jul 27 19:47:50.502371 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:50.502336    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bddf87cdd-zrqts" event={"ID":"93ec60e4-b09b-417b-8518-62d545bd7081","Type":"ContainerStarted","Data":"fd85cd5302d7c4eeb8e0f7c839df96b10110e5d34102baa7c51d1492a0a2822a"}
Jul 27 19:47:50.502781 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:50.502380    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6bddf87cdd-zrqts" podUID="93ec60e4-b09b-417b-8518-62d545bd7081" containerName="authorino" containerID="cri-o://fd85cd5302d7c4eeb8e0f7c839df96b10110e5d34102baa7c51d1492a0a2822a" gracePeriod=30
Jul 27 19:47:50.504074 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:50.504049    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-fh9kw" podUID="1ce9c823-e054-4a01-bee2-06cd4a6e75e7" containerName="authorino" containerID="cri-o://7b508de7c7e195dd51b9538ad98376b3971b0710330481cdc6213efb15b6badf" gracePeriod=30
Jul 27 19:47:50.504232 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:50.504048    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fh9kw" event={"ID":"1ce9c823-e054-4a01-bee2-06cd4a6e75e7","Type":"ContainerStarted","Data":"7b508de7c7e195dd51b9538ad98376b3971b0710330481cdc6213efb15b6badf"}
Jul 27 19:47:50.505751 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:50.505663    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-558f77558d-m2wqh" event={"ID":"b8d1c8ea-230b-4e83-af4c-c5c22393c5ca","Type":"ContainerStarted","Data":"981081a98fe30b8c1c9f6e12fad892d9322d20b5858b1eab971b7a3238db7a6c"}
Jul 27 19:47:50.505751 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:50.505699    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-558f77558d-m2wqh" event={"ID":"b8d1c8ea-230b-4e83-af4c-c5c22393c5ca","Type":"ContainerStarted","Data":"708afc4551362c890957e7d2348b562ef8a1f7d6236788e5b8d97dbbcf2f8175"}
Jul 27 19:47:50.517389 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:50.517346    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6bddf87cdd-zrqts" podStartSLOduration=2.0716968749999998 podStartE2EDuration="2.517329964s" totalImagesPullingTime="445.633089ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:47:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:47:50.516086815 +0000 UTC m=+498.745329013" watchObservedRunningTime="2026-07-27 19:47:50.517329964 +0000 UTC m=+498.746572140"
Jul 27 19:47:50.530988 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:50.530904    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-558f77558d-m2wqh" podStartSLOduration=1.140371929 podStartE2EDuration="1.53088814s" totalImagesPullingTime="390.516211ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:47:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:47:50.530401217 +0000 UTC m=+498.759643416" watchObservedRunningTime="2026-07-27 19:47:50.53088814 +0000 UTC m=+498.760130339"
Jul 27 19:47:50.546305 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:50.546263    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-fh9kw" podStartSLOduration=1.7347808630000001 podStartE2EDuration="2.54624516s" totalImagesPullingTime="811.464297ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:47:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:47:50.544635 +0000 UTC m=+498.773877199" watchObservedRunningTime="2026-07-27 19:47:50.54624516 +0000 UTC m=+498.775487359"
Jul 27 19:47:50.566965 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:50.566936    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bx6gs"]
Jul 27 19:47:50.567165 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:50.567141    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-bx6gs" podUID="526f507e-57ec-4c71-926a-217b99e6d6fa" containerName="authorino" containerID="cri-o://4e9658cd3a1672b1a9b7a1b0c7b60ef1c168e54cfb130b1c03508d01b393c133" gracePeriod=30
Jul 27 19:47:50.808798 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:50.808775    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bddf87cdd-zrqts"
Jul 27 19:47:50.843718 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:50.843687    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fh9kw"
Jul 27 19:47:50.844536 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:50.844506    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2gpn\" (UniqueName: \"kubernetes.io/projected/93ec60e4-b09b-417b-8518-62d545bd7081-kube-api-access-h2gpn\") pod \"93ec60e4-b09b-417b-8518-62d545bd7081\" (UID: \"93ec60e4-b09b-417b-8518-62d545bd7081\") "
Jul 27 19:47:50.846673 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:50.846646    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93ec60e4-b09b-417b-8518-62d545bd7081-kube-api-access-h2gpn" (OuterVolumeSpecName: "kube-api-access-h2gpn") pod "93ec60e4-b09b-417b-8518-62d545bd7081" (UID: "93ec60e4-b09b-417b-8518-62d545bd7081"). InnerVolumeSpecName "kube-api-access-h2gpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:47:50.847792 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:50.847776    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bx6gs"
Jul 27 19:47:50.945976 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:50.945942    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkc4k\" (UniqueName: \"kubernetes.io/projected/526f507e-57ec-4c71-926a-217b99e6d6fa-kube-api-access-fkc4k\") pod \"526f507e-57ec-4c71-926a-217b99e6d6fa\" (UID: \"526f507e-57ec-4c71-926a-217b99e6d6fa\") "
Jul 27 19:47:50.946146 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:50.945999    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7j2bv\" (UniqueName: \"kubernetes.io/projected/1ce9c823-e054-4a01-bee2-06cd4a6e75e7-kube-api-access-7j2bv\") pod \"1ce9c823-e054-4a01-bee2-06cd4a6e75e7\" (UID: \"1ce9c823-e054-4a01-bee2-06cd4a6e75e7\") "
Jul 27 19:47:50.946222 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:50.946151    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2gpn\" (UniqueName: \"kubernetes.io/projected/93ec60e4-b09b-417b-8518-62d545bd7081-kube-api-access-h2gpn\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 19:47:50.948199 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:50.948166    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/526f507e-57ec-4c71-926a-217b99e6d6fa-kube-api-access-fkc4k" (OuterVolumeSpecName: "kube-api-access-fkc4k") pod "526f507e-57ec-4c71-926a-217b99e6d6fa" (UID: "526f507e-57ec-4c71-926a-217b99e6d6fa"). InnerVolumeSpecName "kube-api-access-fkc4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:47:50.948199 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:50.948177    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ce9c823-e054-4a01-bee2-06cd4a6e75e7-kube-api-access-7j2bv" (OuterVolumeSpecName: "kube-api-access-7j2bv") pod "1ce9c823-e054-4a01-bee2-06cd4a6e75e7" (UID: "1ce9c823-e054-4a01-bee2-06cd4a6e75e7"). InnerVolumeSpecName "kube-api-access-7j2bv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:47:51.046535 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.046463    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkc4k\" (UniqueName: \"kubernetes.io/projected/526f507e-57ec-4c71-926a-217b99e6d6fa-kube-api-access-fkc4k\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 19:47:51.046535 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.046489    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7j2bv\" (UniqueName: \"kubernetes.io/projected/1ce9c823-e054-4a01-bee2-06cd4a6e75e7-kube-api-access-7j2bv\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 19:47:51.510951 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.510912    2579 generic.go:356] "Generic (PLEG): container finished" podID="526f507e-57ec-4c71-926a-217b99e6d6fa" containerID="4e9658cd3a1672b1a9b7a1b0c7b60ef1c168e54cfb130b1c03508d01b393c133" exitCode=0
Jul 27 19:47:51.510951 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.510950    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bx6gs"
Jul 27 19:47:51.511447 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.510997    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bx6gs" event={"ID":"526f507e-57ec-4c71-926a-217b99e6d6fa","Type":"ContainerDied","Data":"4e9658cd3a1672b1a9b7a1b0c7b60ef1c168e54cfb130b1c03508d01b393c133"}
Jul 27 19:47:51.511447 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.511043    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bx6gs" event={"ID":"526f507e-57ec-4c71-926a-217b99e6d6fa","Type":"ContainerDied","Data":"066b585f1f157da4a544bf5bdb8f5de9df2a8560acda1aca790af40fe211fe56"}
Jul 27 19:47:51.511447 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.511067    2579 scope.go:117] "RemoveContainer" containerID="4e9658cd3a1672b1a9b7a1b0c7b60ef1c168e54cfb130b1c03508d01b393c133"
Jul 27 19:47:51.512379 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.512356    2579 generic.go:356] "Generic (PLEG): container finished" podID="93ec60e4-b09b-417b-8518-62d545bd7081" containerID="fd85cd5302d7c4eeb8e0f7c839df96b10110e5d34102baa7c51d1492a0a2822a" exitCode=0
Jul 27 19:47:51.512461 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.512397    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bddf87cdd-zrqts"
Jul 27 19:47:51.512461 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.512439    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bddf87cdd-zrqts" event={"ID":"93ec60e4-b09b-417b-8518-62d545bd7081","Type":"ContainerDied","Data":"fd85cd5302d7c4eeb8e0f7c839df96b10110e5d34102baa7c51d1492a0a2822a"}
Jul 27 19:47:51.512565 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.512467    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bddf87cdd-zrqts" event={"ID":"93ec60e4-b09b-417b-8518-62d545bd7081","Type":"ContainerDied","Data":"5f5a4969578789c0313e7c1e9391051f864c16fe8a41d80981bdf1262c448169"}
Jul 27 19:47:51.514004 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.513982    2579 generic.go:356] "Generic (PLEG): container finished" podID="1ce9c823-e054-4a01-bee2-06cd4a6e75e7" containerID="7b508de7c7e195dd51b9538ad98376b3971b0710330481cdc6213efb15b6badf" exitCode=0
Jul 27 19:47:51.514130 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.514094    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fh9kw"
Jul 27 19:47:51.514130 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.514095    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fh9kw" event={"ID":"1ce9c823-e054-4a01-bee2-06cd4a6e75e7","Type":"ContainerDied","Data":"7b508de7c7e195dd51b9538ad98376b3971b0710330481cdc6213efb15b6badf"}
Jul 27 19:47:51.514220 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.514138    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fh9kw" event={"ID":"1ce9c823-e054-4a01-bee2-06cd4a6e75e7","Type":"ContainerDied","Data":"e22730aecfea9c51688bf72551f664cbc743302606eb393a2ba55a3d09570b9d"}
Jul 27 19:47:51.519833 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.519815    2579 scope.go:117] "RemoveContainer" containerID="4e9658cd3a1672b1a9b7a1b0c7b60ef1c168e54cfb130b1c03508d01b393c133"
Jul 27 19:47:51.520090 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:47:51.520070    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e9658cd3a1672b1a9b7a1b0c7b60ef1c168e54cfb130b1c03508d01b393c133\": container with ID starting with 4e9658cd3a1672b1a9b7a1b0c7b60ef1c168e54cfb130b1c03508d01b393c133 not found: ID does not exist" containerID="4e9658cd3a1672b1a9b7a1b0c7b60ef1c168e54cfb130b1c03508d01b393c133"
Jul 27 19:47:51.520167 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.520100    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e9658cd3a1672b1a9b7a1b0c7b60ef1c168e54cfb130b1c03508d01b393c133"} err="failed to get container status \"4e9658cd3a1672b1a9b7a1b0c7b60ef1c168e54cfb130b1c03508d01b393c133\": rpc error: code = NotFound desc = could not find container \"4e9658cd3a1672b1a9b7a1b0c7b60ef1c168e54cfb130b1c03508d01b393c133\": container with ID starting with 4e9658cd3a1672b1a9b7a1b0c7b60ef1c168e54cfb130b1c03508d01b393c133 not found: ID does not exist"
Jul 27 19:47:51.520167 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.520123    2579 scope.go:117] "RemoveContainer" containerID="fd85cd5302d7c4eeb8e0f7c839df96b10110e5d34102baa7c51d1492a0a2822a"
Jul 27 19:47:51.528344 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.528330    2579 scope.go:117] "RemoveContainer" containerID="fd85cd5302d7c4eeb8e0f7c839df96b10110e5d34102baa7c51d1492a0a2822a"
Jul 27 19:47:51.528582 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:47:51.528566    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd85cd5302d7c4eeb8e0f7c839df96b10110e5d34102baa7c51d1492a0a2822a\": container with ID starting with fd85cd5302d7c4eeb8e0f7c839df96b10110e5d34102baa7c51d1492a0a2822a not found: ID does not exist" containerID="fd85cd5302d7c4eeb8e0f7c839df96b10110e5d34102baa7c51d1492a0a2822a"
Jul 27 19:47:51.528623 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.528589    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd85cd5302d7c4eeb8e0f7c839df96b10110e5d34102baa7c51d1492a0a2822a"} err="failed to get container status \"fd85cd5302d7c4eeb8e0f7c839df96b10110e5d34102baa7c51d1492a0a2822a\": rpc error: code = NotFound desc = could not find container \"fd85cd5302d7c4eeb8e0f7c839df96b10110e5d34102baa7c51d1492a0a2822a\": container with ID starting with fd85cd5302d7c4eeb8e0f7c839df96b10110e5d34102baa7c51d1492a0a2822a not found: ID does not exist"
Jul 27 19:47:51.528623 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.528606    2579 scope.go:117] "RemoveContainer" containerID="7b508de7c7e195dd51b9538ad98376b3971b0710330481cdc6213efb15b6badf"
Jul 27 19:47:51.531357 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.531338    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bx6gs"]
Jul 27 19:47:51.535025 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.535003    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bx6gs"]
Jul 27 19:47:51.536425 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.536409    2579 scope.go:117] "RemoveContainer" containerID="7b508de7c7e195dd51b9538ad98376b3971b0710330481cdc6213efb15b6badf"
Jul 27 19:47:51.536677 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:47:51.536662    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b508de7c7e195dd51b9538ad98376b3971b0710330481cdc6213efb15b6badf\": container with ID starting with 7b508de7c7e195dd51b9538ad98376b3971b0710330481cdc6213efb15b6badf not found: ID does not exist" containerID="7b508de7c7e195dd51b9538ad98376b3971b0710330481cdc6213efb15b6badf"
Jul 27 19:47:51.536721 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.536684    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b508de7c7e195dd51b9538ad98376b3971b0710330481cdc6213efb15b6badf"} err="failed to get container status \"7b508de7c7e195dd51b9538ad98376b3971b0710330481cdc6213efb15b6badf\": rpc error: code = NotFound desc = could not find container \"7b508de7c7e195dd51b9538ad98376b3971b0710330481cdc6213efb15b6badf\": container with ID starting with 7b508de7c7e195dd51b9538ad98376b3971b0710330481cdc6213efb15b6badf not found: ID does not exist"
Jul 27 19:47:51.543202 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.543182    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fh9kw"]
Jul 27 19:47:51.545700 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.545682    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fh9kw"]
Jul 27 19:47:51.555490 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.555473    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6bddf87cdd-zrqts"]
Jul 27 19:47:51.559278 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:51.559260    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6bddf87cdd-zrqts"]
Jul 27 19:47:52.265451 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:52.263182    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ce9c823-e054-4a01-bee2-06cd4a6e75e7" path="/var/lib/kubelet/pods/1ce9c823-e054-4a01-bee2-06cd4a6e75e7/volumes"
Jul 27 19:47:52.265451 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:52.263683    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="526f507e-57ec-4c71-926a-217b99e6d6fa" path="/var/lib/kubelet/pods/526f507e-57ec-4c71-926a-217b99e6d6fa/volumes"
Jul 27 19:47:52.265451 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:47:52.264166    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93ec60e4-b09b-417b-8518-62d545bd7081" path="/var/lib/kubelet/pods/93ec60e4-b09b-417b-8518-62d545bd7081/volumes"
Jul 27 19:48:55.233946 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.233904    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"]
Jul 27 19:48:55.234371 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.234262    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93ec60e4-b09b-417b-8518-62d545bd7081" containerName="authorino"
Jul 27 19:48:55.234371 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.234279    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ec60e4-b09b-417b-8518-62d545bd7081" containerName="authorino"
Jul 27 19:48:55.234371 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.234290    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="526f507e-57ec-4c71-926a-217b99e6d6fa" containerName="authorino"
Jul 27 19:48:55.234371 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.234295    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="526f507e-57ec-4c71-926a-217b99e6d6fa" containerName="authorino"
Jul 27 19:48:55.234371 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.234311    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ce9c823-e054-4a01-bee2-06cd4a6e75e7" containerName="authorino"
Jul 27 19:48:55.234371 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.234316    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ce9c823-e054-4a01-bee2-06cd4a6e75e7" containerName="authorino"
Jul 27 19:48:55.234573 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.234396    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ce9c823-e054-4a01-bee2-06cd4a6e75e7" containerName="authorino"
Jul 27 19:48:55.234573 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.234407    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="526f507e-57ec-4c71-926a-217b99e6d6fa" containerName="authorino"
Jul 27 19:48:55.234573 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.234417    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="93ec60e4-b09b-417b-8518-62d545bd7081" containerName="authorino"
Jul 27 19:48:55.237504 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.237477    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:48:55.240131 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.239950    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 27 19:48:55.240131 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.239960    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 27 19:48:55.240131 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.239967    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 27 19:48:55.240131 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.239950    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pvx6j\""
Jul 27 19:48:55.248437 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.248405    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"]
Jul 27 19:48:55.336904 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.336875    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95f0ed57-2cfd-4942-b707-76b26883ea57-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9f2rd\" (UID: \"95f0ed57-2cfd-4942-b707-76b26883ea57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:48:55.336904 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.336906    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95f0ed57-2cfd-4942-b707-76b26883ea57-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9f2rd\" (UID: \"95f0ed57-2cfd-4942-b707-76b26883ea57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:48:55.337062 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.336953    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95f0ed57-2cfd-4942-b707-76b26883ea57-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9f2rd\" (UID: \"95f0ed57-2cfd-4942-b707-76b26883ea57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:48:55.337062 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.336973    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95f0ed57-2cfd-4942-b707-76b26883ea57-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9f2rd\" (UID: \"95f0ed57-2cfd-4942-b707-76b26883ea57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:48:55.337062 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.337004    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9qgx\" (UniqueName: \"kubernetes.io/projected/95f0ed57-2cfd-4942-b707-76b26883ea57-kube-api-access-c9qgx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9f2rd\" (UID: \"95f0ed57-2cfd-4942-b707-76b26883ea57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:48:55.337062 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.337025    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95f0ed57-2cfd-4942-b707-76b26883ea57-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9f2rd\" (UID: \"95f0ed57-2cfd-4942-b707-76b26883ea57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:48:55.437719 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.437688    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95f0ed57-2cfd-4942-b707-76b26883ea57-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9f2rd\" (UID: \"95f0ed57-2cfd-4942-b707-76b26883ea57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:48:55.437840 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.437744    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95f0ed57-2cfd-4942-b707-76b26883ea57-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9f2rd\" (UID: \"95f0ed57-2cfd-4942-b707-76b26883ea57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:48:55.437840 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.437759    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95f0ed57-2cfd-4942-b707-76b26883ea57-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9f2rd\" (UID: \"95f0ed57-2cfd-4942-b707-76b26883ea57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:48:55.437840 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.437784    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9qgx\" (UniqueName: \"kubernetes.io/projected/95f0ed57-2cfd-4942-b707-76b26883ea57-kube-api-access-c9qgx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9f2rd\" (UID: \"95f0ed57-2cfd-4942-b707-76b26883ea57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:48:55.437840 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.437805    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95f0ed57-2cfd-4942-b707-76b26883ea57-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9f2rd\" (UID: \"95f0ed57-2cfd-4942-b707-76b26883ea57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:48:55.437840 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.437823    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95f0ed57-2cfd-4942-b707-76b26883ea57-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9f2rd\" (UID: \"95f0ed57-2cfd-4942-b707-76b26883ea57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:48:55.438232 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.438209    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95f0ed57-2cfd-4942-b707-76b26883ea57-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9f2rd\" (UID: \"95f0ed57-2cfd-4942-b707-76b26883ea57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:48:55.438293 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.438235    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95f0ed57-2cfd-4942-b707-76b26883ea57-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9f2rd\" (UID: \"95f0ed57-2cfd-4942-b707-76b26883ea57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:48:55.438329 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.438289    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95f0ed57-2cfd-4942-b707-76b26883ea57-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9f2rd\" (UID: \"95f0ed57-2cfd-4942-b707-76b26883ea57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:48:55.440169 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.440145    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95f0ed57-2cfd-4942-b707-76b26883ea57-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9f2rd\" (UID: \"95f0ed57-2cfd-4942-b707-76b26883ea57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:48:55.440437 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.440411    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95f0ed57-2cfd-4942-b707-76b26883ea57-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9f2rd\" (UID: \"95f0ed57-2cfd-4942-b707-76b26883ea57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:48:55.445554 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.445534    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9qgx\" (UniqueName: \"kubernetes.io/projected/95f0ed57-2cfd-4942-b707-76b26883ea57-kube-api-access-c9qgx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9f2rd\" (UID: \"95f0ed57-2cfd-4942-b707-76b26883ea57\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:48:55.550535 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.550471    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:48:55.679840 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.679812    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"]
Jul 27 19:48:55.682681 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:48:55.682658    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95f0ed57_2cfd_4942_b707_76b26883ea57.slice/crio-9dee0c5ecbd3d5f5ee57b6fc472f5733b9b6a79349ff86f006deb4d2c95cfea0 WatchSource:0}: Error finding container 9dee0c5ecbd3d5f5ee57b6fc472f5733b9b6a79349ff86f006deb4d2c95cfea0: Status 404 returned error can't find the container with id 9dee0c5ecbd3d5f5ee57b6fc472f5733b9b6a79349ff86f006deb4d2c95cfea0
Jul 27 19:48:55.783192 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:55.783162    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd" event={"ID":"95f0ed57-2cfd-4942-b707-76b26883ea57","Type":"ContainerStarted","Data":"9dee0c5ecbd3d5f5ee57b6fc472f5733b9b6a79349ff86f006deb4d2c95cfea0"}
Jul 27 19:48:57.792245 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:57.792214    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd" event={"ID":"95f0ed57-2cfd-4942-b707-76b26883ea57","Type":"ContainerStarted","Data":"d675477ed512490cf570a7971d88741880ed789881083268c1f23ea413d22f1b"}
Jul 27 19:48:57.809610 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:48:57.809565    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd" podStartSLOduration=1.65715276 podStartE2EDuration="2.809551245s" totalImagesPullingTime="1.152398485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:48:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:48:57.808461455 +0000 UTC m=+566.037703655" watchObservedRunningTime="2026-07-27 19:48:57.809551245 +0000 UTC m=+566.038793442"
Jul 27 19:49:05.550756 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:05.550723    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:49:05.551266 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:05.550776    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:49:05.563553 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:05.563532    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:49:05.834970 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:05.834891    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9f2rd"
Jul 27 19:49:18.229669 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.229639    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"]
Jul 27 19:49:18.233105 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.233083    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:18.235185 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.235166    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 27 19:49:18.243699 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.243677    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"]
Jul 27 19:49:18.302410 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.302383    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g95z4\" (UniqueName: \"kubernetes.io/projected/f6b167f4-4de4-4c51-9415-d69b7129eb40-kube-api-access-g95z4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx\" (UID: \"f6b167f4-4de4-4c51-9415-d69b7129eb40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:18.302538 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.302426    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6b167f4-4de4-4c51-9415-d69b7129eb40-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx\" (UID: \"f6b167f4-4de4-4c51-9415-d69b7129eb40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:18.302538 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.302491    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6b167f4-4de4-4c51-9415-d69b7129eb40-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx\" (UID: \"f6b167f4-4de4-4c51-9415-d69b7129eb40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:18.302538 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.302531    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6b167f4-4de4-4c51-9415-d69b7129eb40-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx\" (UID: \"f6b167f4-4de4-4c51-9415-d69b7129eb40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:18.302668 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.302551    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6b167f4-4de4-4c51-9415-d69b7129eb40-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx\" (UID: \"f6b167f4-4de4-4c51-9415-d69b7129eb40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:18.302668 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.302567    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6b167f4-4de4-4c51-9415-d69b7129eb40-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx\" (UID: \"f6b167f4-4de4-4c51-9415-d69b7129eb40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:18.403624 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.403594    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6b167f4-4de4-4c51-9415-d69b7129eb40-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx\" (UID: \"f6b167f4-4de4-4c51-9415-d69b7129eb40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:18.403741 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.403628    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6b167f4-4de4-4c51-9415-d69b7129eb40-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx\" (UID: \"f6b167f4-4de4-4c51-9415-d69b7129eb40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:18.403741 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.403645    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6b167f4-4de4-4c51-9415-d69b7129eb40-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx\" (UID: \"f6b167f4-4de4-4c51-9415-d69b7129eb40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:18.403741 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.403672    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g95z4\" (UniqueName: \"kubernetes.io/projected/f6b167f4-4de4-4c51-9415-d69b7129eb40-kube-api-access-g95z4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx\" (UID: \"f6b167f4-4de4-4c51-9415-d69b7129eb40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:18.403932 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.403805    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6b167f4-4de4-4c51-9415-d69b7129eb40-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx\" (UID: \"f6b167f4-4de4-4c51-9415-d69b7129eb40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:18.403932 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.403916    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6b167f4-4de4-4c51-9415-d69b7129eb40-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx\" (UID: \"f6b167f4-4de4-4c51-9415-d69b7129eb40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:18.404102 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.404080    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6b167f4-4de4-4c51-9415-d69b7129eb40-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx\" (UID: \"f6b167f4-4de4-4c51-9415-d69b7129eb40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:18.404170 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.404142    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6b167f4-4de4-4c51-9415-d69b7129eb40-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx\" (UID: \"f6b167f4-4de4-4c51-9415-d69b7129eb40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:18.404238 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.404219    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6b167f4-4de4-4c51-9415-d69b7129eb40-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx\" (UID: \"f6b167f4-4de4-4c51-9415-d69b7129eb40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:18.405987 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.405963    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6b167f4-4de4-4c51-9415-d69b7129eb40-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx\" (UID: \"f6b167f4-4de4-4c51-9415-d69b7129eb40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:18.406291 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.406276    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6b167f4-4de4-4c51-9415-d69b7129eb40-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx\" (UID: \"f6b167f4-4de4-4c51-9415-d69b7129eb40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:18.413339 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.413314    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g95z4\" (UniqueName: \"kubernetes.io/projected/f6b167f4-4de4-4c51-9415-d69b7129eb40-kube-api-access-g95z4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx\" (UID: \"f6b167f4-4de4-4c51-9415-d69b7129eb40\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:18.543553 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.543496    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:18.676799 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.676771    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"]
Jul 27 19:49:18.679365 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:49:18.679337    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6b167f4_4de4_4c51_9415_d69b7129eb40.slice/crio-39fdad8c802ed2606fe7e6f6cab2e2983c35b38ef7935d138a497188ab6494e8 WatchSource:0}: Error finding container 39fdad8c802ed2606fe7e6f6cab2e2983c35b38ef7935d138a497188ab6494e8: Status 404 returned error can't find the container with id 39fdad8c802ed2606fe7e6f6cab2e2983c35b38ef7935d138a497188ab6494e8
Jul 27 19:49:18.882711 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:18.882686    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx" event={"ID":"f6b167f4-4de4-4c51-9415-d69b7129eb40","Type":"ContainerStarted","Data":"39fdad8c802ed2606fe7e6f6cab2e2983c35b38ef7935d138a497188ab6494e8"}
Jul 27 19:49:19.888614 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:19.888571    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx" event={"ID":"f6b167f4-4de4-4c51-9415-d69b7129eb40","Type":"ContainerStarted","Data":"7eda704d607e1faa4532bdeeac1826925eddf63b670d07c49be999d5aafeea74"}
Jul 27 19:49:19.912708 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:19.912659    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx" podStartSLOduration=1.7473328430000001 podStartE2EDuration="1.912644484s" totalImagesPullingTime="165.311641ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:49:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:49:19.910792604 +0000 UTC m=+588.140034803" watchObservedRunningTime="2026-07-27 19:49:19.912644484 +0000 UTC m=+588.141886682"
Jul 27 19:49:28.544302 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:28.544262    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:28.544689 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:28.544324    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:28.556423 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:28.556396    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:28.938897 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:28.938847    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx"
Jul 27 19:49:32.225308 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:32.225284    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 19:49:32.225709 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:32.225349    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 19:49:45.531521 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.531481    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"]
Jul 27 19:49:45.535026 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.535003    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:45.537609 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.537589    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 27 19:49:45.544473 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.544448    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"]
Jul 27 19:49:45.597650 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.597622    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfd4f\" (UniqueName: \"kubernetes.io/projected/b6dbec73-ac0a-4f69-9112-36f9b46658d0-kube-api-access-gfd4f\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5\" (UID: \"b6dbec73-ac0a-4f69-9112-36f9b46658d0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:45.597751 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.597677    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6dbec73-ac0a-4f69-9112-36f9b46658d0-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5\" (UID: \"b6dbec73-ac0a-4f69-9112-36f9b46658d0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:45.597751 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.597700    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6dbec73-ac0a-4f69-9112-36f9b46658d0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5\" (UID: \"b6dbec73-ac0a-4f69-9112-36f9b46658d0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:45.597823 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.597754    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6dbec73-ac0a-4f69-9112-36f9b46658d0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5\" (UID: \"b6dbec73-ac0a-4f69-9112-36f9b46658d0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:45.597823 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.597778    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6dbec73-ac0a-4f69-9112-36f9b46658d0-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5\" (UID: \"b6dbec73-ac0a-4f69-9112-36f9b46658d0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:45.597823 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.597798    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6dbec73-ac0a-4f69-9112-36f9b46658d0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5\" (UID: \"b6dbec73-ac0a-4f69-9112-36f9b46658d0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:45.699098 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.699075    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6dbec73-ac0a-4f69-9112-36f9b46658d0-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5\" (UID: \"b6dbec73-ac0a-4f69-9112-36f9b46658d0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:45.699206 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.699102    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6dbec73-ac0a-4f69-9112-36f9b46658d0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5\" (UID: \"b6dbec73-ac0a-4f69-9112-36f9b46658d0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:45.699206 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.699143    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6dbec73-ac0a-4f69-9112-36f9b46658d0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5\" (UID: \"b6dbec73-ac0a-4f69-9112-36f9b46658d0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:45.699206 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.699171    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6dbec73-ac0a-4f69-9112-36f9b46658d0-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5\" (UID: \"b6dbec73-ac0a-4f69-9112-36f9b46658d0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:45.699206 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.699196    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6dbec73-ac0a-4f69-9112-36f9b46658d0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5\" (UID: \"b6dbec73-ac0a-4f69-9112-36f9b46658d0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:45.699422 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.699228    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfd4f\" (UniqueName: \"kubernetes.io/projected/b6dbec73-ac0a-4f69-9112-36f9b46658d0-kube-api-access-gfd4f\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5\" (UID: \"b6dbec73-ac0a-4f69-9112-36f9b46658d0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:45.699607 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.699583    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6dbec73-ac0a-4f69-9112-36f9b46658d0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5\" (UID: \"b6dbec73-ac0a-4f69-9112-36f9b46658d0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:45.699607 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.699601    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6dbec73-ac0a-4f69-9112-36f9b46658d0-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5\" (UID: \"b6dbec73-ac0a-4f69-9112-36f9b46658d0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:45.700017 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.699612    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6dbec73-ac0a-4f69-9112-36f9b46658d0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5\" (UID: \"b6dbec73-ac0a-4f69-9112-36f9b46658d0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:45.701406 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.701380    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6dbec73-ac0a-4f69-9112-36f9b46658d0-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5\" (UID: \"b6dbec73-ac0a-4f69-9112-36f9b46658d0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:45.701734 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.701717    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6dbec73-ac0a-4f69-9112-36f9b46658d0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5\" (UID: \"b6dbec73-ac0a-4f69-9112-36f9b46658d0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:45.708427 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.708410    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfd4f\" (UniqueName: \"kubernetes.io/projected/b6dbec73-ac0a-4f69-9112-36f9b46658d0-kube-api-access-gfd4f\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5\" (UID: \"b6dbec73-ac0a-4f69-9112-36f9b46658d0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:45.845883 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.845808    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:45.976277 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.976250    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"]
Jul 27 19:49:45.979214 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:49:45.979184    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6dbec73_ac0a_4f69_9112_36f9b46658d0.slice/crio-c38fabb7afdcbd9805e938d9fc934209b8936fdd09c60af8905c97c26ea57b20 WatchSource:0}: Error finding container c38fabb7afdcbd9805e938d9fc934209b8936fdd09c60af8905c97c26ea57b20: Status 404 returned error can't find the container with id c38fabb7afdcbd9805e938d9fc934209b8936fdd09c60af8905c97c26ea57b20
Jul 27 19:49:45.980573 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.980556    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 19:49:45.998796 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:45.998774    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5" event={"ID":"b6dbec73-ac0a-4f69-9112-36f9b46658d0","Type":"ContainerStarted","Data":"c38fabb7afdcbd9805e938d9fc934209b8936fdd09c60af8905c97c26ea57b20"}
Jul 27 19:49:47.004212 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:47.004174    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5" event={"ID":"b6dbec73-ac0a-4f69-9112-36f9b46658d0","Type":"ContainerStarted","Data":"d6e9df71474a78c091dab5f561a012c4bd415cd4ad1a5032b4348d0360945fcd"}
Jul 27 19:49:47.024928 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:47.024878    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5" podStartSLOduration=1.826073141 podStartE2EDuration="2.024851426s" totalImagesPullingTime="198.778285ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:49:47.022878709 +0000 UTC m=+615.252120907" watchObservedRunningTime="2026-07-27 19:49:47.024851426 +0000 UTC m=+615.254093623"
Jul 27 19:49:55.846985 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:55.846903    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:55.846985 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:55.846957    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:55.859663 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:55.859633    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:49:56.053408 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:49:56.053376    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5"
Jul 27 19:50:56.661932 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:56.661898    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75c9bf7749-5m9gs"]
Jul 27 19:50:56.665243 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:56.665219    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75c9bf7749-5m9gs"
Jul 27 19:50:56.695915 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:56.695891    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75c9bf7749-5m9gs"]
Jul 27 19:50:56.709345 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:56.709327    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jlnw\" (UniqueName: \"kubernetes.io/projected/7d81d979-b40c-494a-b262-cb16d12581ee-kube-api-access-9jlnw\") pod \"authorino-75c9bf7749-5m9gs\" (UID: \"7d81d979-b40c-494a-b262-cb16d12581ee\") " pod="kuadrant-system/authorino-75c9bf7749-5m9gs"
Jul 27 19:50:56.709447 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:56.709378    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7d81d979-b40c-494a-b262-cb16d12581ee-tls-cert\") pod \"authorino-75c9bf7749-5m9gs\" (UID: \"7d81d979-b40c-494a-b262-cb16d12581ee\") " pod="kuadrant-system/authorino-75c9bf7749-5m9gs"
Jul 27 19:50:56.810478 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:56.810448    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jlnw\" (UniqueName: \"kubernetes.io/projected/7d81d979-b40c-494a-b262-cb16d12581ee-kube-api-access-9jlnw\") pod \"authorino-75c9bf7749-5m9gs\" (UID: \"7d81d979-b40c-494a-b262-cb16d12581ee\") " pod="kuadrant-system/authorino-75c9bf7749-5m9gs"
Jul 27 19:50:56.810575 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:56.810512    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7d81d979-b40c-494a-b262-cb16d12581ee-tls-cert\") pod \"authorino-75c9bf7749-5m9gs\" (UID: \"7d81d979-b40c-494a-b262-cb16d12581ee\") " pod="kuadrant-system/authorino-75c9bf7749-5m9gs"
Jul 27 19:50:56.813086 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:56.813069    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7d81d979-b40c-494a-b262-cb16d12581ee-tls-cert\") pod \"authorino-75c9bf7749-5m9gs\" (UID: \"7d81d979-b40c-494a-b262-cb16d12581ee\") " pod="kuadrant-system/authorino-75c9bf7749-5m9gs"
Jul 27 19:50:56.822825 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:56.822796    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jlnw\" (UniqueName: \"kubernetes.io/projected/7d81d979-b40c-494a-b262-cb16d12581ee-kube-api-access-9jlnw\") pod \"authorino-75c9bf7749-5m9gs\" (UID: \"7d81d979-b40c-494a-b262-cb16d12581ee\") " pod="kuadrant-system/authorino-75c9bf7749-5m9gs"
Jul 27 19:50:56.974036 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:56.973975    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75c9bf7749-5m9gs"
Jul 27 19:50:57.101313 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:57.101236    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75c9bf7749-5m9gs"]
Jul 27 19:50:57.103549 ip-10-0-140-36 kubenswrapper[2579]: W0727 19:50:57.103524    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d81d979_b40c_494a_b262_cb16d12581ee.slice/crio-24524755c777f7eaab576a13c40486717b218c39b448d71922fa9489f479ba93 WatchSource:0}: Error finding container 24524755c777f7eaab576a13c40486717b218c39b448d71922fa9489f479ba93: Status 404 returned error can't find the container with id 24524755c777f7eaab576a13c40486717b218c39b448d71922fa9489f479ba93
Jul 27 19:50:57.309112 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:57.309025    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75c9bf7749-5m9gs" event={"ID":"7d81d979-b40c-494a-b262-cb16d12581ee","Type":"ContainerStarted","Data":"24524755c777f7eaab576a13c40486717b218c39b448d71922fa9489f479ba93"}
Jul 27 19:50:58.316135 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:58.316090    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75c9bf7749-5m9gs" event={"ID":"7d81d979-b40c-494a-b262-cb16d12581ee","Type":"ContainerStarted","Data":"08ab3494bb5e4d0148ebd7af863e5740846e923d81c4765731077f2ca434f08f"}
Jul 27 19:50:58.336606 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:58.336549    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75c9bf7749-5m9gs" podStartSLOduration=1.9286740980000001 podStartE2EDuration="2.33652912s" totalImagesPullingTime="407.855022ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:50:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:50:58.33609846 +0000 UTC m=+686.565340687" watchObservedRunningTime="2026-07-27 19:50:58.33652912 +0000 UTC m=+686.565771320"
Jul 27 19:50:58.378070 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:58.378030    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-558f77558d-m2wqh"]
Jul 27 19:50:58.378314 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:58.378262    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-558f77558d-m2wqh" podUID="b8d1c8ea-230b-4e83-af4c-c5c22393c5ca" containerName="authorino" containerID="cri-o://981081a98fe30b8c1c9f6e12fad892d9322d20b5858b1eab971b7a3238db7a6c" gracePeriod=30
Jul 27 19:50:58.656031 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:58.656007    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-558f77558d-m2wqh"
Jul 27 19:50:58.727308 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:58.727280    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpdxf\" (UniqueName: \"kubernetes.io/projected/b8d1c8ea-230b-4e83-af4c-c5c22393c5ca-kube-api-access-xpdxf\") pod \"b8d1c8ea-230b-4e83-af4c-c5c22393c5ca\" (UID: \"b8d1c8ea-230b-4e83-af4c-c5c22393c5ca\") "
Jul 27 19:50:58.727477 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:58.727336    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b8d1c8ea-230b-4e83-af4c-c5c22393c5ca-tls-cert\") pod \"b8d1c8ea-230b-4e83-af4c-c5c22393c5ca\" (UID: \"b8d1c8ea-230b-4e83-af4c-c5c22393c5ca\") "
Jul 27 19:50:58.730048 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:58.730006    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8d1c8ea-230b-4e83-af4c-c5c22393c5ca-kube-api-access-xpdxf" (OuterVolumeSpecName: "kube-api-access-xpdxf") pod "b8d1c8ea-230b-4e83-af4c-c5c22393c5ca" (UID: "b8d1c8ea-230b-4e83-af4c-c5c22393c5ca"). InnerVolumeSpecName "kube-api-access-xpdxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:50:58.737389 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:58.737367    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8d1c8ea-230b-4e83-af4c-c5c22393c5ca-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "b8d1c8ea-230b-4e83-af4c-c5c22393c5ca" (UID: "b8d1c8ea-230b-4e83-af4c-c5c22393c5ca"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:50:58.828888 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:58.828846    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpdxf\" (UniqueName: \"kubernetes.io/projected/b8d1c8ea-230b-4e83-af4c-c5c22393c5ca-kube-api-access-xpdxf\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 19:50:58.828888 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:58.828887    2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b8d1c8ea-230b-4e83-af4c-c5c22393c5ca-tls-cert\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 19:50:59.321901 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:59.321841    2579 generic.go:356] "Generic (PLEG): container finished" podID="b8d1c8ea-230b-4e83-af4c-c5c22393c5ca" containerID="981081a98fe30b8c1c9f6e12fad892d9322d20b5858b1eab971b7a3238db7a6c" exitCode=0
Jul 27 19:50:59.321901 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:59.321904    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-558f77558d-m2wqh"
Jul 27 19:50:59.322376 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:59.321909    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-558f77558d-m2wqh" event={"ID":"b8d1c8ea-230b-4e83-af4c-c5c22393c5ca","Type":"ContainerDied","Data":"981081a98fe30b8c1c9f6e12fad892d9322d20b5858b1eab971b7a3238db7a6c"}
Jul 27 19:50:59.322376 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:59.321954    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-558f77558d-m2wqh" event={"ID":"b8d1c8ea-230b-4e83-af4c-c5c22393c5ca","Type":"ContainerDied","Data":"708afc4551362c890957e7d2348b562ef8a1f7d6236788e5b8d97dbbcf2f8175"}
Jul 27 19:50:59.322376 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:59.321972    2579 scope.go:117] "RemoveContainer" containerID="981081a98fe30b8c1c9f6e12fad892d9322d20b5858b1eab971b7a3238db7a6c"
Jul 27 19:50:59.333686 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:59.333636    2579 scope.go:117] "RemoveContainer" containerID="981081a98fe30b8c1c9f6e12fad892d9322d20b5858b1eab971b7a3238db7a6c"
Jul 27 19:50:59.334354 ip-10-0-140-36 kubenswrapper[2579]: E0727 19:50:59.334334    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"981081a98fe30b8c1c9f6e12fad892d9322d20b5858b1eab971b7a3238db7a6c\": container with ID starting with 981081a98fe30b8c1c9f6e12fad892d9322d20b5858b1eab971b7a3238db7a6c not found: ID does not exist" containerID="981081a98fe30b8c1c9f6e12fad892d9322d20b5858b1eab971b7a3238db7a6c"
Jul 27 19:50:59.334434 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:59.334363    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"981081a98fe30b8c1c9f6e12fad892d9322d20b5858b1eab971b7a3238db7a6c"} err="failed to get container status \"981081a98fe30b8c1c9f6e12fad892d9322d20b5858b1eab971b7a3238db7a6c\": rpc error: code = NotFound desc = could not find container \"981081a98fe30b8c1c9f6e12fad892d9322d20b5858b1eab971b7a3238db7a6c\": container with ID starting with 981081a98fe30b8c1c9f6e12fad892d9322d20b5858b1eab971b7a3238db7a6c not found: ID does not exist"
Jul 27 19:50:59.353058 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:59.353029    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-558f77558d-m2wqh"]
Jul 27 19:50:59.359514 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:50:59.359494    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-558f77558d-m2wqh"]
Jul 27 19:51:00.261509 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:51:00.261478    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8d1c8ea-230b-4e83-af4c-c5c22393c5ca" path="/var/lib/kubelet/pods/b8d1c8ea-230b-4e83-af4c-c5c22393c5ca/volumes"
Jul 27 19:54:32.250701 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:54:32.250629    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 19:54:32.251254 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:54:32.250964    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 19:59:32.284766 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:59:32.284739    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 19:59:32.287045 ip-10-0-140-36 kubenswrapper[2579]: I0727 19:59:32.287020    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 20:01:39.953104 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:39.953072    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"]
Jul 27 20:01:39.955449 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:39.953397    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8d1c8ea-230b-4e83-af4c-c5c22393c5ca" containerName="authorino"
Jul 27 20:01:39.955449 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:39.953410    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8d1c8ea-230b-4e83-af4c-c5c22393c5ca" containerName="authorino"
Jul 27 20:01:39.955449 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:39.953466    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8d1c8ea-230b-4e83-af4c-c5c22393c5ca" containerName="authorino"
Jul 27 20:01:39.956330 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:39.956317    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:39.958573 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:39.958556    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-8fd18498-kserve-self-signed-certs\""
Jul 27 20:01:39.985878 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:39.985823    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"]
Jul 27 20:01:40.054162 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:40.054133    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-home\") pod \"e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") " pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:40.054162 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:40.054163    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-model-cache\") pod \"e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") " pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:40.054356 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:40.054179    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16782780-c09f-47f2-b4d8-818de8883e3f-tls-certs\") pod \"e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") " pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:40.054356 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:40.054275    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-dshm\") pod \"e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") " pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:40.054356 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:40.054303    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-tmp-dir\") pod \"e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") " pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:40.054356 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:40.054326    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6rkc\" (UniqueName: \"kubernetes.io/projected/16782780-c09f-47f2-b4d8-818de8883e3f-kube-api-access-v6rkc\") pod \"e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") " pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:40.155334 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:40.155297    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-tmp-dir\") pod \"e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") " pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:40.155334 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:40.155341    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6rkc\" (UniqueName: \"kubernetes.io/projected/16782780-c09f-47f2-b4d8-818de8883e3f-kube-api-access-v6rkc\") pod \"e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") " pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:40.155487 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:40.155367    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-home\") pod \"e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") " pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:40.155487 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:40.155476    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-model-cache\") pod \"e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") " pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:40.155571 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:40.155503    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16782780-c09f-47f2-b4d8-818de8883e3f-tls-certs\") pod \"e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") " pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:40.155644 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:40.155581    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-dshm\") pod \"e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") " pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:40.155697 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:40.155673    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-tmp-dir\") pod \"e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") " pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:40.155753 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:40.155729    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-home\") pod \"e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") " pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:40.155914 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:40.155893    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-model-cache\") pod \"e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") " pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:40.157951 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:40.157932    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-dshm\") pod \"e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") " pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:40.158206 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:40.158189    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16782780-c09f-47f2-b4d8-818de8883e3f-tls-certs\") pod \"e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") " pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:40.163011 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:40.162993    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6rkc\" (UniqueName: \"kubernetes.io/projected/16782780-c09f-47f2-b4d8-818de8883e3f-kube-api-access-v6rkc\") pod \"e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") " pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:40.265387 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:40.265337    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:40.394714 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:40.394685    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"]
Jul 27 20:01:40.398031 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:01:40.397998    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16782780_c09f_47f2_b4d8_818de8883e3f.slice/crio-6c14a4e4cb7d8b52dff8ea327f666a98e2945d3ff07d8d9e4c1cfe62af3ef367 WatchSource:0}: Error finding container 6c14a4e4cb7d8b52dff8ea327f666a98e2945d3ff07d8d9e4c1cfe62af3ef367: Status 404 returned error can't find the container with id 6c14a4e4cb7d8b52dff8ea327f666a98e2945d3ff07d8d9e4c1cfe62af3ef367
Jul 27 20:01:41.092246 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:41.092210    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2" event={"ID":"16782780-c09f-47f2-b4d8-818de8883e3f","Type":"ContainerStarted","Data":"97a73abffc63ef2ebeb1844ed917e3ffedd17e983cb02ffe45f18c83443597b6"}
Jul 27 20:01:41.092717 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:41.092255    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2" event={"ID":"16782780-c09f-47f2-b4d8-818de8883e3f","Type":"ContainerStarted","Data":"6c14a4e4cb7d8b52dff8ea327f666a98e2945d3ff07d8d9e4c1cfe62af3ef367"}
Jul 27 20:01:41.110946 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:41.110898    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2" podStartSLOduration=2.110884609 podStartE2EDuration="2.110884609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:01:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:01:41.108955906 +0000 UTC m=+1329.338198105" watchObservedRunningTime="2026-07-27 20:01:41.110884609 +0000 UTC m=+1329.340126808"
Jul 27 20:01:49.036031 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.036001    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"]
Jul 27 20:01:49.039458 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.039442    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:49.041346 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.041326    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-8fd18498-kserve-self-signed-certs\""
Jul 27 20:01:49.047851 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.047832    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"]
Jul 27 20:01:49.129020 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.128991    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmvxp\" (UniqueName: \"kubernetes.io/projected/2e97303a-3276-4f97-a659-406545aa8e27-kube-api-access-jmvxp\") pod \"e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") " pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:49.129131 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.129041    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e97303a-3276-4f97-a659-406545aa8e27-tls-certs\") pod \"e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") " pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:49.129131 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.129099    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-home\") pod \"e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") " pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:49.129220 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.129144    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-dshm\") pod \"e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") " pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:49.129220 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.129172    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-model-cache\") pod \"e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") " pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:49.129220 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.129200    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-tmp-dir\") pod \"e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") " pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:49.230550 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.230523    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-model-cache\") pod \"e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") " pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:49.230648 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.230557    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-tmp-dir\") pod \"e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") " pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:49.230648 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.230614    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmvxp\" (UniqueName: \"kubernetes.io/projected/2e97303a-3276-4f97-a659-406545aa8e27-kube-api-access-jmvxp\") pod \"e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") " pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:49.230742 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.230647    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e97303a-3276-4f97-a659-406545aa8e27-tls-certs\") pod \"e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") " pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:49.230742 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.230685    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-home\") pod \"e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") " pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:49.230742 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.230734    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-dshm\") pod \"e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") " pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:49.231031 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.231002    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-model-cache\") pod \"e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") " pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:49.231031 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.231029    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-tmp-dir\") pod \"e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") " pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:49.231173 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.231069    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-home\") pod \"e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") " pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:49.233125 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.233103    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-dshm\") pod \"e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") " pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:49.233377 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.233360    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e97303a-3276-4f97-a659-406545aa8e27-tls-certs\") pod \"e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") " pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:49.237690 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.237671    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmvxp\" (UniqueName: \"kubernetes.io/projected/2e97303a-3276-4f97-a659-406545aa8e27-kube-api-access-jmvxp\") pod \"e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") " pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:49.349829 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.349776    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:49.479060 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:49.479036    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"]
Jul 27 20:01:49.481627 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:01:49.481604    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e97303a_3276_4f97_a659_406545aa8e27.slice/crio-74d2877d5992ff6279b12bba6255fc666b18cdd12b5b13d852b4776781c408fb WatchSource:0}: Error finding container 74d2877d5992ff6279b12bba6255fc666b18cdd12b5b13d852b4776781c408fb: Status 404 returned error can't find the container with id 74d2877d5992ff6279b12bba6255fc666b18cdd12b5b13d852b4776781c408fb
Jul 27 20:01:50.131569 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:50.131531    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5" event={"ID":"2e97303a-3276-4f97-a659-406545aa8e27","Type":"ContainerStarted","Data":"bec96d21ca03c6210d639e69d963ac2c2e779a5f39b6841f2c6d975be2285caf"}
Jul 27 20:01:50.131569 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:50.131573    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5" event={"ID":"2e97303a-3276-4f97-a659-406545aa8e27","Type":"ContainerStarted","Data":"74d2877d5992ff6279b12bba6255fc666b18cdd12b5b13d852b4776781c408fb"}
Jul 27 20:01:50.149754 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:50.149712    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5" podStartSLOduration=1.149697392 podStartE2EDuration="1.149697392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:01:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:01:50.148134236 +0000 UTC m=+1338.377376435" watchObservedRunningTime="2026-07-27 20:01:50.149697392 +0000 UTC m=+1338.378939606"
Jul 27 20:01:50.266315 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:50.266292    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:50.266447 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:50.266330    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:50.278644 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:50.278624    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:51.147115 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:51.147085    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:01:59.350978 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:59.350885    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:59.350978 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:59.350938    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:01:59.363337 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:01:59.363313    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:02:00.186892 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:00.186846    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:02:17.868390 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:17.868355    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"]
Jul 27 20:02:19.513193 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:19.513162    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"]
Jul 27 20:02:32.887746 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:32.887681    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2" podUID="16782780-c09f-47f2-b4d8-818de8883e3f" containerName="main" containerID="cri-o://97a73abffc63ef2ebeb1844ed917e3ffedd17e983cb02ffe45f18c83443597b6" gracePeriod=45
Jul 27 20:02:33.134911 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.134884    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:02:33.185632 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.185607    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16782780-c09f-47f2-b4d8-818de8883e3f-tls-certs\") pod \"16782780-c09f-47f2-b4d8-818de8883e3f\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") "
Jul 27 20:02:33.185805 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.185702    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-home\") pod \"16782780-c09f-47f2-b4d8-818de8883e3f\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") "
Jul 27 20:02:33.185805 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.185734    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6rkc\" (UniqueName: \"kubernetes.io/projected/16782780-c09f-47f2-b4d8-818de8883e3f-kube-api-access-v6rkc\") pod \"16782780-c09f-47f2-b4d8-818de8883e3f\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") "
Jul 27 20:02:33.185805 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.185756    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-tmp-dir\") pod \"16782780-c09f-47f2-b4d8-818de8883e3f\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") "
Jul 27 20:02:33.185995 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.185805    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-dshm\") pod \"16782780-c09f-47f2-b4d8-818de8883e3f\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") "
Jul 27 20:02:33.185995 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.185894    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-model-cache\") pod \"16782780-c09f-47f2-b4d8-818de8883e3f\" (UID: \"16782780-c09f-47f2-b4d8-818de8883e3f\") "
Jul 27 20:02:33.186096 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.186005    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-home" (OuterVolumeSpecName: "home") pod "16782780-c09f-47f2-b4d8-818de8883e3f" (UID: "16782780-c09f-47f2-b4d8-818de8883e3f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:02:33.186096 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.186056    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "16782780-c09f-47f2-b4d8-818de8883e3f" (UID: "16782780-c09f-47f2-b4d8-818de8883e3f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:02:33.186206 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.186182    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-model-cache" (OuterVolumeSpecName: "model-cache") pod "16782780-c09f-47f2-b4d8-818de8883e3f" (UID: "16782780-c09f-47f2-b4d8-818de8883e3f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:02:33.186259 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.186223    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-home\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:02:33.186259 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.186240    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-tmp-dir\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:02:33.188036 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.188010    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16782780-c09f-47f2-b4d8-818de8883e3f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "16782780-c09f-47f2-b4d8-818de8883e3f" (UID: "16782780-c09f-47f2-b4d8-818de8883e3f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:02:33.188131 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.188046    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-dshm" (OuterVolumeSpecName: "dshm") pod "16782780-c09f-47f2-b4d8-818de8883e3f" (UID: "16782780-c09f-47f2-b4d8-818de8883e3f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:02:33.188131 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.188061    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16782780-c09f-47f2-b4d8-818de8883e3f-kube-api-access-v6rkc" (OuterVolumeSpecName: "kube-api-access-v6rkc") pod "16782780-c09f-47f2-b4d8-818de8883e3f" (UID: "16782780-c09f-47f2-b4d8-818de8883e3f"). InnerVolumeSpecName "kube-api-access-v6rkc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:02:33.287197 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.287166    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6rkc\" (UniqueName: \"kubernetes.io/projected/16782780-c09f-47f2-b4d8-818de8883e3f-kube-api-access-v6rkc\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:02:33.287197 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.287193    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-dshm\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:02:33.287197 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.287203    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16782780-c09f-47f2-b4d8-818de8883e3f-model-cache\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:02:33.287433 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.287211    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16782780-c09f-47f2-b4d8-818de8883e3f-tls-certs\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:02:33.320928 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.320896    2579 generic.go:356] "Generic (PLEG): container finished" podID="16782780-c09f-47f2-b4d8-818de8883e3f" containerID="97a73abffc63ef2ebeb1844ed917e3ffedd17e983cb02ffe45f18c83443597b6" exitCode=0
Jul 27 20:02:33.321074 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.320940    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"
Jul 27 20:02:33.321074 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.320946    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2" event={"ID":"16782780-c09f-47f2-b4d8-818de8883e3f","Type":"ContainerDied","Data":"97a73abffc63ef2ebeb1844ed917e3ffedd17e983cb02ffe45f18c83443597b6"}
Jul 27 20:02:33.321074 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.320984    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2" event={"ID":"16782780-c09f-47f2-b4d8-818de8883e3f","Type":"ContainerDied","Data":"6c14a4e4cb7d8b52dff8ea327f666a98e2945d3ff07d8d9e4c1cfe62af3ef367"}
Jul 27 20:02:33.321074 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.321007    2579 scope.go:117] "RemoveContainer" containerID="97a73abffc63ef2ebeb1844ed917e3ffedd17e983cb02ffe45f18c83443597b6"
Jul 27 20:02:33.329840 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.329820    2579 scope.go:117] "RemoveContainer" containerID="97a73abffc63ef2ebeb1844ed917e3ffedd17e983cb02ffe45f18c83443597b6"
Jul 27 20:02:33.330131 ip-10-0-140-36 kubenswrapper[2579]: E0727 20:02:33.330110    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97a73abffc63ef2ebeb1844ed917e3ffedd17e983cb02ffe45f18c83443597b6\": container with ID starting with 97a73abffc63ef2ebeb1844ed917e3ffedd17e983cb02ffe45f18c83443597b6 not found: ID does not exist" containerID="97a73abffc63ef2ebeb1844ed917e3ffedd17e983cb02ffe45f18c83443597b6"
Jul 27 20:02:33.330181 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.330139    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a73abffc63ef2ebeb1844ed917e3ffedd17e983cb02ffe45f18c83443597b6"} err="failed to get container status \"97a73abffc63ef2ebeb1844ed917e3ffedd17e983cb02ffe45f18c83443597b6\": rpc error: code = NotFound desc = could not find container \"97a73abffc63ef2ebeb1844ed917e3ffedd17e983cb02ffe45f18c83443597b6\": container with ID starting with 97a73abffc63ef2ebeb1844ed917e3ffedd17e983cb02ffe45f18c83443597b6 not found: ID does not exist"
Jul 27 20:02:33.340091 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.340069    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"]
Jul 27 20:02:33.343519 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:33.343501    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-8fd18498-kserve-5d89dc9b75-w24s2"]
Jul 27 20:02:34.261051 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:34.261017    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16782780-c09f-47f2-b4d8-818de8883e3f" path="/var/lib/kubelet/pods/16782780-c09f-47f2-b4d8-818de8883e3f/volumes"
Jul 27 20:02:34.532445 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:34.532356    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5" podUID="2e97303a-3276-4f97-a659-406545aa8e27" containerName="main" containerID="cri-o://bec96d21ca03c6210d639e69d963ac2c2e779a5f39b6841f2c6d975be2285caf" gracePeriod=45
Jul 27 20:02:34.775711 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:34.775681    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:02:34.900217 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:34.900131    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-home\") pod \"2e97303a-3276-4f97-a659-406545aa8e27\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") "
Jul 27 20:02:34.900217 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:34.900172    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-tmp-dir\") pod \"2e97303a-3276-4f97-a659-406545aa8e27\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") "
Jul 27 20:02:34.900217 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:34.900199    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-dshm\") pod \"2e97303a-3276-4f97-a659-406545aa8e27\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") "
Jul 27 20:02:34.900492 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:34.900264    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmvxp\" (UniqueName: \"kubernetes.io/projected/2e97303a-3276-4f97-a659-406545aa8e27-kube-api-access-jmvxp\") pod \"2e97303a-3276-4f97-a659-406545aa8e27\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") "
Jul 27 20:02:34.900492 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:34.900280    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e97303a-3276-4f97-a659-406545aa8e27-tls-certs\") pod \"2e97303a-3276-4f97-a659-406545aa8e27\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") "
Jul 27 20:02:34.900492 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:34.900428    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-model-cache\") pod \"2e97303a-3276-4f97-a659-406545aa8e27\" (UID: \"2e97303a-3276-4f97-a659-406545aa8e27\") "
Jul 27 20:02:34.900492 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:34.900463    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-home" (OuterVolumeSpecName: "home") pod "2e97303a-3276-4f97-a659-406545aa8e27" (UID: "2e97303a-3276-4f97-a659-406545aa8e27"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:02:34.900752 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:34.900510    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2e97303a-3276-4f97-a659-406545aa8e27" (UID: "2e97303a-3276-4f97-a659-406545aa8e27"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:02:34.900752 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:34.900639    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-model-cache" (OuterVolumeSpecName: "model-cache") pod "2e97303a-3276-4f97-a659-406545aa8e27" (UID: "2e97303a-3276-4f97-a659-406545aa8e27"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:02:34.900842 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:34.900757    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-model-cache\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:02:34.900842 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:34.900772    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-home\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:02:34.900842 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:34.900785    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-tmp-dir\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:02:34.902570 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:34.902539    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e97303a-3276-4f97-a659-406545aa8e27-kube-api-access-jmvxp" (OuterVolumeSpecName: "kube-api-access-jmvxp") pod "2e97303a-3276-4f97-a659-406545aa8e27" (UID: "2e97303a-3276-4f97-a659-406545aa8e27"). InnerVolumeSpecName "kube-api-access-jmvxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:02:34.902838 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:34.902815    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-dshm" (OuterVolumeSpecName: "dshm") pod "2e97303a-3276-4f97-a659-406545aa8e27" (UID: "2e97303a-3276-4f97-a659-406545aa8e27"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:02:34.902935 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:34.902917    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e97303a-3276-4f97-a659-406545aa8e27-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2e97303a-3276-4f97-a659-406545aa8e27" (UID: "2e97303a-3276-4f97-a659-406545aa8e27"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:02:35.001456 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:35.001424    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e97303a-3276-4f97-a659-406545aa8e27-dshm\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:02:35.001456 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:35.001450    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmvxp\" (UniqueName: \"kubernetes.io/projected/2e97303a-3276-4f97-a659-406545aa8e27-kube-api-access-jmvxp\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:02:35.001456 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:35.001460    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e97303a-3276-4f97-a659-406545aa8e27-tls-certs\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:02:35.331175 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:35.331137    2579 generic.go:356] "Generic (PLEG): container finished" podID="2e97303a-3276-4f97-a659-406545aa8e27" containerID="bec96d21ca03c6210d639e69d963ac2c2e779a5f39b6841f2c6d975be2285caf" exitCode=0
Jul 27 20:02:35.331611 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:35.331179    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"
Jul 27 20:02:35.331611 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:35.331189    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5" event={"ID":"2e97303a-3276-4f97-a659-406545aa8e27","Type":"ContainerDied","Data":"bec96d21ca03c6210d639e69d963ac2c2e779a5f39b6841f2c6d975be2285caf"}
Jul 27 20:02:35.331611 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:35.331228    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5" event={"ID":"2e97303a-3276-4f97-a659-406545aa8e27","Type":"ContainerDied","Data":"74d2877d5992ff6279b12bba6255fc666b18cdd12b5b13d852b4776781c408fb"}
Jul 27 20:02:35.331611 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:35.331249    2579 scope.go:117] "RemoveContainer" containerID="bec96d21ca03c6210d639e69d963ac2c2e779a5f39b6841f2c6d975be2285caf"
Jul 27 20:02:35.340215 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:35.340195    2579 scope.go:117] "RemoveContainer" containerID="bec96d21ca03c6210d639e69d963ac2c2e779a5f39b6841f2c6d975be2285caf"
Jul 27 20:02:35.340459 ip-10-0-140-36 kubenswrapper[2579]: E0727 20:02:35.340440    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bec96d21ca03c6210d639e69d963ac2c2e779a5f39b6841f2c6d975be2285caf\": container with ID starting with bec96d21ca03c6210d639e69d963ac2c2e779a5f39b6841f2c6d975be2285caf not found: ID does not exist" containerID="bec96d21ca03c6210d639e69d963ac2c2e779a5f39b6841f2c6d975be2285caf"
Jul 27 20:02:35.340506 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:35.340468    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bec96d21ca03c6210d639e69d963ac2c2e779a5f39b6841f2c6d975be2285caf"} err="failed to get container status \"bec96d21ca03c6210d639e69d963ac2c2e779a5f39b6841f2c6d975be2285caf\": rpc error: code = NotFound desc = could not find container \"bec96d21ca03c6210d639e69d963ac2c2e779a5f39b6841f2c6d975be2285caf\": container with ID starting with bec96d21ca03c6210d639e69d963ac2c2e779a5f39b6841f2c6d975be2285caf not found: ID does not exist"
Jul 27 20:02:35.351026 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:35.351007    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"]
Jul 27 20:02:35.354689 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:35.354668    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-8fd18498-kserve-65fccfbd5f-hv5g5"]
Jul 27 20:02:36.261679 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:02:36.261644    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e97303a-3276-4f97-a659-406545aa8e27" path="/var/lib/kubelet/pods/2e97303a-3276-4f97-a659-406545aa8e27/volumes"
Jul 27 20:04:32.315648 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:04:32.315622    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 20:04:32.319639 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:04:32.319617    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 20:09:32.342620 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:09:32.342589    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 20:09:32.347434 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:09:32.347412    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 20:11:59.756910 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.756877    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"]
Jul 27 20:11:59.757337 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.757172    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16782780-c09f-47f2-b4d8-818de8883e3f" containerName="main"
Jul 27 20:11:59.757337 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.757182    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="16782780-c09f-47f2-b4d8-818de8883e3f" containerName="main"
Jul 27 20:11:59.757337 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.757204    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e97303a-3276-4f97-a659-406545aa8e27" containerName="main"
Jul 27 20:11:59.757337 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.757210    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e97303a-3276-4f97-a659-406545aa8e27" containerName="main"
Jul 27 20:11:59.757337 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.757259    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="16782780-c09f-47f2-b4d8-818de8883e3f" containerName="main"
Jul 27 20:11:59.757337 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.757268    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e97303a-3276-4f97-a659-406545aa8e27" containerName="main"
Jul 27 20:11:59.760225 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.760204    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:11:59.762123 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.762097    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-25d7f4d7\"/\"e2e-lifecycle-model-25d7f4d7-kserve-self-signed-certs\""
Jul 27 20:11:59.762984 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.762960    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-25d7f4d7\"/\"kube-root-ca.crt\""
Jul 27 20:11:59.763100 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.763023    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-25d7f4d7\"/\"default-dockercfg-tztmf\""
Jul 27 20:11:59.763168 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.763121    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-25d7f4d7\"/\"openshift-service-ca.crt\""
Jul 27 20:11:59.768065 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.768045    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"]
Jul 27 20:11:59.853252 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.853223    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-home\") pod \"e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") " pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:11:59.853252 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.853252    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-model-cache\") pod \"e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") " pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:11:59.853483 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.853278    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-tls-certs\") pod \"e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") " pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:11:59.853483 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.853392    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-dshm\") pod \"e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") " pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:11:59.853483 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.853450    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-tmp-dir\") pod \"e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") " pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:11:59.853609 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.853493    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xhch\" (UniqueName: \"kubernetes.io/projected/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-kube-api-access-9xhch\") pod \"e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") " pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:11:59.954241 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.954209    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xhch\" (UniqueName: \"kubernetes.io/projected/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-kube-api-access-9xhch\") pod \"e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") " pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:11:59.954386 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.954258    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-home\") pod \"e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") " pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:11:59.954386 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.954284    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-model-cache\") pod \"e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") " pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:11:59.954386 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.954317    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-tls-certs\") pod \"e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") " pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:11:59.954386 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.954356    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-dshm\") pod \"e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") " pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:11:59.954602 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.954401    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-tmp-dir\") pod \"e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") " pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:11:59.954702 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.954682    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-home\") pod \"e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") " pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:11:59.954765 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.954720    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-model-cache\") pod \"e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") " pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:11:59.954765 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.954740    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-tmp-dir\") pod \"e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") " pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:11:59.956890 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.956836    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-dshm\") pod \"e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") " pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:11:59.957045 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.957027    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-tls-certs\") pod \"e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") " pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:11:59.964257 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:11:59.964237    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xhch\" (UniqueName: \"kubernetes.io/projected/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-kube-api-access-9xhch\") pod \"e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") " pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:12:00.069933 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:12:00.069851    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:12:00.199581 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:12:00.199546    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"]
Jul 27 20:12:00.203257 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:12:00.203230    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb12f8a8_5874_4a1c_a0af_0803932bf7b1.slice/crio-72ddb187b6f468909701e79b7f43629bf5e061bd2025b838a4b505401e428880 WatchSource:0}: Error finding container 72ddb187b6f468909701e79b7f43629bf5e061bd2025b838a4b505401e428880: Status 404 returned error can't find the container with id 72ddb187b6f468909701e79b7f43629bf5e061bd2025b838a4b505401e428880
Jul 27 20:12:00.801148 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:12:00.801104    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2" event={"ID":"fb12f8a8-5874-4a1c-a0af-0803932bf7b1","Type":"ContainerStarted","Data":"6ada2fefe54a778d5b12be76fed2d59761b761330ec5d6093f02202f7db36ba3"}
Jul 27 20:12:00.801148 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:12:00.801150    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2" event={"ID":"fb12f8a8-5874-4a1c-a0af-0803932bf7b1","Type":"ContainerStarted","Data":"72ddb187b6f468909701e79b7f43629bf5e061bd2025b838a4b505401e428880"}
Jul 27 20:12:00.820345 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:12:00.820285    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2" podStartSLOduration=1.820263632 podStartE2EDuration="1.820263632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:11:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:12:00.817977043 +0000 UTC m=+1949.047219255" watchObservedRunningTime="2026-07-27 20:12:00.820263632 +0000 UTC m=+1949.049505831"
Jul 27 20:12:10.070957 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:12:10.070922    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:12:10.071438 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:12:10.070978    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:12:10.083254 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:12:10.083228    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:12:10.853403 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:12:10.853373    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:13:07.367728 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:07.367680    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"]
Jul 27 20:13:22.387427 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:22.387380    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2" podUID="fb12f8a8-5874-4a1c-a0af-0803932bf7b1" containerName="main" containerID="cri-o://6ada2fefe54a778d5b12be76fed2d59761b761330ec5d6093f02202f7db36ba3" gracePeriod=45
Jul 27 20:13:22.629513 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:22.629485    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:13:22.694430 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:22.694404    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xhch\" (UniqueName: \"kubernetes.io/projected/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-kube-api-access-9xhch\") pod \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") "
Jul 27 20:13:22.694597 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:22.694453    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-home\") pod \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") "
Jul 27 20:13:22.694597 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:22.694475    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-tmp-dir\") pod \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") "
Jul 27 20:13:22.694597 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:22.694522    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-tls-certs\") pod \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") "
Jul 27 20:13:22.694597 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:22.694551    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-dshm\") pod \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") "
Jul 27 20:13:22.694817 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:22.694614    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-model-cache\") pod \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\" (UID: \"fb12f8a8-5874-4a1c-a0af-0803932bf7b1\") "
Jul 27 20:13:22.694817 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:22.694695    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-home" (OuterVolumeSpecName: "home") pod "fb12f8a8-5874-4a1c-a0af-0803932bf7b1" (UID: "fb12f8a8-5874-4a1c-a0af-0803932bf7b1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:13:22.694817 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:22.694781    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fb12f8a8-5874-4a1c-a0af-0803932bf7b1" (UID: "fb12f8a8-5874-4a1c-a0af-0803932bf7b1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:13:22.694999 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:22.694916    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-model-cache" (OuterVolumeSpecName: "model-cache") pod "fb12f8a8-5874-4a1c-a0af-0803932bf7b1" (UID: "fb12f8a8-5874-4a1c-a0af-0803932bf7b1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:13:22.694999 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:22.694918    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-home\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:13:22.694999 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:22.694945    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-tmp-dir\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:13:22.696794 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:22.696774    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-dshm" (OuterVolumeSpecName: "dshm") pod "fb12f8a8-5874-4a1c-a0af-0803932bf7b1" (UID: "fb12f8a8-5874-4a1c-a0af-0803932bf7b1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:13:22.696995 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:22.696805    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-kube-api-access-9xhch" (OuterVolumeSpecName: "kube-api-access-9xhch") pod "fb12f8a8-5874-4a1c-a0af-0803932bf7b1" (UID: "fb12f8a8-5874-4a1c-a0af-0803932bf7b1"). InnerVolumeSpecName "kube-api-access-9xhch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:13:22.696995 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:22.696819    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fb12f8a8-5874-4a1c-a0af-0803932bf7b1" (UID: "fb12f8a8-5874-4a1c-a0af-0803932bf7b1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:13:22.795645 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:22.795617    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-model-cache\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:13:22.795645 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:22.795638    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xhch\" (UniqueName: \"kubernetes.io/projected/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-kube-api-access-9xhch\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:13:22.795818 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:22.795654    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-tls-certs\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:13:22.795818 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:22.795666    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb12f8a8-5874-4a1c-a0af-0803932bf7b1-dshm\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:13:23.149268 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:23.149188    2579 generic.go:356] "Generic (PLEG): container finished" podID="fb12f8a8-5874-4a1c-a0af-0803932bf7b1" containerID="6ada2fefe54a778d5b12be76fed2d59761b761330ec5d6093f02202f7db36ba3" exitCode=0
Jul 27 20:13:23.149268 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:23.149228    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"
Jul 27 20:13:23.149448 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:23.149267    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2" event={"ID":"fb12f8a8-5874-4a1c-a0af-0803932bf7b1","Type":"ContainerDied","Data":"6ada2fefe54a778d5b12be76fed2d59761b761330ec5d6093f02202f7db36ba3"}
Jul 27 20:13:23.149448 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:23.149305    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2" event={"ID":"fb12f8a8-5874-4a1c-a0af-0803932bf7b1","Type":"ContainerDied","Data":"72ddb187b6f468909701e79b7f43629bf5e061bd2025b838a4b505401e428880"}
Jul 27 20:13:23.149448 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:23.149324    2579 scope.go:117] "RemoveContainer" containerID="6ada2fefe54a778d5b12be76fed2d59761b761330ec5d6093f02202f7db36ba3"
Jul 27 20:13:23.160898 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:23.160832    2579 scope.go:117] "RemoveContainer" containerID="6ada2fefe54a778d5b12be76fed2d59761b761330ec5d6093f02202f7db36ba3"
Jul 27 20:13:23.161810 ip-10-0-140-36 kubenswrapper[2579]: E0727 20:13:23.161788    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ada2fefe54a778d5b12be76fed2d59761b761330ec5d6093f02202f7db36ba3\": container with ID starting with 6ada2fefe54a778d5b12be76fed2d59761b761330ec5d6093f02202f7db36ba3 not found: ID does not exist" containerID="6ada2fefe54a778d5b12be76fed2d59761b761330ec5d6093f02202f7db36ba3"
Jul 27 20:13:23.161929 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:23.161816    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ada2fefe54a778d5b12be76fed2d59761b761330ec5d6093f02202f7db36ba3"} err="failed to get container status \"6ada2fefe54a778d5b12be76fed2d59761b761330ec5d6093f02202f7db36ba3\": rpc error: code = NotFound desc = could not find container \"6ada2fefe54a778d5b12be76fed2d59761b761330ec5d6093f02202f7db36ba3\": container with ID starting with 6ada2fefe54a778d5b12be76fed2d59761b761330ec5d6093f02202f7db36ba3 not found: ID does not exist"
Jul 27 20:13:23.170622 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:23.170602    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"]
Jul 27 20:13:23.173923 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:23.173904    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-25d7f4d7/e2e-lifecycle-model-25d7f4d7-kserve-6d7c895455-d68g2"]
Jul 27 20:13:24.260621 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:13:24.260588    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb12f8a8-5874-4a1c-a0af-0803932bf7b1" path="/var/lib/kubelet/pods/fb12f8a8-5874-4a1c-a0af-0803932bf7b1/volumes"
Jul 27 20:14:32.370918 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:14:32.370890    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 20:14:32.379130 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:14:32.379104    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 20:15:00.134668 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:00.134638    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-849f6b-29753055-b5fk8"]
Jul 27 20:15:00.135088 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:00.134976    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb12f8a8-5874-4a1c-a0af-0803932bf7b1" containerName="main"
Jul 27 20:15:00.135088 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:00.134987    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb12f8a8-5874-4a1c-a0af-0803932bf7b1" containerName="main"
Jul 27 20:15:00.135088 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:00.135046    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb12f8a8-5874-4a1c-a0af-0803932bf7b1" containerName="main"
Jul 27 20:15:00.137918 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:00.137901    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-849f6b-29753055-b5fk8"
Jul 27 20:15:00.139676 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:00.139654    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x6cbr\""
Jul 27 20:15:00.145316 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:00.145289    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-849f6b-29753055-b5fk8"]
Jul 27 20:15:00.148534 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:00.148512    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-849f6b-29753055-b5fk8"
Jul 27 20:15:00.284578 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:00.284557    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-849f6b-29753055-b5fk8"]
Jul 27 20:15:00.286992 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:15:00.286968    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5822b7b_eff2_4cde_906e_4834ce15a9a1.slice/crio-d4cca299f832f582bf33edf7f73458b01945aa420b4761054c04926a67a9a9e8 WatchSource:0}: Error finding container d4cca299f832f582bf33edf7f73458b01945aa420b4761054c04926a67a9a9e8: Status 404 returned error can't find the container with id d4cca299f832f582bf33edf7f73458b01945aa420b4761054c04926a67a9a9e8
Jul 27 20:15:00.288698 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:00.288683    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 20:15:00.565658 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:00.565630    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-849f6b-29753055-b5fk8" event={"ID":"c5822b7b-eff2-4cde-906e-4834ce15a9a1","Type":"ContainerStarted","Data":"d4cca299f832f582bf33edf7f73458b01945aa420b4761054c04926a67a9a9e8"}
Jul 27 20:15:03.582332 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:03.582299    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-849f6b-29753055-b5fk8" event={"ID":"c5822b7b-eff2-4cde-906e-4834ce15a9a1","Type":"ContainerStarted","Data":"8450ae80ab1777e7a2c5191e3119a773ee3a1953e24a2c6ddfc329968eef69c9"}
Jul 27 20:15:03.597827 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:03.597782    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-849f6b-29753055-b5fk8" podStartSLOduration=1.162094522 podStartE2EDuration="3.59776723s" totalImagesPullingTime="2.435672708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:15:03.596621679 +0000 UTC m=+2131.825863879" watchObservedRunningTime="2026-07-27 20:15:03.59776723 +0000 UTC m=+2131.827009468"
Jul 27 20:15:04.588521 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:04.588482    2579 generic.go:356] "Generic (PLEG): container finished" podID="c5822b7b-eff2-4cde-906e-4834ce15a9a1" containerID="8450ae80ab1777e7a2c5191e3119a773ee3a1953e24a2c6ddfc329968eef69c9" exitCode=0
Jul 27 20:15:04.588932 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:04.588572    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-849f6b-29753055-b5fk8" event={"ID":"c5822b7b-eff2-4cde-906e-4834ce15a9a1","Type":"ContainerDied","Data":"8450ae80ab1777e7a2c5191e3119a773ee3a1953e24a2c6ddfc329968eef69c9"}
Jul 27 20:15:05.720148 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:05.720117    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-849f6b-29753055-b5fk8"
Jul 27 20:15:06.598118 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:06.598042    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-849f6b-29753055-b5fk8"
Jul 27 20:15:06.598118 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:06.598066    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-849f6b-29753055-b5fk8" event={"ID":"c5822b7b-eff2-4cde-906e-4834ce15a9a1","Type":"ContainerDied","Data":"d4cca299f832f582bf33edf7f73458b01945aa420b4761054c04926a67a9a9e8"}
Jul 27 20:15:06.598118 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:06.598107    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4cca299f832f582bf33edf7f73458b01945aa420b4761054c04926a67a9a9e8"
Jul 27 20:15:10.667963 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.667931    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"]
Jul 27 20:15:10.668310 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.668220    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5822b7b-eff2-4cde-906e-4834ce15a9a1" containerName="cleanup"
Jul 27 20:15:10.668310 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.668230    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5822b7b-eff2-4cde-906e-4834ce15a9a1" containerName="cleanup"
Jul 27 20:15:10.668310 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.668301    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5822b7b-eff2-4cde-906e-4834ce15a9a1" containerName="cleanup"
Jul 27 20:15:10.671176 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.671159    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:10.673973 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.673949    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-849f6b\"/\"kube-root-ca.crt\""
Jul 27 20:15:10.674092 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.673975    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-849f6b\"/\"openshift-service-ca.crt\""
Jul 27 20:15:10.674092 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.673950    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-849f6b\"/\"default-dockercfg-r7mw7\""
Jul 27 20:15:10.674092 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.673950    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-849f6b\"/\"test-model-849f6b-kserve-self-signed-certs\""
Jul 27 20:15:10.680655 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.680632    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"]
Jul 27 20:15:10.809397 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.809359    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/205fdedf-b877-46ec-85c8-010f61ee37b4-tls-certs\") pod \"test-model-849f6b-kserve-7546fb85d7-xtqmv\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") " pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:10.809583 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.809419    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-dshm\") pod \"test-model-849f6b-kserve-7546fb85d7-xtqmv\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") " pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:10.809583 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.809547    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbx8l\" (UniqueName: \"kubernetes.io/projected/205fdedf-b877-46ec-85c8-010f61ee37b4-kube-api-access-jbx8l\") pod \"test-model-849f6b-kserve-7546fb85d7-xtqmv\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") " pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:10.809713 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.809606    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-home\") pod \"test-model-849f6b-kserve-7546fb85d7-xtqmv\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") " pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:10.809713 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.809661    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-model-cache\") pod \"test-model-849f6b-kserve-7546fb85d7-xtqmv\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") " pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:10.809713 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.809701    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-tmp-dir\") pod \"test-model-849f6b-kserve-7546fb85d7-xtqmv\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") " pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:10.910490 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.910458    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/205fdedf-b877-46ec-85c8-010f61ee37b4-tls-certs\") pod \"test-model-849f6b-kserve-7546fb85d7-xtqmv\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") " pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:10.910639 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.910508    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-dshm\") pod \"test-model-849f6b-kserve-7546fb85d7-xtqmv\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") " pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:10.910639 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.910555    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbx8l\" (UniqueName: \"kubernetes.io/projected/205fdedf-b877-46ec-85c8-010f61ee37b4-kube-api-access-jbx8l\") pod \"test-model-849f6b-kserve-7546fb85d7-xtqmv\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") " pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:10.910639 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.910589    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-home\") pod \"test-model-849f6b-kserve-7546fb85d7-xtqmv\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") " pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:10.910639 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.910637    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-model-cache\") pod \"test-model-849f6b-kserve-7546fb85d7-xtqmv\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") " pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:10.910831 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.910672    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-tmp-dir\") pod \"test-model-849f6b-kserve-7546fb85d7-xtqmv\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") " pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:10.911082 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.911058    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-model-cache\") pod \"test-model-849f6b-kserve-7546fb85d7-xtqmv\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") " pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:10.911156 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.911086    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-tmp-dir\") pod \"test-model-849f6b-kserve-7546fb85d7-xtqmv\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") " pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:10.911156 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.911069    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-home\") pod \"test-model-849f6b-kserve-7546fb85d7-xtqmv\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") " pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:10.913015 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.912989    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-dshm\") pod \"test-model-849f6b-kserve-7546fb85d7-xtqmv\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") " pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:10.913273 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.913257    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/205fdedf-b877-46ec-85c8-010f61ee37b4-tls-certs\") pod \"test-model-849f6b-kserve-7546fb85d7-xtqmv\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") " pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:10.918045 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.918002    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbx8l\" (UniqueName: \"kubernetes.io/projected/205fdedf-b877-46ec-85c8-010f61ee37b4-kube-api-access-jbx8l\") pod \"test-model-849f6b-kserve-7546fb85d7-xtqmv\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") " pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:10.979898 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:10.979842    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:11.113750 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:11.113703    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"]
Jul 27 20:15:11.116518 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:15:11.116490    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod205fdedf_b877_46ec_85c8_010f61ee37b4.slice/crio-17caf980e9b48eb0dc1a68ae5e4f9fdd81269be62f2f4459d01e77866ff4cc2d WatchSource:0}: Error finding container 17caf980e9b48eb0dc1a68ae5e4f9fdd81269be62f2f4459d01e77866ff4cc2d: Status 404 returned error can't find the container with id 17caf980e9b48eb0dc1a68ae5e4f9fdd81269be62f2f4459d01e77866ff4cc2d
Jul 27 20:15:11.623730 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:11.623691    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv" event={"ID":"205fdedf-b877-46ec-85c8-010f61ee37b4","Type":"ContainerStarted","Data":"053e2d1ff5b9abeca187fe4e38084b07aa84dcb4661e2757263282cc60baf1eb"}
Jul 27 20:15:11.623730 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:11.623734    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv" event={"ID":"205fdedf-b877-46ec-85c8-010f61ee37b4","Type":"ContainerStarted","Data":"17caf980e9b48eb0dc1a68ae5e4f9fdd81269be62f2f4459d01e77866ff4cc2d"}
Jul 27 20:15:11.641833 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:11.641785    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv" podStartSLOduration=1.641770587 podStartE2EDuration="1.641770587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:15:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:15:11.640899458 +0000 UTC m=+2139.870141656" watchObservedRunningTime="2026-07-27 20:15:11.641770587 +0000 UTC m=+2139.871012784"
Jul 27 20:15:20.980991 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:20.980952    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:20.981435 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:20.981011    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:20.996235 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:20.996207    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:21.680167 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:21.680138    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:15:30.342208 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.342128    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"]
Jul 27 20:15:30.350169 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.350146    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:30.352893 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.352842    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0a5392\"/\"openshift-service-ca.crt\""
Jul 27 20:15:30.353022 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.352845    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0a5392\"/\"default-dockercfg-hlgw4\""
Jul 27 20:15:30.353022 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.352845    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0a5392\"/\"test-model-0a5392-kserve-self-signed-certs\""
Jul 27 20:15:30.353022 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.352845    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0a5392\"/\"kube-root-ca.crt\""
Jul 27 20:15:30.358704 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.358683    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"]
Jul 27 20:15:30.466264 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.466233    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-model-cache\") pod \"test-model-0a5392-kserve-694c4cd4c7-n97gq\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") " pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:30.466377 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.466276    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25271c69-8598-485c-a978-1dafc371bb7d-tls-certs\") pod \"test-model-0a5392-kserve-694c4cd4c7-n97gq\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") " pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:30.466377 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.466310    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-dshm\") pod \"test-model-0a5392-kserve-694c4cd4c7-n97gq\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") " pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:30.466456 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.466368    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-home\") pod \"test-model-0a5392-kserve-694c4cd4c7-n97gq\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") " pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:30.466456 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.466407    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrvtg\" (UniqueName: \"kubernetes.io/projected/25271c69-8598-485c-a978-1dafc371bb7d-kube-api-access-mrvtg\") pod \"test-model-0a5392-kserve-694c4cd4c7-n97gq\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") " pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:30.466456 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.466429    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-tmp-dir\") pod \"test-model-0a5392-kserve-694c4cd4c7-n97gq\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") " pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:30.567311 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.567281    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-model-cache\") pod \"test-model-0a5392-kserve-694c4cd4c7-n97gq\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") " pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:30.567438 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.567316    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25271c69-8598-485c-a978-1dafc371bb7d-tls-certs\") pod \"test-model-0a5392-kserve-694c4cd4c7-n97gq\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") " pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:30.567438 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.567338    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-dshm\") pod \"test-model-0a5392-kserve-694c4cd4c7-n97gq\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") " pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:30.567438 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.567364    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-home\") pod \"test-model-0a5392-kserve-694c4cd4c7-n97gq\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") " pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:30.567438 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.567400    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrvtg\" (UniqueName: \"kubernetes.io/projected/25271c69-8598-485c-a978-1dafc371bb7d-kube-api-access-mrvtg\") pod \"test-model-0a5392-kserve-694c4cd4c7-n97gq\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") " pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:30.567438 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.567428    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-tmp-dir\") pod \"test-model-0a5392-kserve-694c4cd4c7-n97gq\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") " pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:30.567696 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.567642    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-model-cache\") pod \"test-model-0a5392-kserve-694c4cd4c7-n97gq\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") " pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:30.567788 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.567770    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-home\") pod \"test-model-0a5392-kserve-694c4cd4c7-n97gq\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") " pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:30.567838 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.567807    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-tmp-dir\") pod \"test-model-0a5392-kserve-694c4cd4c7-n97gq\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") " pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:30.569609 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.569590    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-dshm\") pod \"test-model-0a5392-kserve-694c4cd4c7-n97gq\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") " pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:30.569851 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.569831    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25271c69-8598-485c-a978-1dafc371bb7d-tls-certs\") pod \"test-model-0a5392-kserve-694c4cd4c7-n97gq\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") " pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:30.575245 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.575229    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrvtg\" (UniqueName: \"kubernetes.io/projected/25271c69-8598-485c-a978-1dafc371bb7d-kube-api-access-mrvtg\") pod \"test-model-0a5392-kserve-694c4cd4c7-n97gq\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") " pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:30.660996 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.660944    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:30.790447 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:30.790426    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"]
Jul 27 20:15:30.792433 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:15:30.792412    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25271c69_8598_485c_a978_1dafc371bb7d.slice/crio-5c3a89800abf1dee17239e6dc5b9e69db3ca49ceff8f809f6687728756427221 WatchSource:0}: Error finding container 5c3a89800abf1dee17239e6dc5b9e69db3ca49ceff8f809f6687728756427221: Status 404 returned error can't find the container with id 5c3a89800abf1dee17239e6dc5b9e69db3ca49ceff8f809f6687728756427221
Jul 27 20:15:31.715411 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:31.715375    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq" event={"ID":"25271c69-8598-485c-a978-1dafc371bb7d","Type":"ContainerStarted","Data":"61295fa27390662211babfca99a7ca0af0a644331e0bb70e293ef34017e1b5ce"}
Jul 27 20:15:31.715816 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:31.715420    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq" event={"ID":"25271c69-8598-485c-a978-1dafc371bb7d","Type":"ContainerStarted","Data":"5c3a89800abf1dee17239e6dc5b9e69db3ca49ceff8f809f6687728756427221"}
Jul 27 20:15:31.732368 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:31.732325    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq" podStartSLOduration=1.732310847 podStartE2EDuration="1.732310847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:15:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:15:31.731192797 +0000 UTC m=+2159.960434994" watchObservedRunningTime="2026-07-27 20:15:31.732310847 +0000 UTC m=+2159.961553045"
Jul 27 20:15:40.661310 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:40.661268    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:40.661310 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:40.661316    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:40.674051 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:40.674019    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:15:40.767816 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:15:40.767787    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:16:11.401404 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:11.401371    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"]
Jul 27 20:16:12.240622 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:12.240584    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"]
Jul 27 20:16:23.026448 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:23.026409    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-849f6b-29753055-b5fk8"]
Jul 27 20:16:23.028726 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:23.028699    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-849f6b-29753055-b5fk8"]
Jul 27 20:16:24.261034 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:24.260989    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5822b7b-eff2-4cde-906e-4834ce15a9a1" path="/var/lib/kubelet/pods/c5822b7b-eff2-4cde-906e-4834ce15a9a1/volumes"
Jul 27 20:16:26.420717 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.420675    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv" podUID="205fdedf-b877-46ec-85c8-010f61ee37b4" containerName="main" containerID="cri-o://053e2d1ff5b9abeca187fe4e38084b07aa84dcb4661e2757263282cc60baf1eb" gracePeriod=45
Jul 27 20:16:26.663634 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.663612    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:16:26.792040 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.791964    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-home\") pod \"205fdedf-b877-46ec-85c8-010f61ee37b4\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") "
Jul 27 20:16:26.792040 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.792015    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/205fdedf-b877-46ec-85c8-010f61ee37b4-tls-certs\") pod \"205fdedf-b877-46ec-85c8-010f61ee37b4\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") "
Jul 27 20:16:26.792227 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.792044    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-dshm\") pod \"205fdedf-b877-46ec-85c8-010f61ee37b4\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") "
Jul 27 20:16:26.792227 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.792060    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-model-cache\") pod \"205fdedf-b877-46ec-85c8-010f61ee37b4\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") "
Jul 27 20:16:26.792227 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.792075    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-tmp-dir\") pod \"205fdedf-b877-46ec-85c8-010f61ee37b4\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") "
Jul 27 20:16:26.792227 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.792140    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbx8l\" (UniqueName: \"kubernetes.io/projected/205fdedf-b877-46ec-85c8-010f61ee37b4-kube-api-access-jbx8l\") pod \"205fdedf-b877-46ec-85c8-010f61ee37b4\" (UID: \"205fdedf-b877-46ec-85c8-010f61ee37b4\") "
Jul 27 20:16:26.792429 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.792263    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-home" (OuterVolumeSpecName: "home") pod "205fdedf-b877-46ec-85c8-010f61ee37b4" (UID: "205fdedf-b877-46ec-85c8-010f61ee37b4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:16:26.792429 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.792342    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "205fdedf-b877-46ec-85c8-010f61ee37b4" (UID: "205fdedf-b877-46ec-85c8-010f61ee37b4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:16:26.792429 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.792362    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-model-cache" (OuterVolumeSpecName: "model-cache") pod "205fdedf-b877-46ec-85c8-010f61ee37b4" (UID: "205fdedf-b877-46ec-85c8-010f61ee37b4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:16:26.792569 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.792442    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-home\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:16:26.792569 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.792463    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-model-cache\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:16:26.792569 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.792478    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-tmp-dir\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:16:26.794434 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.794392    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/205fdedf-b877-46ec-85c8-010f61ee37b4-kube-api-access-jbx8l" (OuterVolumeSpecName: "kube-api-access-jbx8l") pod "205fdedf-b877-46ec-85c8-010f61ee37b4" (UID: "205fdedf-b877-46ec-85c8-010f61ee37b4"). InnerVolumeSpecName "kube-api-access-jbx8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:16:26.794536 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.794464    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-dshm" (OuterVolumeSpecName: "dshm") pod "205fdedf-b877-46ec-85c8-010f61ee37b4" (UID: "205fdedf-b877-46ec-85c8-010f61ee37b4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:16:26.794536 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.794475    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/205fdedf-b877-46ec-85c8-010f61ee37b4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "205fdedf-b877-46ec-85c8-010f61ee37b4" (UID: "205fdedf-b877-46ec-85c8-010f61ee37b4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:16:26.893254 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.893229    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbx8l\" (UniqueName: \"kubernetes.io/projected/205fdedf-b877-46ec-85c8-010f61ee37b4-kube-api-access-jbx8l\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:16:26.893254 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.893251    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/205fdedf-b877-46ec-85c8-010f61ee37b4-tls-certs\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:16:26.893254 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.893261    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/205fdedf-b877-46ec-85c8-010f61ee37b4-dshm\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:16:26.963298 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.963269    2579 generic.go:356] "Generic (PLEG): container finished" podID="205fdedf-b877-46ec-85c8-010f61ee37b4" containerID="053e2d1ff5b9abeca187fe4e38084b07aa84dcb4661e2757263282cc60baf1eb" exitCode=0
Jul 27 20:16:26.963399 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.963304    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv" event={"ID":"205fdedf-b877-46ec-85c8-010f61ee37b4","Type":"ContainerDied","Data":"053e2d1ff5b9abeca187fe4e38084b07aa84dcb4661e2757263282cc60baf1eb"}
Jul 27 20:16:26.963399 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.963313    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"
Jul 27 20:16:26.963399 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.963335    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv" event={"ID":"205fdedf-b877-46ec-85c8-010f61ee37b4","Type":"ContainerDied","Data":"17caf980e9b48eb0dc1a68ae5e4f9fdd81269be62f2f4459d01e77866ff4cc2d"}
Jul 27 20:16:26.963399 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.963352    2579 scope.go:117] "RemoveContainer" containerID="053e2d1ff5b9abeca187fe4e38084b07aa84dcb4661e2757263282cc60baf1eb"
Jul 27 20:16:26.972001 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.971986    2579 scope.go:117] "RemoveContainer" containerID="053e2d1ff5b9abeca187fe4e38084b07aa84dcb4661e2757263282cc60baf1eb"
Jul 27 20:16:26.972256 ip-10-0-140-36 kubenswrapper[2579]: E0727 20:16:26.972239    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"053e2d1ff5b9abeca187fe4e38084b07aa84dcb4661e2757263282cc60baf1eb\": container with ID starting with 053e2d1ff5b9abeca187fe4e38084b07aa84dcb4661e2757263282cc60baf1eb not found: ID does not exist" containerID="053e2d1ff5b9abeca187fe4e38084b07aa84dcb4661e2757263282cc60baf1eb"
Jul 27 20:16:26.972303 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.972265    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"053e2d1ff5b9abeca187fe4e38084b07aa84dcb4661e2757263282cc60baf1eb"} err="failed to get container status \"053e2d1ff5b9abeca187fe4e38084b07aa84dcb4661e2757263282cc60baf1eb\": rpc error: code = NotFound desc = could not find container \"053e2d1ff5b9abeca187fe4e38084b07aa84dcb4661e2757263282cc60baf1eb\": container with ID starting with 053e2d1ff5b9abeca187fe4e38084b07aa84dcb4661e2757263282cc60baf1eb not found: ID does not exist"
Jul 27 20:16:26.984076 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.984019    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"]
Jul 27 20:16:26.985578 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:26.985558    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-849f6b/test-model-849f6b-kserve-7546fb85d7-xtqmv"]
Jul 27 20:16:27.259995 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.259961    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq" podUID="25271c69-8598-485c-a978-1dafc371bb7d" containerName="main" containerID="cri-o://61295fa27390662211babfca99a7ca0af0a644331e0bb70e293ef34017e1b5ce" gracePeriod=45
Jul 27 20:16:27.508803 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.508779    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:16:27.599052 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.598962    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-tmp-dir\") pod \"25271c69-8598-485c-a978-1dafc371bb7d\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") "
Jul 27 20:16:27.599052 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.599015    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-model-cache\") pod \"25271c69-8598-485c-a978-1dafc371bb7d\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") "
Jul 27 20:16:27.599052 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.599048    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-home\") pod \"25271c69-8598-485c-a978-1dafc371bb7d\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") "
Jul 27 20:16:27.599287 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.599087    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-dshm\") pod \"25271c69-8598-485c-a978-1dafc371bb7d\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") "
Jul 27 20:16:27.599287 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.599127    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25271c69-8598-485c-a978-1dafc371bb7d-tls-certs\") pod \"25271c69-8598-485c-a978-1dafc371bb7d\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") "
Jul 27 20:16:27.599287 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.599168    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrvtg\" (UniqueName: \"kubernetes.io/projected/25271c69-8598-485c-a978-1dafc371bb7d-kube-api-access-mrvtg\") pod \"25271c69-8598-485c-a978-1dafc371bb7d\" (UID: \"25271c69-8598-485c-a978-1dafc371bb7d\") "
Jul 27 20:16:27.599417 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.599276    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "25271c69-8598-485c-a978-1dafc371bb7d" (UID: "25271c69-8598-485c-a978-1dafc371bb7d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:16:27.599417 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.599363    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-home" (OuterVolumeSpecName: "home") pod "25271c69-8598-485c-a978-1dafc371bb7d" (UID: "25271c69-8598-485c-a978-1dafc371bb7d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:16:27.599417 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.599381    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-model-cache" (OuterVolumeSpecName: "model-cache") pod "25271c69-8598-485c-a978-1dafc371bb7d" (UID: "25271c69-8598-485c-a978-1dafc371bb7d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:16:27.599573 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.599468    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-home\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:16:27.599573 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.599485    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-tmp-dir\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:16:27.599573 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.599501    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-model-cache\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:16:27.601618 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.601593    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25271c69-8598-485c-a978-1dafc371bb7d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "25271c69-8598-485c-a978-1dafc371bb7d" (UID: "25271c69-8598-485c-a978-1dafc371bb7d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:16:27.601618 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.601606    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25271c69-8598-485c-a978-1dafc371bb7d-kube-api-access-mrvtg" (OuterVolumeSpecName: "kube-api-access-mrvtg") pod "25271c69-8598-485c-a978-1dafc371bb7d" (UID: "25271c69-8598-485c-a978-1dafc371bb7d"). InnerVolumeSpecName "kube-api-access-mrvtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:16:27.601791 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.601643    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-dshm" (OuterVolumeSpecName: "dshm") pod "25271c69-8598-485c-a978-1dafc371bb7d" (UID: "25271c69-8598-485c-a978-1dafc371bb7d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:16:27.700365 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.700338    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25271c69-8598-485c-a978-1dafc371bb7d-dshm\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:16:27.700365 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.700360    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25271c69-8598-485c-a978-1dafc371bb7d-tls-certs\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:16:27.700365 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.700370    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrvtg\" (UniqueName: \"kubernetes.io/projected/25271c69-8598-485c-a978-1dafc371bb7d-kube-api-access-mrvtg\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:16:27.969692 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.969660    2579 generic.go:356] "Generic (PLEG): container finished" podID="25271c69-8598-485c-a978-1dafc371bb7d" containerID="61295fa27390662211babfca99a7ca0af0a644331e0bb70e293ef34017e1b5ce" exitCode=0
Jul 27 20:16:27.969885 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.969711    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"
Jul 27 20:16:27.969885 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.969737    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq" event={"ID":"25271c69-8598-485c-a978-1dafc371bb7d","Type":"ContainerDied","Data":"61295fa27390662211babfca99a7ca0af0a644331e0bb70e293ef34017e1b5ce"}
Jul 27 20:16:27.969885 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.969786    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq" event={"ID":"25271c69-8598-485c-a978-1dafc371bb7d","Type":"ContainerDied","Data":"5c3a89800abf1dee17239e6dc5b9e69db3ca49ceff8f809f6687728756427221"}
Jul 27 20:16:27.969885 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.969813    2579 scope.go:117] "RemoveContainer" containerID="61295fa27390662211babfca99a7ca0af0a644331e0bb70e293ef34017e1b5ce"
Jul 27 20:16:27.983023 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.983005    2579 scope.go:117] "RemoveContainer" containerID="61295fa27390662211babfca99a7ca0af0a644331e0bb70e293ef34017e1b5ce"
Jul 27 20:16:27.983274 ip-10-0-140-36 kubenswrapper[2579]: E0727 20:16:27.983254    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61295fa27390662211babfca99a7ca0af0a644331e0bb70e293ef34017e1b5ce\": container with ID starting with 61295fa27390662211babfca99a7ca0af0a644331e0bb70e293ef34017e1b5ce not found: ID does not exist" containerID="61295fa27390662211babfca99a7ca0af0a644331e0bb70e293ef34017e1b5ce"
Jul 27 20:16:27.983340 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.983281    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61295fa27390662211babfca99a7ca0af0a644331e0bb70e293ef34017e1b5ce"} err="failed to get container status \"61295fa27390662211babfca99a7ca0af0a644331e0bb70e293ef34017e1b5ce\": rpc error: code = NotFound desc = could not find container \"61295fa27390662211babfca99a7ca0af0a644331e0bb70e293ef34017e1b5ce\": container with ID starting with 61295fa27390662211babfca99a7ca0af0a644331e0bb70e293ef34017e1b5ce not found: ID does not exist"
Jul 27 20:16:27.996523 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:27.996497    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"]
Jul 27 20:16:28.005617 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:28.005595    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-0a5392/test-model-0a5392-kserve-694c4cd4c7-n97gq"]
Jul 27 20:16:28.261769 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:28.261701    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="205fdedf-b877-46ec-85c8-010f61ee37b4" path="/var/lib/kubelet/pods/205fdedf-b877-46ec-85c8-010f61ee37b4/volumes"
Jul 27 20:16:28.262114 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:28.262100    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25271c69-8598-485c-a978-1dafc371bb7d" path="/var/lib/kubelet/pods/25271c69-8598-485c-a978-1dafc371bb7d/volumes"
Jul 27 20:16:34.490001 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:34.489964    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a5392-d2tkh"]
Jul 27 20:16:34.490377 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:34.490315    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25271c69-8598-485c-a978-1dafc371bb7d" containerName="main"
Jul 27 20:16:34.490377 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:34.490326    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="25271c69-8598-485c-a978-1dafc371bb7d" containerName="main"
Jul 27 20:16:34.490377 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:34.490342    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="205fdedf-b877-46ec-85c8-010f61ee37b4" containerName="main"
Jul 27 20:16:34.490377 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:34.490352    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="205fdedf-b877-46ec-85c8-010f61ee37b4" containerName="main"
Jul 27 20:16:34.490498 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:34.490408    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="25271c69-8598-485c-a978-1dafc371bb7d" containerName="main"
Jul 27 20:16:34.490498 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:34.490418    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="205fdedf-b877-46ec-85c8-010f61ee37b4" containerName="main"
Jul 27 20:16:34.494756 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:34.494740    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a5392-d2tkh"
Jul 27 20:16:34.497004 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:34.496979    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 20:16:34.497163 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:34.497146    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x6cbr\""
Jul 27 20:16:34.502497 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:34.502470    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a5392-d2tkh"]
Jul 27 20:16:34.659033 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:34.658994    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e0d6237-6a0e-4217-b12a-fb3ba1435327-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0a5392-d2tkh\" (UID: \"4e0d6237-6a0e-4217-b12a-fb3ba1435327\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a5392-d2tkh"
Jul 27 20:16:34.759606 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:34.759526    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e0d6237-6a0e-4217-b12a-fb3ba1435327-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0a5392-d2tkh\" (UID: \"4e0d6237-6a0e-4217-b12a-fb3ba1435327\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a5392-d2tkh"
Jul 27 20:16:34.760227 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:34.760209    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e0d6237-6a0e-4217-b12a-fb3ba1435327-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0a5392-d2tkh\" (UID: \"4e0d6237-6a0e-4217-b12a-fb3ba1435327\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a5392-d2tkh"
Jul 27 20:16:34.805720 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:34.805682    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a5392-d2tkh"
Jul 27 20:16:34.935787 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:34.935762    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a5392-d2tkh"]
Jul 27 20:16:34.938275 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:16:34.938247    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e0d6237_6a0e_4217_b12a_fb3ba1435327.slice/crio-01a96d01ab682eb5c62f3ef346576bcf40d8c9d03ea3af99eecca2614e817b7b WatchSource:0}: Error finding container 01a96d01ab682eb5c62f3ef346576bcf40d8c9d03ea3af99eecca2614e817b7b: Status 404 returned error can't find the container with id 01a96d01ab682eb5c62f3ef346576bcf40d8c9d03ea3af99eecca2614e817b7b
Jul 27 20:16:35.004402 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:35.004373    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a5392-d2tkh" event={"ID":"4e0d6237-6a0e-4217-b12a-fb3ba1435327","Type":"ContainerStarted","Data":"01a96d01ab682eb5c62f3ef346576bcf40d8c9d03ea3af99eecca2614e817b7b"}
Jul 27 20:16:36.010938 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:36.010901    2579 generic.go:356] "Generic (PLEG): container finished" podID="4e0d6237-6a0e-4217-b12a-fb3ba1435327" containerID="7ce46b23dd55902a5e3cbe296a2014f34103e885179c7f22eeda5d3bc5411609" exitCode=0
Jul 27 20:16:36.011299 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:36.010985    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a5392-d2tkh" event={"ID":"4e0d6237-6a0e-4217-b12a-fb3ba1435327","Type":"ContainerDied","Data":"7ce46b23dd55902a5e3cbe296a2014f34103e885179c7f22eeda5d3bc5411609"}
Jul 27 20:16:37.138998 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:37.138974    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a5392-d2tkh"
Jul 27 20:16:37.284152 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:37.284054    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e0d6237-6a0e-4217-b12a-fb3ba1435327-maas-api-service-ca\") pod \"4e0d6237-6a0e-4217-b12a-fb3ba1435327\" (UID: \"4e0d6237-6a0e-4217-b12a-fb3ba1435327\") "
Jul 27 20:16:37.284483 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:37.284451    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e0d6237-6a0e-4217-b12a-fb3ba1435327-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4e0d6237-6a0e-4217-b12a-fb3ba1435327" (UID: "4e0d6237-6a0e-4217-b12a-fb3ba1435327"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:16:37.385468 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:37.385431    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e0d6237-6a0e-4217-b12a-fb3ba1435327-maas-api-service-ca\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:16:38.021511 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:38.021479    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a5392-d2tkh" event={"ID":"4e0d6237-6a0e-4217-b12a-fb3ba1435327","Type":"ContainerDied","Data":"01a96d01ab682eb5c62f3ef346576bcf40d8c9d03ea3af99eecca2614e817b7b"}
Jul 27 20:16:38.021678 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:38.021525    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="01a96d01ab682eb5c62f3ef346576bcf40d8c9d03ea3af99eecca2614e817b7b"
Jul 27 20:16:38.021678 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:38.021490    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a5392-d2tkh"
Jul 27 20:16:49.596844 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:49.596811    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a5392-d2tkh"]
Jul 27 20:16:49.609579 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:49.609553    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a5392-d2tkh"]
Jul 27 20:16:50.261972 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:16:50.261939    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e0d6237-6a0e-4217-b12a-fb3ba1435327" path="/var/lib/kubelet/pods/4e0d6237-6a0e-4217-b12a-fb3ba1435327/volumes"
Jul 27 20:17:20.965018 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:20.964983    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d453a8-flmkn"]
Jul 27 20:17:20.965564 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:20.965427    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e0d6237-6a0e-4217-b12a-fb3ba1435327" containerName="revoke-keys"
Jul 27 20:17:20.965564 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:20.965447    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e0d6237-6a0e-4217-b12a-fb3ba1435327" containerName="revoke-keys"
Jul 27 20:17:20.965564 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:20.965555    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e0d6237-6a0e-4217-b12a-fb3ba1435327" containerName="revoke-keys"
Jul 27 20:17:20.968828 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:20.968806    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d453a8-flmkn"
Jul 27 20:17:20.970992 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:20.970972    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 20:17:20.971176 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:20.971159    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x6cbr\""
Jul 27 20:17:20.978031 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:20.978009    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d453a8-flmkn"]
Jul 27 20:17:21.006393 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:21.006363    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a93f6fa4-7959-40c7-b891-7825dccbdf0a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d453a8-flmkn\" (UID: \"a93f6fa4-7959-40c7-b891-7825dccbdf0a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d453a8-flmkn"
Jul 27 20:17:21.107549 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:21.107512    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a93f6fa4-7959-40c7-b891-7825dccbdf0a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d453a8-flmkn\" (UID: \"a93f6fa4-7959-40c7-b891-7825dccbdf0a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d453a8-flmkn"
Jul 27 20:17:21.108117 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:21.108101    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a93f6fa4-7959-40c7-b891-7825dccbdf0a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d453a8-flmkn\" (UID: \"a93f6fa4-7959-40c7-b891-7825dccbdf0a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d453a8-flmkn"
Jul 27 20:17:21.279752 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:21.279676    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d453a8-flmkn"
Jul 27 20:17:21.407367 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:21.407225    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d453a8-flmkn"]
Jul 27 20:17:21.410083 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:17:21.410052    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda93f6fa4_7959_40c7_b891_7825dccbdf0a.slice/crio-ce76cade6e81868cfe30d728bda2d1d94aa29a62c2c92af12bc4c14daaed8895 WatchSource:0}: Error finding container ce76cade6e81868cfe30d728bda2d1d94aa29a62c2c92af12bc4c14daaed8895: Status 404 returned error can't find the container with id ce76cade6e81868cfe30d728bda2d1d94aa29a62c2c92af12bc4c14daaed8895
Jul 27 20:17:22.220815 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:22.220785    2579 generic.go:356] "Generic (PLEG): container finished" podID="a93f6fa4-7959-40c7-b891-7825dccbdf0a" containerID="edd9820fb93fc43cd9f5c7cf285913275ea486954edcfaeed6f472efc028f380" exitCode=0
Jul 27 20:17:22.221171 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:22.220891    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d453a8-flmkn" event={"ID":"a93f6fa4-7959-40c7-b891-7825dccbdf0a","Type":"ContainerDied","Data":"edd9820fb93fc43cd9f5c7cf285913275ea486954edcfaeed6f472efc028f380"}
Jul 27 20:17:22.221171 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:22.220957    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d453a8-flmkn" event={"ID":"a93f6fa4-7959-40c7-b891-7825dccbdf0a","Type":"ContainerStarted","Data":"ce76cade6e81868cfe30d728bda2d1d94aa29a62c2c92af12bc4c14daaed8895"}
Jul 27 20:17:23.355980 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:23.355955    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d453a8-flmkn"
Jul 27 20:17:23.424921 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:23.424875    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a93f6fa4-7959-40c7-b891-7825dccbdf0a-maas-api-service-ca\") pod \"a93f6fa4-7959-40c7-b891-7825dccbdf0a\" (UID: \"a93f6fa4-7959-40c7-b891-7825dccbdf0a\") "
Jul 27 20:17:23.425232 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:23.425205    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a93f6fa4-7959-40c7-b891-7825dccbdf0a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a93f6fa4-7959-40c7-b891-7825dccbdf0a" (UID: "a93f6fa4-7959-40c7-b891-7825dccbdf0a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:17:23.526392 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:23.526339    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a93f6fa4-7959-40c7-b891-7825dccbdf0a-maas-api-service-ca\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:17:24.231445 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:24.231417    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d453a8-flmkn"
Jul 27 20:17:24.231630 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:24.231409    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d453a8-flmkn" event={"ID":"a93f6fa4-7959-40c7-b891-7825dccbdf0a","Type":"ContainerDied","Data":"ce76cade6e81868cfe30d728bda2d1d94aa29a62c2c92af12bc4c14daaed8895"}
Jul 27 20:17:24.231630 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:24.231554    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce76cade6e81868cfe30d728bda2d1d94aa29a62c2c92af12bc4c14daaed8895"
Jul 27 20:17:41.055163 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:41.055129    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d453a8-flmkn"]
Jul 27 20:17:41.061533 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:41.061509    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d453a8-flmkn"]
Jul 27 20:17:42.261587 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:17:42.261552    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a93f6fa4-7959-40c7-b891-7825dccbdf0a" path="/var/lib/kubelet/pods/a93f6fa4-7959-40c7-b891-7825dccbdf0a/volumes"
Jul 27 20:18:55.237038 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.236995    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"]
Jul 27 20:18:55.237595 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.237387    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a93f6fa4-7959-40c7-b891-7825dccbdf0a" containerName="revoke-keys"
Jul 27 20:18:55.237595 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.237402    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a93f6fa4-7959-40c7-b891-7825dccbdf0a" containerName="revoke-keys"
Jul 27 20:18:55.237595 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.237460    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a93f6fa4-7959-40c7-b891-7825dccbdf0a" containerName="revoke-keys"
Jul 27 20:18:55.240372 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.240353    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:18:55.242196 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.242169    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c3a53e\"/\"openshift-service-ca.crt\""
Jul 27 20:18:55.242771 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.242740    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c3a53e\"/\"kube-root-ca.crt\""
Jul 27 20:18:55.242875 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.242809    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c3a53e\"/\"default-dockercfg-bcpqh\""
Jul 27 20:18:55.242942 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.242847    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c3a53e\"/\"auth-test-model-c3a53e-kserve-self-signed-certs\""
Jul 27 20:18:55.249159 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.249137    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"]
Jul 27 20:18:55.316100 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.316074    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-dshm\") pod \"auth-test-model-c3a53e-kserve-694b9cf77f-msxxc\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") " pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:18:55.316233 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.316108    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5821d826-ddc6-48ed-8da0-c7ac28d01e98-tls-certs\") pod \"auth-test-model-c3a53e-kserve-694b9cf77f-msxxc\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") " pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:18:55.316233 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.316142    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-tmp-dir\") pod \"auth-test-model-c3a53e-kserve-694b9cf77f-msxxc\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") " pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:18:55.316351 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.316299    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-model-cache\") pod \"auth-test-model-c3a53e-kserve-694b9cf77f-msxxc\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") " pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:18:55.316407 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.316365    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfwvm\" (UniqueName: \"kubernetes.io/projected/5821d826-ddc6-48ed-8da0-c7ac28d01e98-kube-api-access-nfwvm\") pod \"auth-test-model-c3a53e-kserve-694b9cf77f-msxxc\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") " pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:18:55.316458 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.316425    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-home\") pod \"auth-test-model-c3a53e-kserve-694b9cf77f-msxxc\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") " pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:18:55.417169 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.417143    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-home\") pod \"auth-test-model-c3a53e-kserve-694b9cf77f-msxxc\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") " pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:18:55.417268 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.417170    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-dshm\") pod \"auth-test-model-c3a53e-kserve-694b9cf77f-msxxc\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") " pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:18:55.417268 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.417189    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5821d826-ddc6-48ed-8da0-c7ac28d01e98-tls-certs\") pod \"auth-test-model-c3a53e-kserve-694b9cf77f-msxxc\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") " pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:18:55.417344 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.417301    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-tmp-dir\") pod \"auth-test-model-c3a53e-kserve-694b9cf77f-msxxc\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") " pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:18:55.417396 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.417367    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-model-cache\") pod \"auth-test-model-c3a53e-kserve-694b9cf77f-msxxc\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") " pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:18:55.417452 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.417408    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfwvm\" (UniqueName: \"kubernetes.io/projected/5821d826-ddc6-48ed-8da0-c7ac28d01e98-kube-api-access-nfwvm\") pod \"auth-test-model-c3a53e-kserve-694b9cf77f-msxxc\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") " pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:18:55.417571 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.417550    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-home\") pod \"auth-test-model-c3a53e-kserve-694b9cf77f-msxxc\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") " pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:18:55.417657 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.417639    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-tmp-dir\") pod \"auth-test-model-c3a53e-kserve-694b9cf77f-msxxc\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") " pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:18:55.417712 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.417670    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-model-cache\") pod \"auth-test-model-c3a53e-kserve-694b9cf77f-msxxc\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") " pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:18:55.419474 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.419454    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-dshm\") pod \"auth-test-model-c3a53e-kserve-694b9cf77f-msxxc\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") " pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:18:55.419749 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.419734    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5821d826-ddc6-48ed-8da0-c7ac28d01e98-tls-certs\") pod \"auth-test-model-c3a53e-kserve-694b9cf77f-msxxc\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") " pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:18:55.424659 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.424637    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfwvm\" (UniqueName: \"kubernetes.io/projected/5821d826-ddc6-48ed-8da0-c7ac28d01e98-kube-api-access-nfwvm\") pod \"auth-test-model-c3a53e-kserve-694b9cf77f-msxxc\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") " pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:18:55.550735 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.550670    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:18:55.694151 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:55.694126    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"]
Jul 27 20:18:55.696636 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:18:55.696608    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5821d826_ddc6_48ed_8da0_c7ac28d01e98.slice/crio-df1486bdac23a623079c43cb039adac3802fa72eb536be26bacbebd32e10174f WatchSource:0}: Error finding container df1486bdac23a623079c43cb039adac3802fa72eb536be26bacbebd32e10174f: Status 404 returned error can't find the container with id df1486bdac23a623079c43cb039adac3802fa72eb536be26bacbebd32e10174f
Jul 27 20:18:56.624782 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:56.624738    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc" event={"ID":"5821d826-ddc6-48ed-8da0-c7ac28d01e98","Type":"ContainerStarted","Data":"e19a7f8ea247d4e98acadbb7c55097b802e28c6a76bf57930ceecbf4806242e2"}
Jul 27 20:18:56.625158 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:56.624792    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc" event={"ID":"5821d826-ddc6-48ed-8da0-c7ac28d01e98","Type":"ContainerStarted","Data":"df1486bdac23a623079c43cb039adac3802fa72eb536be26bacbebd32e10174f"}
Jul 27 20:18:56.642193 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:18:56.642145    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc" podStartSLOduration=1.642131902 podStartE2EDuration="1.642131902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:18:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:18:56.640244949 +0000 UTC m=+2364.869487148" watchObservedRunningTime="2026-07-27 20:18:56.642131902 +0000 UTC m=+2364.871374100"
Jul 27 20:19:05.550910 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:05.550847    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:19:05.551390 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:05.550929    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:19:05.563500 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:05.563478    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:19:05.675082 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:05.675054    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:19:13.035644 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.035613    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"]
Jul 27 20:19:13.038923 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.038907    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:13.041604 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.041568    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f44149\"/\"openshift-service-ca.crt\""
Jul 27 20:19:13.041727 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.041573    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f44149\"/\"auth-test-model-f44149-kserve-self-signed-certs\""
Jul 27 20:19:13.041727 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.041622    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f44149\"/\"kube-root-ca.crt\""
Jul 27 20:19:13.041727 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.041699    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f44149\"/\"default-dockercfg-s65vc\""
Jul 27 20:19:13.047724 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.047696    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"]
Jul 27 20:19:13.154615 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.154591    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-dshm\") pod \"auth-test-model-f44149-kserve-78487cdff5-29k96\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") " pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:13.154734 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.154627    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-model-cache\") pod \"auth-test-model-f44149-kserve-78487cdff5-29k96\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") " pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:13.154734 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.154646    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfgxs\" (UniqueName: \"kubernetes.io/projected/81a9e12a-f557-4dfb-aaba-6b4c7c047042-kube-api-access-gfgxs\") pod \"auth-test-model-f44149-kserve-78487cdff5-29k96\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") " pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:13.154734 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.154677    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-tmp-dir\") pod \"auth-test-model-f44149-kserve-78487cdff5-29k96\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") " pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:13.154734 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.154730    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-home\") pod \"auth-test-model-f44149-kserve-78487cdff5-29k96\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") " pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:13.154922 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.154755    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81a9e12a-f557-4dfb-aaba-6b4c7c047042-tls-certs\") pod \"auth-test-model-f44149-kserve-78487cdff5-29k96\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") " pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:13.255092 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.255063    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-dshm\") pod \"auth-test-model-f44149-kserve-78487cdff5-29k96\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") " pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:13.255211 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.255108    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-model-cache\") pod \"auth-test-model-f44149-kserve-78487cdff5-29k96\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") " pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:13.255211 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.255132    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfgxs\" (UniqueName: \"kubernetes.io/projected/81a9e12a-f557-4dfb-aaba-6b4c7c047042-kube-api-access-gfgxs\") pod \"auth-test-model-f44149-kserve-78487cdff5-29k96\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") " pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:13.255211 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.255176    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-tmp-dir\") pod \"auth-test-model-f44149-kserve-78487cdff5-29k96\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") " pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:13.255372 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.255235    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-home\") pod \"auth-test-model-f44149-kserve-78487cdff5-29k96\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") " pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:13.255372 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.255272    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81a9e12a-f557-4dfb-aaba-6b4c7c047042-tls-certs\") pod \"auth-test-model-f44149-kserve-78487cdff5-29k96\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") " pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:13.255517 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.255497    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-model-cache\") pod \"auth-test-model-f44149-kserve-78487cdff5-29k96\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") " pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:13.255577 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.255542    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-home\") pod \"auth-test-model-f44149-kserve-78487cdff5-29k96\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") " pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:13.255648 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.255622    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-tmp-dir\") pod \"auth-test-model-f44149-kserve-78487cdff5-29k96\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") " pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:13.257497 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.257475    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-dshm\") pod \"auth-test-model-f44149-kserve-78487cdff5-29k96\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") " pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:13.257877 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.257844    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81a9e12a-f557-4dfb-aaba-6b4c7c047042-tls-certs\") pod \"auth-test-model-f44149-kserve-78487cdff5-29k96\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") " pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:13.263105 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.263087    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfgxs\" (UniqueName: \"kubernetes.io/projected/81a9e12a-f557-4dfb-aaba-6b4c7c047042-kube-api-access-gfgxs\") pod \"auth-test-model-f44149-kserve-78487cdff5-29k96\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") " pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:13.349132 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.349069    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:13.481380 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.481351    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"]
Jul 27 20:19:13.484497 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:19:13.484469    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81a9e12a_f557_4dfb_aaba_6b4c7c047042.slice/crio-fb70831b83339807a3d9c850af32b2cc419619035a4839634beabef36e2be6e5 WatchSource:0}: Error finding container fb70831b83339807a3d9c850af32b2cc419619035a4839634beabef36e2be6e5: Status 404 returned error can't find the container with id fb70831b83339807a3d9c850af32b2cc419619035a4839634beabef36e2be6e5
Jul 27 20:19:13.701582 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.701551    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96" event={"ID":"81a9e12a-f557-4dfb-aaba-6b4c7c047042","Type":"ContainerStarted","Data":"057a46b766b7ffd24613195ee8f09e773936ac9578d3e091647f0b8cf5267fb4"}
Jul 27 20:19:13.701731 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.701589    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96" event={"ID":"81a9e12a-f557-4dfb-aaba-6b4c7c047042","Type":"ContainerStarted","Data":"fb70831b83339807a3d9c850af32b2cc419619035a4839634beabef36e2be6e5"}
Jul 27 20:19:13.718612 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:13.718558    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96" podStartSLOduration=0.718545542 podStartE2EDuration="718.545542ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:19:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:19:13.717524103 +0000 UTC m=+2381.946766303" watchObservedRunningTime="2026-07-27 20:19:13.718545542 +0000 UTC m=+2381.947788130"
Jul 27 20:19:23.349664 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:23.349630    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:23.350057 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:23.349685    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:23.361888 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:23.361848    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:23.758076 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:23.758046    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:19:32.400480 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:32.400453    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 20:19:32.414415 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:19:32.414386    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 20:21:32.479194 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:21:32.479127    2579 scope.go:117] "RemoveContainer" containerID="8450ae80ab1777e7a2c5191e3119a773ee3a1953e24a2c6ddfc329968eef69c9"
Jul 27 20:23:32.506719 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:23:32.506691    2579 scope.go:117] "RemoveContainer" containerID="edd9820fb93fc43cd9f5c7cf285913275ea486954edcfaeed6f472efc028f380"
Jul 27 20:23:32.514968 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:23:32.514949    2579 scope.go:117] "RemoveContainer" containerID="7ce46b23dd55902a5e3cbe296a2014f34103e885179c7f22eeda5d3bc5411609"
Jul 27 20:24:00.550554 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:00.550520    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"]
Jul 27 20:24:01.553445 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:01.553415    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"]
Jul 27 20:24:09.941584 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:09.941544    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"]
Jul 27 20:24:09.950590 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:09.950563    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:09.952776 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:09.952750    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c3a53e\"/\"sub-test-model-c3a53e-kserve-self-signed-certs\""
Jul 27 20:24:09.953966 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:09.953942    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"]
Jul 27 20:24:09.998990 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:09.998965    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-home\") pod \"sub-test-model-c3a53e-kserve-56c45574dc-lrcmt\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") " pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:09.999091 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:09.998995    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-dshm\") pod \"sub-test-model-c3a53e-kserve-56c45574dc-lrcmt\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") " pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:09.999091 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:09.999026    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-tmp-dir\") pod \"sub-test-model-c3a53e-kserve-56c45574dc-lrcmt\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") " pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:09.999166 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:09.999110    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-tls-certs\") pod \"sub-test-model-c3a53e-kserve-56c45574dc-lrcmt\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") " pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:09.999166 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:09.999146    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-model-cache\") pod \"sub-test-model-c3a53e-kserve-56c45574dc-lrcmt\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") " pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:09.999236 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:09.999182    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjr6l\" (UniqueName: \"kubernetes.io/projected/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-kube-api-access-zjr6l\") pod \"sub-test-model-c3a53e-kserve-56c45574dc-lrcmt\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") " pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:10.100073 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:10.100043    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-tls-certs\") pod \"sub-test-model-c3a53e-kserve-56c45574dc-lrcmt\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") " pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:10.100200 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:10.100081    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-model-cache\") pod \"sub-test-model-c3a53e-kserve-56c45574dc-lrcmt\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") " pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:10.100200 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:10.100125    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjr6l\" (UniqueName: \"kubernetes.io/projected/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-kube-api-access-zjr6l\") pod \"sub-test-model-c3a53e-kserve-56c45574dc-lrcmt\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") " pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:10.100314 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:10.100290    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-home\") pod \"sub-test-model-c3a53e-kserve-56c45574dc-lrcmt\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") " pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:10.100368 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:10.100332    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-dshm\") pod \"sub-test-model-c3a53e-kserve-56c45574dc-lrcmt\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") " pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:10.100428 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:10.100368    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-tmp-dir\") pod \"sub-test-model-c3a53e-kserve-56c45574dc-lrcmt\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") " pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:10.100551 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:10.100531    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-model-cache\") pod \"sub-test-model-c3a53e-kserve-56c45574dc-lrcmt\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") " pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:10.100656 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:10.100634    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-home\") pod \"sub-test-model-c3a53e-kserve-56c45574dc-lrcmt\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") " pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:10.100758 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:10.100741    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-tmp-dir\") pod \"sub-test-model-c3a53e-kserve-56c45574dc-lrcmt\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") " pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:10.102614 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:10.102593    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-dshm\") pod \"sub-test-model-c3a53e-kserve-56c45574dc-lrcmt\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") " pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:10.102688 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:10.102674    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-tls-certs\") pod \"sub-test-model-c3a53e-kserve-56c45574dc-lrcmt\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") " pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:10.114992 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:10.114974    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjr6l\" (UniqueName: \"kubernetes.io/projected/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-kube-api-access-zjr6l\") pod \"sub-test-model-c3a53e-kserve-56c45574dc-lrcmt\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") " pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:10.260661 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:10.260594    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:10.390545 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:10.390523    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"]
Jul 27 20:24:10.392566 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:24:10.392540    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ca94e3c_6bfb_44bd_84b2_d1fc5bb870ed.slice/crio-16c00c7e4c16e5dd52dba43644b16f629ba9f370f472d577f5e5eea3eb0b5389 WatchSource:0}: Error finding container 16c00c7e4c16e5dd52dba43644b16f629ba9f370f472d577f5e5eea3eb0b5389: Status 404 returned error can't find the container with id 16c00c7e4c16e5dd52dba43644b16f629ba9f370f472d577f5e5eea3eb0b5389
Jul 27 20:24:11.031639 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:11.031606    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt" event={"ID":"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed","Type":"ContainerStarted","Data":"5af731318f8413a6447359b9cb682a2a9d22473e537963674499def302b78d89"}
Jul 27 20:24:11.031639 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:11.031646    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt" event={"ID":"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed","Type":"ContainerStarted","Data":"16c00c7e4c16e5dd52dba43644b16f629ba9f370f472d577f5e5eea3eb0b5389"}
Jul 27 20:24:11.046320 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:11.046276    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt" podStartSLOduration=2.046261517 podStartE2EDuration="2.046261517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:24:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:24:11.045340203 +0000 UTC m=+2679.274582406" watchObservedRunningTime="2026-07-27 20:24:11.046261517 +0000 UTC m=+2679.275503714"
Jul 27 20:24:15.568902 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:15.568826    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc" podUID="5821d826-ddc6-48ed-8da0-c7ac28d01e98" containerName="main" containerID="cri-o://e19a7f8ea247d4e98acadbb7c55097b802e28c6a76bf57930ceecbf4806242e2" gracePeriod=45
Jul 27 20:24:15.664900 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:15.664848    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc" podUID="5821d826-ddc6-48ed-8da0-c7ac28d01e98" containerName="main" probeResult="failure" output="Get \"https://10.132.0.42:8000/ready\": dial tcp 10.132.0.42:8000: connect: connection refused"
Jul 27 20:24:15.819360 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:15.819305    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:24:15.851161 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:15.851131    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-model-cache\") pod \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") "
Jul 27 20:24:15.851298 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:15.851177    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5821d826-ddc6-48ed-8da0-c7ac28d01e98-tls-certs\") pod \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") "
Jul 27 20:24:15.851298 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:15.851214    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-tmp-dir\") pod \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") "
Jul 27 20:24:15.851298 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:15.851265    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-home\") pod \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") "
Jul 27 20:24:15.851471 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:15.851408    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-model-cache" (OuterVolumeSpecName: "model-cache") pod "5821d826-ddc6-48ed-8da0-c7ac28d01e98" (UID: "5821d826-ddc6-48ed-8da0-c7ac28d01e98"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:24:15.851471 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:15.851446    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-dshm\") pod \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") "
Jul 27 20:24:15.851574 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:15.851520    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-home" (OuterVolumeSpecName: "home") pod "5821d826-ddc6-48ed-8da0-c7ac28d01e98" (UID: "5821d826-ddc6-48ed-8da0-c7ac28d01e98"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:24:15.851574 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:15.851537    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5821d826-ddc6-48ed-8da0-c7ac28d01e98" (UID: "5821d826-ddc6-48ed-8da0-c7ac28d01e98"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:24:15.851574 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:15.851547    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfwvm\" (UniqueName: \"kubernetes.io/projected/5821d826-ddc6-48ed-8da0-c7ac28d01e98-kube-api-access-nfwvm\") pod \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\" (UID: \"5821d826-ddc6-48ed-8da0-c7ac28d01e98\") "
Jul 27 20:24:15.851997 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:15.851929    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-model-cache\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:24:15.851997 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:15.851955    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-tmp-dir\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:24:15.851997 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:15.851968    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-home\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:24:15.853629 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:15.853603    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5821d826-ddc6-48ed-8da0-c7ac28d01e98-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5821d826-ddc6-48ed-8da0-c7ac28d01e98" (UID: "5821d826-ddc6-48ed-8da0-c7ac28d01e98"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:24:15.853720 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:15.853605    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-dshm" (OuterVolumeSpecName: "dshm") pod "5821d826-ddc6-48ed-8da0-c7ac28d01e98" (UID: "5821d826-ddc6-48ed-8da0-c7ac28d01e98"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:24:15.853811 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:15.853793    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5821d826-ddc6-48ed-8da0-c7ac28d01e98-kube-api-access-nfwvm" (OuterVolumeSpecName: "kube-api-access-nfwvm") pod "5821d826-ddc6-48ed-8da0-c7ac28d01e98" (UID: "5821d826-ddc6-48ed-8da0-c7ac28d01e98"). InnerVolumeSpecName "kube-api-access-nfwvm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:24:15.952480 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:15.952460    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfwvm\" (UniqueName: \"kubernetes.io/projected/5821d826-ddc6-48ed-8da0-c7ac28d01e98-kube-api-access-nfwvm\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:24:15.952582 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:15.952482    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5821d826-ddc6-48ed-8da0-c7ac28d01e98-tls-certs\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:24:15.952582 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:15.952492    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5821d826-ddc6-48ed-8da0-c7ac28d01e98-dshm\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:24:16.055193 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.055165    2579 generic.go:356] "Generic (PLEG): container finished" podID="5821d826-ddc6-48ed-8da0-c7ac28d01e98" containerID="e19a7f8ea247d4e98acadbb7c55097b802e28c6a76bf57930ceecbf4806242e2" exitCode=0
Jul 27 20:24:16.055316 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.055216    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"
Jul 27 20:24:16.055316 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.055262    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc" event={"ID":"5821d826-ddc6-48ed-8da0-c7ac28d01e98","Type":"ContainerDied","Data":"e19a7f8ea247d4e98acadbb7c55097b802e28c6a76bf57930ceecbf4806242e2"}
Jul 27 20:24:16.055316 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.055310    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc" event={"ID":"5821d826-ddc6-48ed-8da0-c7ac28d01e98","Type":"ContainerDied","Data":"df1486bdac23a623079c43cb039adac3802fa72eb536be26bacbebd32e10174f"}
Jul 27 20:24:16.055485 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.055334    2579 scope.go:117] "RemoveContainer" containerID="e19a7f8ea247d4e98acadbb7c55097b802e28c6a76bf57930ceecbf4806242e2"
Jul 27 20:24:16.064643 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.064629    2579 scope.go:117] "RemoveContainer" containerID="e19a7f8ea247d4e98acadbb7c55097b802e28c6a76bf57930ceecbf4806242e2"
Jul 27 20:24:16.064930 ip-10-0-140-36 kubenswrapper[2579]: E0727 20:24:16.064912    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e19a7f8ea247d4e98acadbb7c55097b802e28c6a76bf57930ceecbf4806242e2\": container with ID starting with e19a7f8ea247d4e98acadbb7c55097b802e28c6a76bf57930ceecbf4806242e2 not found: ID does not exist" containerID="e19a7f8ea247d4e98acadbb7c55097b802e28c6a76bf57930ceecbf4806242e2"
Jul 27 20:24:16.064988 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.064940    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e19a7f8ea247d4e98acadbb7c55097b802e28c6a76bf57930ceecbf4806242e2"} err="failed to get container status \"e19a7f8ea247d4e98acadbb7c55097b802e28c6a76bf57930ceecbf4806242e2\": rpc error: code = NotFound desc = could not find container \"e19a7f8ea247d4e98acadbb7c55097b802e28c6a76bf57930ceecbf4806242e2\": container with ID starting with e19a7f8ea247d4e98acadbb7c55097b802e28c6a76bf57930ceecbf4806242e2 not found: ID does not exist"
Jul 27 20:24:16.077948 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.077901    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"]
Jul 27 20:24:16.083203 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.083185    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c3a53e/auth-test-model-c3a53e-kserve-694b9cf77f-msxxc"]
Jul 27 20:24:16.261098 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.261069    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5821d826-ddc6-48ed-8da0-c7ac28d01e98" path="/var/lib/kubelet/pods/5821d826-ddc6-48ed-8da0-c7ac28d01e98/volumes"
Jul 27 20:24:16.572578 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.572516    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96" podUID="81a9e12a-f557-4dfb-aaba-6b4c7c047042" containerName="main" containerID="cri-o://057a46b766b7ffd24613195ee8f09e773936ac9578d3e091647f0b8cf5267fb4" gracePeriod=45
Jul 27 20:24:16.818597 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.818576    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:24:16.859018 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.858960    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-dshm\") pod \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") "
Jul 27 20:24:16.859018 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.859000    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfgxs\" (UniqueName: \"kubernetes.io/projected/81a9e12a-f557-4dfb-aaba-6b4c7c047042-kube-api-access-gfgxs\") pod \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") "
Jul 27 20:24:16.859189 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.859036    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-model-cache\") pod \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") "
Jul 27 20:24:16.859189 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.859051    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81a9e12a-f557-4dfb-aaba-6b4c7c047042-tls-certs\") pod \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") "
Jul 27 20:24:16.859189 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.859078    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-tmp-dir\") pod \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") "
Jul 27 20:24:16.859189 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.859110    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-home\") pod \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\" (UID: \"81a9e12a-f557-4dfb-aaba-6b4c7c047042\") "
Jul 27 20:24:16.859392 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.859291    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-model-cache" (OuterVolumeSpecName: "model-cache") pod "81a9e12a-f557-4dfb-aaba-6b4c7c047042" (UID: "81a9e12a-f557-4dfb-aaba-6b4c7c047042"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:24:16.859444 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.859392    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "81a9e12a-f557-4dfb-aaba-6b4c7c047042" (UID: "81a9e12a-f557-4dfb-aaba-6b4c7c047042"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:24:16.859507 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.859482    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-home" (OuterVolumeSpecName: "home") pod "81a9e12a-f557-4dfb-aaba-6b4c7c047042" (UID: "81a9e12a-f557-4dfb-aaba-6b4c7c047042"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:24:16.861381 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.861357    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81a9e12a-f557-4dfb-aaba-6b4c7c047042-kube-api-access-gfgxs" (OuterVolumeSpecName: "kube-api-access-gfgxs") pod "81a9e12a-f557-4dfb-aaba-6b4c7c047042" (UID: "81a9e12a-f557-4dfb-aaba-6b4c7c047042"). InnerVolumeSpecName "kube-api-access-gfgxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:24:16.861730 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.861709    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81a9e12a-f557-4dfb-aaba-6b4c7c047042-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "81a9e12a-f557-4dfb-aaba-6b4c7c047042" (UID: "81a9e12a-f557-4dfb-aaba-6b4c7c047042"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:24:16.861800 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.861707    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-dshm" (OuterVolumeSpecName: "dshm") pod "81a9e12a-f557-4dfb-aaba-6b4c7c047042" (UID: "81a9e12a-f557-4dfb-aaba-6b4c7c047042"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:24:16.960327 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.960305    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-dshm\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:24:16.960327 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.960326    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfgxs\" (UniqueName: \"kubernetes.io/projected/81a9e12a-f557-4dfb-aaba-6b4c7c047042-kube-api-access-gfgxs\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:24:16.960454 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.960336    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-model-cache\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:24:16.960454 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.960346    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81a9e12a-f557-4dfb-aaba-6b4c7c047042-tls-certs\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:24:16.960454 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.960369    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-tmp-dir\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:24:16.960454 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:16.960376    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81a9e12a-f557-4dfb-aaba-6b4c7c047042-home\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:24:17.060581 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:17.060553    2579 generic.go:356] "Generic (PLEG): container finished" podID="81a9e12a-f557-4dfb-aaba-6b4c7c047042" containerID="057a46b766b7ffd24613195ee8f09e773936ac9578d3e091647f0b8cf5267fb4" exitCode=0
Jul 27 20:24:17.060710 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:17.060603    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"
Jul 27 20:24:17.060710 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:17.060650    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96" event={"ID":"81a9e12a-f557-4dfb-aaba-6b4c7c047042","Type":"ContainerDied","Data":"057a46b766b7ffd24613195ee8f09e773936ac9578d3e091647f0b8cf5267fb4"}
Jul 27 20:24:17.060710 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:17.060698    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96" event={"ID":"81a9e12a-f557-4dfb-aaba-6b4c7c047042","Type":"ContainerDied","Data":"fb70831b83339807a3d9c850af32b2cc419619035a4839634beabef36e2be6e5"}
Jul 27 20:24:17.060959 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:17.060718    2579 scope.go:117] "RemoveContainer" containerID="057a46b766b7ffd24613195ee8f09e773936ac9578d3e091647f0b8cf5267fb4"
Jul 27 20:24:17.069880 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:17.069847    2579 scope.go:117] "RemoveContainer" containerID="057a46b766b7ffd24613195ee8f09e773936ac9578d3e091647f0b8cf5267fb4"
Jul 27 20:24:17.070142 ip-10-0-140-36 kubenswrapper[2579]: E0727 20:24:17.070121    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"057a46b766b7ffd24613195ee8f09e773936ac9578d3e091647f0b8cf5267fb4\": container with ID starting with 057a46b766b7ffd24613195ee8f09e773936ac9578d3e091647f0b8cf5267fb4 not found: ID does not exist" containerID="057a46b766b7ffd24613195ee8f09e773936ac9578d3e091647f0b8cf5267fb4"
Jul 27 20:24:17.070195 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:17.070150    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"057a46b766b7ffd24613195ee8f09e773936ac9578d3e091647f0b8cf5267fb4"} err="failed to get container status \"057a46b766b7ffd24613195ee8f09e773936ac9578d3e091647f0b8cf5267fb4\": rpc error: code = NotFound desc = could not find container \"057a46b766b7ffd24613195ee8f09e773936ac9578d3e091647f0b8cf5267fb4\": container with ID starting with 057a46b766b7ffd24613195ee8f09e773936ac9578d3e091647f0b8cf5267fb4 not found: ID does not exist"
Jul 27 20:24:17.081662 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:17.081636    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"]
Jul 27 20:24:17.084976 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:17.084953    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f44149/auth-test-model-f44149-kserve-78487cdff5-29k96"]
Jul 27 20:24:18.261412 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:18.261378    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81a9e12a-f557-4dfb-aaba-6b4c7c047042" path="/var/lib/kubelet/pods/81a9e12a-f557-4dfb-aaba-6b4c7c047042/volumes"
Jul 27 20:24:20.260802 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:20.260776    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:20.260802 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:20.260807    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:20.273454 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:20.273426    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:21.091711 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:21.091684    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:24:32.431586 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:32.431503    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 20:24:32.439706 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:32.439684    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 20:24:46.639069 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.639040    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"]
Jul 27 20:24:46.639565 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.639332    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81a9e12a-f557-4dfb-aaba-6b4c7c047042" containerName="main"
Jul 27 20:24:46.639565 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.639342    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="81a9e12a-f557-4dfb-aaba-6b4c7c047042" containerName="main"
Jul 27 20:24:46.639565 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.639361    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5821d826-ddc6-48ed-8da0-c7ac28d01e98" containerName="main"
Jul 27 20:24:46.639565 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.639367    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5821d826-ddc6-48ed-8da0-c7ac28d01e98" containerName="main"
Jul 27 20:24:46.639565 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.639421    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5821d826-ddc6-48ed-8da0-c7ac28d01e98" containerName="main"
Jul 27 20:24:46.639565 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.639428    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="81a9e12a-f557-4dfb-aaba-6b4c7c047042" containerName="main"
Jul 27 20:24:46.642330 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.642313    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:46.644337 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.644316    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f44149\"/\"openshift-service-ca.crt\""
Jul 27 20:24:46.644788 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.644769    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f44149\"/\"kube-root-ca.crt\""
Jul 27 20:24:46.644884 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.644785    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f44149\"/\"sub-test-model-f44149-kserve-self-signed-certs\""
Jul 27 20:24:46.644884 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.644816    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f44149\"/\"default-dockercfg-s65vc\""
Jul 27 20:24:46.652502 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.652479    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"]
Jul 27 20:24:46.783237 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.783212    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-tmp-dir\") pod \"sub-test-model-f44149-kserve-6f44974c6c-4qbfk\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") " pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:46.783343 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.783240    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-dshm\") pod \"sub-test-model-f44149-kserve-6f44974c6c-4qbfk\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") " pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:46.783343 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.783320    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnt67\" (UniqueName: \"kubernetes.io/projected/7c054710-6674-435e-97bf-c0668a2f7144-kube-api-access-jnt67\") pod \"sub-test-model-f44149-kserve-6f44974c6c-4qbfk\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") " pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:46.783433 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.783349    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-home\") pod \"sub-test-model-f44149-kserve-6f44974c6c-4qbfk\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") " pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:46.783433 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.783392    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c054710-6674-435e-97bf-c0668a2f7144-tls-certs\") pod \"sub-test-model-f44149-kserve-6f44974c6c-4qbfk\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") " pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:46.783433 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.783417    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-model-cache\") pod \"sub-test-model-f44149-kserve-6f44974c6c-4qbfk\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") " pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:46.884581 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.884558    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnt67\" (UniqueName: \"kubernetes.io/projected/7c054710-6674-435e-97bf-c0668a2f7144-kube-api-access-jnt67\") pod \"sub-test-model-f44149-kserve-6f44974c6c-4qbfk\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") " pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:46.884720 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.884592    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-home\") pod \"sub-test-model-f44149-kserve-6f44974c6c-4qbfk\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") " pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:46.884720 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.884611    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c054710-6674-435e-97bf-c0668a2f7144-tls-certs\") pod \"sub-test-model-f44149-kserve-6f44974c6c-4qbfk\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") " pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:46.884720 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.884627    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-model-cache\") pod \"sub-test-model-f44149-kserve-6f44974c6c-4qbfk\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") " pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:46.884720 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.884660    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-tmp-dir\") pod \"sub-test-model-f44149-kserve-6f44974c6c-4qbfk\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") " pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:46.884720 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.884683    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-dshm\") pod \"sub-test-model-f44149-kserve-6f44974c6c-4qbfk\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") " pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:46.885046 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.885023    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-home\") pod \"sub-test-model-f44149-kserve-6f44974c6c-4qbfk\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") " pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:46.885107 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.885073    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-tmp-dir\") pod \"sub-test-model-f44149-kserve-6f44974c6c-4qbfk\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") " pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:46.885152 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.885118    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-model-cache\") pod \"sub-test-model-f44149-kserve-6f44974c6c-4qbfk\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") " pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:46.887036 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.887016    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-dshm\") pod \"sub-test-model-f44149-kserve-6f44974c6c-4qbfk\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") " pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:46.887463 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.887440    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c054710-6674-435e-97bf-c0668a2f7144-tls-certs\") pod \"sub-test-model-f44149-kserve-6f44974c6c-4qbfk\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") " pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:46.891911 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.891846    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnt67\" (UniqueName: \"kubernetes.io/projected/7c054710-6674-435e-97bf-c0668a2f7144-kube-api-access-jnt67\") pod \"sub-test-model-f44149-kserve-6f44974c6c-4qbfk\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") " pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:46.951892 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:46.951847    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:47.084541 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:47.084512    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"]
Jul 27 20:24:47.087032 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:24:47.087004    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c054710_6674_435e_97bf_c0668a2f7144.slice/crio-02e6472014cbf99a996e893a5f9f73cc25450aa394b9fbfab9be96defec96fb8 WatchSource:0}: Error finding container 02e6472014cbf99a996e893a5f9f73cc25450aa394b9fbfab9be96defec96fb8: Status 404 returned error can't find the container with id 02e6472014cbf99a996e893a5f9f73cc25450aa394b9fbfab9be96defec96fb8
Jul 27 20:24:47.204828 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:47.204794    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk" event={"ID":"7c054710-6674-435e-97bf-c0668a2f7144","Type":"ContainerStarted","Data":"14887212a58e588ccac5881c92915d7ef17643ac5f9810383c9dbc45af1ceb2e"}
Jul 27 20:24:47.204992 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:47.204842    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk" event={"ID":"7c054710-6674-435e-97bf-c0668a2f7144","Type":"ContainerStarted","Data":"02e6472014cbf99a996e893a5f9f73cc25450aa394b9fbfab9be96defec96fb8"}
Jul 27 20:24:47.223472 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:47.223427    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk" podStartSLOduration=1.223413192 podStartE2EDuration="1.223413192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:24:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:24:47.221532873 +0000 UTC m=+2715.450775093" watchObservedRunningTime="2026-07-27 20:24:47.223413192 +0000 UTC m=+2715.452655391"
Jul 27 20:24:56.952768 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:56.952719    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:56.953203 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:56.952786    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:56.965122 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:56.965101    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:24:57.264006 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:24:57.263932    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:25:51.244549 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:25:51.244508    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"]
Jul 27 20:25:52.209529 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:25:52.209498    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"]
Jul 27 20:26:01.233106 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.233074    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"]
Jul 27 20:26:01.236297 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.236281    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:01.238136 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.238112    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c3a53e\"/\"rate-test-model-c3a53e-kserve-self-signed-certs\""
Jul 27 20:26:01.244899 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.244876    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"]
Jul 27 20:26:01.311927 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.311905    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jrt4\" (UniqueName: \"kubernetes.io/projected/cfa01a19-9b6a-4d62-a335-47e7addade73-kube-api-access-5jrt4\") pod \"rate-test-model-c3a53e-kserve-5bf995467f-kwcr5\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") " pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:01.312030 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.311953    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-tmp-dir\") pod \"rate-test-model-c3a53e-kserve-5bf995467f-kwcr5\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") " pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:01.312030 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.311992    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-dshm\") pod \"rate-test-model-c3a53e-kserve-5bf995467f-kwcr5\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") " pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:01.312030 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.312008    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-model-cache\") pod \"rate-test-model-c3a53e-kserve-5bf995467f-kwcr5\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") " pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:01.312178 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.312033    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfa01a19-9b6a-4d62-a335-47e7addade73-tls-certs\") pod \"rate-test-model-c3a53e-kserve-5bf995467f-kwcr5\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") " pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:01.312178 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.312058    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-home\") pod \"rate-test-model-c3a53e-kserve-5bf995467f-kwcr5\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") " pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:01.412791 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.412749    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jrt4\" (UniqueName: \"kubernetes.io/projected/cfa01a19-9b6a-4d62-a335-47e7addade73-kube-api-access-5jrt4\") pod \"rate-test-model-c3a53e-kserve-5bf995467f-kwcr5\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") " pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:01.412979 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.412805    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-tmp-dir\") pod \"rate-test-model-c3a53e-kserve-5bf995467f-kwcr5\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") " pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:01.412979 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.412841    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-dshm\") pod \"rate-test-model-c3a53e-kserve-5bf995467f-kwcr5\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") " pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:01.412979 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.412892    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-model-cache\") pod \"rate-test-model-c3a53e-kserve-5bf995467f-kwcr5\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") " pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:01.412979 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.412930    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfa01a19-9b6a-4d62-a335-47e7addade73-tls-certs\") pod \"rate-test-model-c3a53e-kserve-5bf995467f-kwcr5\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") " pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:01.412979 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.412956    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-home\") pod \"rate-test-model-c3a53e-kserve-5bf995467f-kwcr5\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") " pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:01.413270 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.413250    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-tmp-dir\") pod \"rate-test-model-c3a53e-kserve-5bf995467f-kwcr5\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") " pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:01.413362 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.413346    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-model-cache\") pod \"rate-test-model-c3a53e-kserve-5bf995467f-kwcr5\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") " pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:01.413399 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.413379    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-home\") pod \"rate-test-model-c3a53e-kserve-5bf995467f-kwcr5\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") " pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:01.415365 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.415346    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-dshm\") pod \"rate-test-model-c3a53e-kserve-5bf995467f-kwcr5\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") " pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:01.415472 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.415451    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfa01a19-9b6a-4d62-a335-47e7addade73-tls-certs\") pod \"rate-test-model-c3a53e-kserve-5bf995467f-kwcr5\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") " pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:01.420021 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.420001    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jrt4\" (UniqueName: \"kubernetes.io/projected/cfa01a19-9b6a-4d62-a335-47e7addade73-kube-api-access-5jrt4\") pod \"rate-test-model-c3a53e-kserve-5bf995467f-kwcr5\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") " pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:01.546226 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.546137    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:01.675583 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:01.675506    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"]
Jul 27 20:26:01.679123 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:26:01.679096    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfa01a19_9b6a_4d62_a335_47e7addade73.slice/crio-48b0b8c824de56253bcfc2aca68db150b774ab43311c3e3f734bc14fb12e6edd WatchSource:0}: Error finding container 48b0b8c824de56253bcfc2aca68db150b774ab43311c3e3f734bc14fb12e6edd: Status 404 returned error can't find the container with id 48b0b8c824de56253bcfc2aca68db150b774ab43311c3e3f734bc14fb12e6edd
Jul 27 20:26:02.539156 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:02.539119    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5" event={"ID":"cfa01a19-9b6a-4d62-a335-47e7addade73","Type":"ContainerStarted","Data":"274e971db84f27f232fa4849e97b9cdba2693e9844b000aa28a40fc2a75d7cb8"}
Jul 27 20:26:02.539156 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:02.539162    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5" event={"ID":"cfa01a19-9b6a-4d62-a335-47e7addade73","Type":"ContainerStarted","Data":"48b0b8c824de56253bcfc2aca68db150b774ab43311c3e3f734bc14fb12e6edd"}
Jul 27 20:26:02.559833 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:02.559784    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5" podStartSLOduration=1.5597685860000001 podStartE2EDuration="1.559768586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:26:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:26:02.558642264 +0000 UTC m=+2790.787884461" watchObservedRunningTime="2026-07-27 20:26:02.559768586 +0000 UTC m=+2790.789010787"
Jul 27 20:26:06.263739 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.263686    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt" podUID="4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed" containerName="main" containerID="cri-o://5af731318f8413a6447359b9cb682a2a9d22473e537963674499def302b78d89" gracePeriod=45
Jul 27 20:26:06.517950 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.517891    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:26:06.558102 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.558076    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-tls-certs\") pod \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") "
Jul 27 20:26:06.558269 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.558136    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjr6l\" (UniqueName: \"kubernetes.io/projected/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-kube-api-access-zjr6l\") pod \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") "
Jul 27 20:26:06.558269 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.558176    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-dshm\") pod \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") "
Jul 27 20:26:06.558269 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.558248    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-tmp-dir\") pod \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") "
Jul 27 20:26:06.558437 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.558288    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-model-cache\") pod \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") "
Jul 27 20:26:06.558437 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.558333    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-home\") pod \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\" (UID: \"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed\") "
Jul 27 20:26:06.558723 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.558675    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed" (UID: "4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:26:06.558826 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.558754    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-home" (OuterVolumeSpecName: "home") pod "4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed" (UID: "4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:26:06.558826 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.558775    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-model-cache" (OuterVolumeSpecName: "model-cache") pod "4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed" (UID: "4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:26:06.559623 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.559594    2579 generic.go:356] "Generic (PLEG): container finished" podID="4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed" containerID="5af731318f8413a6447359b9cb682a2a9d22473e537963674499def302b78d89" exitCode=0
Jul 27 20:26:06.559740 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.559653    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"
Jul 27 20:26:06.559740 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.559708    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt" event={"ID":"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed","Type":"ContainerDied","Data":"5af731318f8413a6447359b9cb682a2a9d22473e537963674499def302b78d89"}
Jul 27 20:26:06.559848 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.559767    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt" event={"ID":"4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed","Type":"ContainerDied","Data":"16c00c7e4c16e5dd52dba43644b16f629ba9f370f472d577f5e5eea3eb0b5389"}
Jul 27 20:26:06.559848 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.559792    2579 scope.go:117] "RemoveContainer" containerID="5af731318f8413a6447359b9cb682a2a9d22473e537963674499def302b78d89"
Jul 27 20:26:06.560888 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.560825    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed" (UID: "4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:26:06.560995 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.560886    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-dshm" (OuterVolumeSpecName: "dshm") pod "4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed" (UID: "4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:26:06.560995 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.560906    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-kube-api-access-zjr6l" (OuterVolumeSpecName: "kube-api-access-zjr6l") pod "4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed" (UID: "4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed"). InnerVolumeSpecName "kube-api-access-zjr6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:26:06.577359 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.577338    2579 scope.go:117] "RemoveContainer" containerID="5af731318f8413a6447359b9cb682a2a9d22473e537963674499def302b78d89"
Jul 27 20:26:06.577611 ip-10-0-140-36 kubenswrapper[2579]: E0727 20:26:06.577594    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5af731318f8413a6447359b9cb682a2a9d22473e537963674499def302b78d89\": container with ID starting with 5af731318f8413a6447359b9cb682a2a9d22473e537963674499def302b78d89 not found: ID does not exist" containerID="5af731318f8413a6447359b9cb682a2a9d22473e537963674499def302b78d89"
Jul 27 20:26:06.577673 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.577620    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5af731318f8413a6447359b9cb682a2a9d22473e537963674499def302b78d89"} err="failed to get container status \"5af731318f8413a6447359b9cb682a2a9d22473e537963674499def302b78d89\": rpc error: code = NotFound desc = could not find container \"5af731318f8413a6447359b9cb682a2a9d22473e537963674499def302b78d89\": container with ID starting with 5af731318f8413a6447359b9cb682a2a9d22473e537963674499def302b78d89 not found: ID does not exist"
Jul 27 20:26:06.659844 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.659811    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-tmp-dir\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:26:06.659844 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.659837    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-model-cache\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:26:06.660066 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.659849    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-home\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:26:06.660066 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.659881    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-tls-certs\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:26:06.660066 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.659890    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjr6l\" (UniqueName: \"kubernetes.io/projected/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-kube-api-access-zjr6l\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:26:06.660066 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.659899    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed-dshm\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:26:06.879589 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.879558    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"]
Jul 27 20:26:06.881574 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:06.881548    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c3a53e/sub-test-model-c3a53e-kserve-56c45574dc-lrcmt"]
Jul 27 20:26:07.228622 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.228585    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk" podUID="7c054710-6674-435e-97bf-c0668a2f7144" containerName="main" containerID="cri-o://14887212a58e588ccac5881c92915d7ef17643ac5f9810383c9dbc45af1ceb2e" gracePeriod=45
Jul 27 20:26:07.253506 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.253473    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk" podUID="7c054710-6674-435e-97bf-c0668a2f7144" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": dial tcp 10.132.0.45:8000: connect: connection refused"
Jul 27 20:26:07.471736 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.471714    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:26:07.566351 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.566262    2579 generic.go:356] "Generic (PLEG): container finished" podID="7c054710-6674-435e-97bf-c0668a2f7144" containerID="14887212a58e588ccac5881c92915d7ef17643ac5f9810383c9dbc45af1ceb2e" exitCode=0
Jul 27 20:26:07.566351 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.566307    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"
Jul 27 20:26:07.566555 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.566347    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk" event={"ID":"7c054710-6674-435e-97bf-c0668a2f7144","Type":"ContainerDied","Data":"14887212a58e588ccac5881c92915d7ef17643ac5f9810383c9dbc45af1ceb2e"}
Jul 27 20:26:07.566555 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.566386    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk" event={"ID":"7c054710-6674-435e-97bf-c0668a2f7144","Type":"ContainerDied","Data":"02e6472014cbf99a996e893a5f9f73cc25450aa394b9fbfab9be96defec96fb8"}
Jul 27 20:26:07.566555 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.566403    2579 scope.go:117] "RemoveContainer" containerID="14887212a58e588ccac5881c92915d7ef17643ac5f9810383c9dbc45af1ceb2e"
Jul 27 20:26:07.567221 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.567202    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnt67\" (UniqueName: \"kubernetes.io/projected/7c054710-6674-435e-97bf-c0668a2f7144-kube-api-access-jnt67\") pod \"7c054710-6674-435e-97bf-c0668a2f7144\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") "
Jul 27 20:26:07.567328 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.567229    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-home\") pod \"7c054710-6674-435e-97bf-c0668a2f7144\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") "
Jul 27 20:26:07.567328 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.567263    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-dshm\") pod \"7c054710-6674-435e-97bf-c0668a2f7144\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") "
Jul 27 20:26:07.567328 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.567305    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c054710-6674-435e-97bf-c0668a2f7144-tls-certs\") pod \"7c054710-6674-435e-97bf-c0668a2f7144\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") "
Jul 27 20:26:07.567547 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.567352    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-model-cache\") pod \"7c054710-6674-435e-97bf-c0668a2f7144\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") "
Jul 27 20:26:07.567547 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.567428    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-tmp-dir\") pod \"7c054710-6674-435e-97bf-c0668a2f7144\" (UID: \"7c054710-6674-435e-97bf-c0668a2f7144\") "
Jul 27 20:26:07.567718 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.567692    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-home" (OuterVolumeSpecName: "home") pod "7c054710-6674-435e-97bf-c0668a2f7144" (UID: "7c054710-6674-435e-97bf-c0668a2f7144"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:26:07.567718 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.567705    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-model-cache" (OuterVolumeSpecName: "model-cache") pod "7c054710-6674-435e-97bf-c0668a2f7144" (UID: "7c054710-6674-435e-97bf-c0668a2f7144"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:26:07.567926 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.567738    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7c054710-6674-435e-97bf-c0668a2f7144" (UID: "7c054710-6674-435e-97bf-c0668a2f7144"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:26:07.569532 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.569503    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c054710-6674-435e-97bf-c0668a2f7144-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7c054710-6674-435e-97bf-c0668a2f7144" (UID: "7c054710-6674-435e-97bf-c0668a2f7144"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:26:07.570082 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.570062    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c054710-6674-435e-97bf-c0668a2f7144-kube-api-access-jnt67" (OuterVolumeSpecName: "kube-api-access-jnt67") pod "7c054710-6674-435e-97bf-c0668a2f7144" (UID: "7c054710-6674-435e-97bf-c0668a2f7144"). InnerVolumeSpecName "kube-api-access-jnt67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:26:07.570082 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.570063    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-dshm" (OuterVolumeSpecName: "dshm") pod "7c054710-6674-435e-97bf-c0668a2f7144" (UID: "7c054710-6674-435e-97bf-c0668a2f7144"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:26:07.585620 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.585601    2579 scope.go:117] "RemoveContainer" containerID="14887212a58e588ccac5881c92915d7ef17643ac5f9810383c9dbc45af1ceb2e"
Jul 27 20:26:07.585911 ip-10-0-140-36 kubenswrapper[2579]: E0727 20:26:07.585885    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14887212a58e588ccac5881c92915d7ef17643ac5f9810383c9dbc45af1ceb2e\": container with ID starting with 14887212a58e588ccac5881c92915d7ef17643ac5f9810383c9dbc45af1ceb2e not found: ID does not exist" containerID="14887212a58e588ccac5881c92915d7ef17643ac5f9810383c9dbc45af1ceb2e"
Jul 27 20:26:07.585996 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.585913    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14887212a58e588ccac5881c92915d7ef17643ac5f9810383c9dbc45af1ceb2e"} err="failed to get container status \"14887212a58e588ccac5881c92915d7ef17643ac5f9810383c9dbc45af1ceb2e\": rpc error: code = NotFound desc = could not find container \"14887212a58e588ccac5881c92915d7ef17643ac5f9810383c9dbc45af1ceb2e\": container with ID starting with 14887212a58e588ccac5881c92915d7ef17643ac5f9810383c9dbc45af1ceb2e not found: ID does not exist"
Jul 27 20:26:07.668806 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.668782    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c054710-6674-435e-97bf-c0668a2f7144-tls-certs\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:26:07.668806 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.668806    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-model-cache\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:26:07.668975 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.668816    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-tmp-dir\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:26:07.668975 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.668827    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnt67\" (UniqueName: \"kubernetes.io/projected/7c054710-6674-435e-97bf-c0668a2f7144-kube-api-access-jnt67\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:26:07.668975 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.668841    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-home\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:26:07.668975 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.668849    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c054710-6674-435e-97bf-c0668a2f7144-dshm\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:26:07.894360 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.894323    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"]
Jul 27 20:26:07.898687 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:07.898660    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f44149/sub-test-model-f44149-kserve-6f44974c6c-4qbfk"]
Jul 27 20:26:08.261927 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:08.261896    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed" path="/var/lib/kubelet/pods/4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed/volumes"
Jul 27 20:26:08.262340 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:08.262320    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c054710-6674-435e-97bf-c0668a2f7144" path="/var/lib/kubelet/pods/7c054710-6674-435e-97bf-c0668a2f7144/volumes"
Jul 27 20:26:11.546344 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:11.546307    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:11.546825 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:11.546365    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:11.558377 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:11.558355    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:11.597710 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:11.597686    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:26:23.835647 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:23.835610    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"]
Jul 27 20:26:23.836165 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:23.835934    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c054710-6674-435e-97bf-c0668a2f7144" containerName="main"
Jul 27 20:26:23.836165 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:23.835947    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c054710-6674-435e-97bf-c0668a2f7144" containerName="main"
Jul 27 20:26:23.836165 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:23.835963    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed" containerName="main"
Jul 27 20:26:23.836165 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:23.835968    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed" containerName="main"
Jul 27 20:26:23.836165 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:23.836032    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ca94e3c-6bfb-44bd-84b2-d1fc5bb870ed" containerName="main"
Jul 27 20:26:23.836165 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:23.836044    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c054710-6674-435e-97bf-c0668a2f7144" containerName="main"
Jul 27 20:26:23.838465 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:23.838449    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:23.841028 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:23.840997    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f44149\"/\"kube-root-ca.crt\""
Jul 27 20:26:23.841161 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:23.841066    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f44149\"/\"openshift-service-ca.crt\""
Jul 27 20:26:23.842302 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:23.842274    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f44149\"/\"rate-test-model-f44149-kserve-self-signed-certs\""
Jul 27 20:26:23.843124 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:23.843098    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f44149\"/\"default-dockercfg-s65vc\""
Jul 27 20:26:23.852329 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:23.852305    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"]
Jul 27 20:26:23.990451 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:23.990427    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d64285f5-13e5-400f-be86-913b84d4e212-tls-certs\") pod \"rate-test-model-f44149-kserve-5df88d447f-ng47l\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") " pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:23.990578 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:23.990468    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8twk\" (UniqueName: \"kubernetes.io/projected/d64285f5-13e5-400f-be86-913b84d4e212-kube-api-access-j8twk\") pod \"rate-test-model-f44149-kserve-5df88d447f-ng47l\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") " pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:23.990578 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:23.990499    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-dshm\") pod \"rate-test-model-f44149-kserve-5df88d447f-ng47l\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") " pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:23.990699 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:23.990583    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-home\") pod \"rate-test-model-f44149-kserve-5df88d447f-ng47l\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") " pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:23.990699 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:23.990622    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-model-cache\") pod \"rate-test-model-f44149-kserve-5df88d447f-ng47l\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") " pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:23.990699 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:23.990662    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-tmp-dir\") pod \"rate-test-model-f44149-kserve-5df88d447f-ng47l\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") " pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:24.092041 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:24.091956    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8twk\" (UniqueName: \"kubernetes.io/projected/d64285f5-13e5-400f-be86-913b84d4e212-kube-api-access-j8twk\") pod \"rate-test-model-f44149-kserve-5df88d447f-ng47l\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") " pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:24.092041 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:24.092027    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-dshm\") pod \"rate-test-model-f44149-kserve-5df88d447f-ng47l\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") " pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:24.092207 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:24.092067    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-home\") pod \"rate-test-model-f44149-kserve-5df88d447f-ng47l\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") " pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:24.092207 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:24.092094    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-model-cache\") pod \"rate-test-model-f44149-kserve-5df88d447f-ng47l\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") " pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:24.092296 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:24.092226    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-tmp-dir\") pod \"rate-test-model-f44149-kserve-5df88d447f-ng47l\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") " pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:24.092296 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:24.092287    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d64285f5-13e5-400f-be86-913b84d4e212-tls-certs\") pod \"rate-test-model-f44149-kserve-5df88d447f-ng47l\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") " pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:24.092486 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:24.092467    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-home\") pod \"rate-test-model-f44149-kserve-5df88d447f-ng47l\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") " pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:24.092553 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:24.092498    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-model-cache\") pod \"rate-test-model-f44149-kserve-5df88d447f-ng47l\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") " pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:24.092593 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:24.092551    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-tmp-dir\") pod \"rate-test-model-f44149-kserve-5df88d447f-ng47l\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") " pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:24.094444 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:24.094425    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-dshm\") pod \"rate-test-model-f44149-kserve-5df88d447f-ng47l\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") " pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:24.094928 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:24.094908    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d64285f5-13e5-400f-be86-913b84d4e212-tls-certs\") pod \"rate-test-model-f44149-kserve-5df88d447f-ng47l\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") " pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:24.100065 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:24.100044    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8twk\" (UniqueName: \"kubernetes.io/projected/d64285f5-13e5-400f-be86-913b84d4e212-kube-api-access-j8twk\") pod \"rate-test-model-f44149-kserve-5df88d447f-ng47l\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") " pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:24.150046 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:24.150025    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:24.284337 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:24.284314    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"]
Jul 27 20:26:24.285930 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:26:24.285902    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd64285f5_13e5_400f_be86_913b84d4e212.slice/crio-a5775fda77c6f287fd5c882a5250d0ed6ac1229a6059a183389bb44f3c63663b WatchSource:0}: Error finding container a5775fda77c6f287fd5c882a5250d0ed6ac1229a6059a183389bb44f3c63663b: Status 404 returned error can't find the container with id a5775fda77c6f287fd5c882a5250d0ed6ac1229a6059a183389bb44f3c63663b
Jul 27 20:26:24.642225 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:24.642191    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l" event={"ID":"d64285f5-13e5-400f-be86-913b84d4e212","Type":"ContainerStarted","Data":"5ff422994bdf7bf9d289e6b2d65dc731a999e4e4a571afc4bc731b1f5fa92adf"}
Jul 27 20:26:24.642225 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:24.642230    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l" event={"ID":"d64285f5-13e5-400f-be86-913b84d4e212","Type":"ContainerStarted","Data":"a5775fda77c6f287fd5c882a5250d0ed6ac1229a6059a183389bb44f3c63663b"}
Jul 27 20:26:24.660281 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:24.660238    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l" podStartSLOduration=1.66022545 podStartE2EDuration="1.66022545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:26:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:26:24.659008094 +0000 UTC m=+2812.888250292" watchObservedRunningTime="2026-07-27 20:26:24.66022545 +0000 UTC m=+2812.889467678"
Jul 27 20:26:34.150447 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:34.150412    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:34.150840 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:34.150461    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:34.163083 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:34.163056    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:26:34.699677 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:26:34.699651    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:27:47.372903 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:47.372793    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"]
Jul 27 20:27:48.513669 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:48.513637    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"]
Jul 27 20:27:49.424295 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:49.424261    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7342-139753588705456"]
Jul 27 20:27:49.426845 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:49.426828    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7342-139753588705456"
Jul 27 20:27:49.429097 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:49.429065    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-kljmn\""
Jul 27 20:27:49.434163 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:49.434078    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7342-139753588705456"]
Jul 27 20:27:49.535637 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:49.535610    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98sp9\" (UniqueName: \"kubernetes.io/projected/435e5708-7f3b-4589-a7e1-bb82ef093e44-kube-api-access-98sp9\") pod \"test-curl-7342-139753588705456\" (UID: \"435e5708-7f3b-4589-a7e1-bb82ef093e44\") " pod="opendatahub/test-curl-7342-139753588705456"
Jul 27 20:27:49.636002 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:49.635973    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98sp9\" (UniqueName: \"kubernetes.io/projected/435e5708-7f3b-4589-a7e1-bb82ef093e44-kube-api-access-98sp9\") pod \"test-curl-7342-139753588705456\" (UID: \"435e5708-7f3b-4589-a7e1-bb82ef093e44\") " pod="opendatahub/test-curl-7342-139753588705456"
Jul 27 20:27:49.645498 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:49.645469    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98sp9\" (UniqueName: \"kubernetes.io/projected/435e5708-7f3b-4589-a7e1-bb82ef093e44-kube-api-access-98sp9\") pod \"test-curl-7342-139753588705456\" (UID: \"435e5708-7f3b-4589-a7e1-bb82ef093e44\") " pod="opendatahub/test-curl-7342-139753588705456"
Jul 27 20:27:49.736579 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:49.736466    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7342-139753588705456"
Jul 27 20:27:49.875449 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:49.875424    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7342-139753588705456"]
Jul 27 20:27:49.878026 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:27:49.877987    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod435e5708_7f3b_4589_a7e1_bb82ef093e44.slice/crio-b3e29dba5a888f767e87d1df36984fac35b3f9e8912c2013e7345d99d0d881bf WatchSource:0}: Error finding container b3e29dba5a888f767e87d1df36984fac35b3f9e8912c2013e7345d99d0d881bf: Status 404 returned error can't find the container with id b3e29dba5a888f767e87d1df36984fac35b3f9e8912c2013e7345d99d0d881bf
Jul 27 20:27:49.879228 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:49.879211    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 20:27:50.021380 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:50.021316    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7342-139753588705456" event={"ID":"435e5708-7f3b-4589-a7e1-bb82ef093e44","Type":"ContainerStarted","Data":"b3e29dba5a888f767e87d1df36984fac35b3f9e8912c2013e7345d99d0d881bf"}
Jul 27 20:27:51.026963 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:51.026926    2579 generic.go:356] "Generic (PLEG): container finished" podID="435e5708-7f3b-4589-a7e1-bb82ef093e44" containerID="7ecf41e61e314f53b7a84ac7b70976845cbd654c212a17dab79307be59c1e875" exitCode=0
Jul 27 20:27:51.027311 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:51.026972    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7342-139753588705456" event={"ID":"435e5708-7f3b-4589-a7e1-bb82ef093e44","Type":"ContainerDied","Data":"7ecf41e61e314f53b7a84ac7b70976845cbd654c212a17dab79307be59c1e875"}
Jul 27 20:27:52.167445 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:52.167420    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7342-139753588705456"
Jul 27 20:27:52.199683 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:52.199663    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7342-139753588705456_435e5708-7f3b-4589-a7e1-bb82ef093e44/test-curl-7342-139753588705456/0.log"
Jul 27 20:27:52.219741 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:52.219718    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7342-139753588705456"]
Jul 27 20:27:52.223523 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:52.223504    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7342-139753588705456"]
Jul 27 20:27:52.256146 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:52.256126    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98sp9\" (UniqueName: \"kubernetes.io/projected/435e5708-7f3b-4589-a7e1-bb82ef093e44-kube-api-access-98sp9\") pod \"435e5708-7f3b-4589-a7e1-bb82ef093e44\" (UID: \"435e5708-7f3b-4589-a7e1-bb82ef093e44\") "
Jul 27 20:27:52.258469 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:52.258448    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/435e5708-7f3b-4589-a7e1-bb82ef093e44-kube-api-access-98sp9" (OuterVolumeSpecName: "kube-api-access-98sp9") pod "435e5708-7f3b-4589-a7e1-bb82ef093e44" (UID: "435e5708-7f3b-4589-a7e1-bb82ef093e44"). InnerVolumeSpecName "kube-api-access-98sp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:27:52.262716 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:52.262693    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="435e5708-7f3b-4589-a7e1-bb82ef093e44" path="/var/lib/kubelet/pods/435e5708-7f3b-4589-a7e1-bb82ef093e44/volumes"
Jul 27 20:27:52.356952 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:52.356902    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98sp9\" (UniqueName: \"kubernetes.io/projected/435e5708-7f3b-4589-a7e1-bb82ef093e44-kube-api-access-98sp9\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:27:53.036834 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:53.036805    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7342-139753588705456"
Jul 27 20:27:53.037028 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:53.036874    2579 scope.go:117] "RemoveContainer" containerID="7ecf41e61e314f53b7a84ac7b70976845cbd654c212a17dab79307be59c1e875"
Jul 27 20:27:55.598216 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:55.598184    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7342-139753583413120"]
Jul 27 20:27:55.598568 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:55.598474    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="435e5708-7f3b-4589-a7e1-bb82ef093e44" containerName="test-curl-7342-139753588705456"
Jul 27 20:27:55.598568 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:55.598485    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="435e5708-7f3b-4589-a7e1-bb82ef093e44" containerName="test-curl-7342-139753588705456"
Jul 27 20:27:55.598568 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:55.598554    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="435e5708-7f3b-4589-a7e1-bb82ef093e44" containerName="test-curl-7342-139753588705456"
Jul 27 20:27:55.602496 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:55.602477    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7342-139753583413120"
Jul 27 20:27:55.604276 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:55.604257    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-kljmn\""
Jul 27 20:27:55.606720 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:55.606698    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7342-139753583413120"]
Jul 27 20:27:55.680296 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:55.680266    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpw68\" (UniqueName: \"kubernetes.io/projected/fc050ea4-6f62-49ec-baad-979449b3ef43-kube-api-access-hpw68\") pod \"test-curl-7342-139753583413120\" (UID: \"fc050ea4-6f62-49ec-baad-979449b3ef43\") " pod="opendatahub/test-curl-7342-139753583413120"
Jul 27 20:27:55.780927 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:55.780902    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpw68\" (UniqueName: \"kubernetes.io/projected/fc050ea4-6f62-49ec-baad-979449b3ef43-kube-api-access-hpw68\") pod \"test-curl-7342-139753583413120\" (UID: \"fc050ea4-6f62-49ec-baad-979449b3ef43\") " pod="opendatahub/test-curl-7342-139753583413120"
Jul 27 20:27:55.787890 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:55.787871    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpw68\" (UniqueName: \"kubernetes.io/projected/fc050ea4-6f62-49ec-baad-979449b3ef43-kube-api-access-hpw68\") pod \"test-curl-7342-139753583413120\" (UID: \"fc050ea4-6f62-49ec-baad-979449b3ef43\") " pod="opendatahub/test-curl-7342-139753583413120"
Jul 27 20:27:55.914036 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:55.914015    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7342-139753583413120"
Jul 27 20:27:56.048874 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:56.048778    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7342-139753583413120"]
Jul 27 20:27:56.051521 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:27:56.051474    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc050ea4_6f62_49ec_baad_979449b3ef43.slice/crio-59244a06139da68f24a9bb5958ec15e588ef326b8105c73aee33607cae27348a WatchSource:0}: Error finding container 59244a06139da68f24a9bb5958ec15e588ef326b8105c73aee33607cae27348a: Status 404 returned error can't find the container with id 59244a06139da68f24a9bb5958ec15e588ef326b8105c73aee33607cae27348a
Jul 27 20:27:57.058768 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:57.058687    2579 generic.go:356] "Generic (PLEG): container finished" podID="fc050ea4-6f62-49ec-baad-979449b3ef43" containerID="a30af83424025a9726b45dafdaa5e5ff0ad4daae0d861af9c4e13a95a0b68f42" exitCode=0
Jul 27 20:27:57.058768 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:57.058730    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7342-139753583413120" event={"ID":"fc050ea4-6f62-49ec-baad-979449b3ef43","Type":"ContainerDied","Data":"a30af83424025a9726b45dafdaa5e5ff0ad4daae0d861af9c4e13a95a0b68f42"}
Jul 27 20:27:57.058768 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:57.058758    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7342-139753583413120" event={"ID":"fc050ea4-6f62-49ec-baad-979449b3ef43","Type":"ContainerStarted","Data":"59244a06139da68f24a9bb5958ec15e588ef326b8105c73aee33607cae27348a"}
Jul 27 20:27:58.189149 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:58.189126    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7342-139753583413120"
Jul 27 20:27:58.208721 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:58.208699    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7342-139753583413120_fc050ea4-6f62-49ec-baad-979449b3ef43/test-curl-7342-139753583413120/0.log"
Jul 27 20:27:58.229820 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:58.229794    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7342-139753583413120"]
Jul 27 20:27:58.231494 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:58.231474    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7342-139753583413120"]
Jul 27 20:27:58.303383 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:58.303363    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpw68\" (UniqueName: \"kubernetes.io/projected/fc050ea4-6f62-49ec-baad-979449b3ef43-kube-api-access-hpw68\") pod \"fc050ea4-6f62-49ec-baad-979449b3ef43\" (UID: \"fc050ea4-6f62-49ec-baad-979449b3ef43\") "
Jul 27 20:27:58.305524 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:58.305503    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc050ea4-6f62-49ec-baad-979449b3ef43-kube-api-access-hpw68" (OuterVolumeSpecName: "kube-api-access-hpw68") pod "fc050ea4-6f62-49ec-baad-979449b3ef43" (UID: "fc050ea4-6f62-49ec-baad-979449b3ef43"). InnerVolumeSpecName "kube-api-access-hpw68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:27:58.404034 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:58.403977    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpw68\" (UniqueName: \"kubernetes.io/projected/fc050ea4-6f62-49ec-baad-979449b3ef43-kube-api-access-hpw68\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:27:59.068533 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:59.068508    2579 scope.go:117] "RemoveContainer" containerID="a30af83424025a9726b45dafdaa5e5ff0ad4daae0d861af9c4e13a95a0b68f42"
Jul 27 20:27:59.068533 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:27:59.068523    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7342-139753583413120"
Jul 27 20:28:00.261449 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:00.261411    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc050ea4-6f62-49ec-baad-979449b3ef43" path="/var/lib/kubelet/pods/fc050ea4-6f62-49ec-baad-979449b3ef43/volumes"
Jul 27 20:28:02.391786 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:02.391740    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5" podUID="cfa01a19-9b6a-4d62-a335-47e7addade73" containerName="main" containerID="cri-o://274e971db84f27f232fa4849e97b9cdba2693e9844b000aa28a40fc2a75d7cb8" gracePeriod=45
Jul 27 20:28:02.635814 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:02.635791    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:28:02.739216 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:02.739181    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jrt4\" (UniqueName: \"kubernetes.io/projected/cfa01a19-9b6a-4d62-a335-47e7addade73-kube-api-access-5jrt4\") pod \"cfa01a19-9b6a-4d62-a335-47e7addade73\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") "
Jul 27 20:28:02.739392 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:02.739266    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-model-cache\") pod \"cfa01a19-9b6a-4d62-a335-47e7addade73\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") "
Jul 27 20:28:02.739392 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:02.739300    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-dshm\") pod \"cfa01a19-9b6a-4d62-a335-47e7addade73\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") "
Jul 27 20:28:02.739392 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:02.739321    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-tmp-dir\") pod \"cfa01a19-9b6a-4d62-a335-47e7addade73\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") "
Jul 27 20:28:02.739392 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:02.739344    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-home\") pod \"cfa01a19-9b6a-4d62-a335-47e7addade73\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") "
Jul 27 20:28:02.739392 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:02.739384    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfa01a19-9b6a-4d62-a335-47e7addade73-tls-certs\") pod \"cfa01a19-9b6a-4d62-a335-47e7addade73\" (UID: \"cfa01a19-9b6a-4d62-a335-47e7addade73\") "
Jul 27 20:28:02.739645 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:02.739585    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-model-cache" (OuterVolumeSpecName: "model-cache") pod "cfa01a19-9b6a-4d62-a335-47e7addade73" (UID: "cfa01a19-9b6a-4d62-a335-47e7addade73"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:28:02.739699 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:02.739610    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cfa01a19-9b6a-4d62-a335-47e7addade73" (UID: "cfa01a19-9b6a-4d62-a335-47e7addade73"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:28:02.739699 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:02.739670    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-home" (OuterVolumeSpecName: "home") pod "cfa01a19-9b6a-4d62-a335-47e7addade73" (UID: "cfa01a19-9b6a-4d62-a335-47e7addade73"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:28:02.741539 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:02.741508    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-dshm" (OuterVolumeSpecName: "dshm") pod "cfa01a19-9b6a-4d62-a335-47e7addade73" (UID: "cfa01a19-9b6a-4d62-a335-47e7addade73"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:28:02.741646 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:02.741564    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfa01a19-9b6a-4d62-a335-47e7addade73-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cfa01a19-9b6a-4d62-a335-47e7addade73" (UID: "cfa01a19-9b6a-4d62-a335-47e7addade73"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:28:02.741646 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:02.741632    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfa01a19-9b6a-4d62-a335-47e7addade73-kube-api-access-5jrt4" (OuterVolumeSpecName: "kube-api-access-5jrt4") pod "cfa01a19-9b6a-4d62-a335-47e7addade73" (UID: "cfa01a19-9b6a-4d62-a335-47e7addade73"). InnerVolumeSpecName "kube-api-access-5jrt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:28:02.840052 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:02.840020    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfa01a19-9b6a-4d62-a335-47e7addade73-tls-certs\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:28:02.840052 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:02.840048    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jrt4\" (UniqueName: \"kubernetes.io/projected/cfa01a19-9b6a-4d62-a335-47e7addade73-kube-api-access-5jrt4\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:28:02.840052 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:02.840058    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-model-cache\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:28:02.840266 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:02.840066    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-dshm\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:28:02.840266 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:02.840073    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-tmp-dir\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:28:02.840266 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:02.840081    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfa01a19-9b6a-4d62-a335-47e7addade73-home\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:28:03.087996 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.087908    2579 generic.go:356] "Generic (PLEG): container finished" podID="cfa01a19-9b6a-4d62-a335-47e7addade73" containerID="274e971db84f27f232fa4849e97b9cdba2693e9844b000aa28a40fc2a75d7cb8" exitCode=0
Jul 27 20:28:03.087996 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.087948    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"
Jul 27 20:28:03.087996 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.087976    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5" event={"ID":"cfa01a19-9b6a-4d62-a335-47e7addade73","Type":"ContainerDied","Data":"274e971db84f27f232fa4849e97b9cdba2693e9844b000aa28a40fc2a75d7cb8"}
Jul 27 20:28:03.088235 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.088019    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5" event={"ID":"cfa01a19-9b6a-4d62-a335-47e7addade73","Type":"ContainerDied","Data":"48b0b8c824de56253bcfc2aca68db150b774ab43311c3e3f734bc14fb12e6edd"}
Jul 27 20:28:03.088235 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.088038    2579 scope.go:117] "RemoveContainer" containerID="274e971db84f27f232fa4849e97b9cdba2693e9844b000aa28a40fc2a75d7cb8"
Jul 27 20:28:03.097094 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.097076    2579 scope.go:117] "RemoveContainer" containerID="274e971db84f27f232fa4849e97b9cdba2693e9844b000aa28a40fc2a75d7cb8"
Jul 27 20:28:03.097352 ip-10-0-140-36 kubenswrapper[2579]: E0727 20:28:03.097332    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"274e971db84f27f232fa4849e97b9cdba2693e9844b000aa28a40fc2a75d7cb8\": container with ID starting with 274e971db84f27f232fa4849e97b9cdba2693e9844b000aa28a40fc2a75d7cb8 not found: ID does not exist" containerID="274e971db84f27f232fa4849e97b9cdba2693e9844b000aa28a40fc2a75d7cb8"
Jul 27 20:28:03.097395 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.097361    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"274e971db84f27f232fa4849e97b9cdba2693e9844b000aa28a40fc2a75d7cb8"} err="failed to get container status \"274e971db84f27f232fa4849e97b9cdba2693e9844b000aa28a40fc2a75d7cb8\": rpc error: code = NotFound desc = could not find container \"274e971db84f27f232fa4849e97b9cdba2693e9844b000aa28a40fc2a75d7cb8\": container with ID starting with 274e971db84f27f232fa4849e97b9cdba2693e9844b000aa28a40fc2a75d7cb8 not found: ID does not exist"
Jul 27 20:28:03.107113 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.107087    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"]
Jul 27 20:28:03.108794 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.108773    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c3a53e/rate-test-model-c3a53e-kserve-5bf995467f-kwcr5"]
Jul 27 20:28:03.534923 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.534880    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l" podUID="d64285f5-13e5-400f-be86-913b84d4e212" containerName="main" containerID="cri-o://5ff422994bdf7bf9d289e6b2d65dc731a999e4e4a571afc4bc731b1f5fa92adf" gracePeriod=45
Jul 27 20:28:03.606354 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.606323    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7342-139753588852320"]
Jul 27 20:28:03.606627 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.606613    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc050ea4-6f62-49ec-baad-979449b3ef43" containerName="test-curl-7342-139753583413120"
Jul 27 20:28:03.606627 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.606627    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc050ea4-6f62-49ec-baad-979449b3ef43" containerName="test-curl-7342-139753583413120"
Jul 27 20:28:03.606708 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.606651    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfa01a19-9b6a-4d62-a335-47e7addade73" containerName="main"
Jul 27 20:28:03.606708 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.606658    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfa01a19-9b6a-4d62-a335-47e7addade73" containerName="main"
Jul 27 20:28:03.606708 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.606707    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc050ea4-6f62-49ec-baad-979449b3ef43" containerName="test-curl-7342-139753583413120"
Jul 27 20:28:03.606816 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.606715    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfa01a19-9b6a-4d62-a335-47e7addade73" containerName="main"
Jul 27 20:28:03.609398 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.609382    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7342-139753588852320"
Jul 27 20:28:03.611512 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.611490    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-kljmn\""
Jul 27 20:28:03.616667 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.616633    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7342-139753588852320"]
Jul 27 20:28:03.746054 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.746022    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g6kd\" (UniqueName: \"kubernetes.io/projected/beefd8c8-e591-4aaf-aa13-9aebb737a86b-kube-api-access-6g6kd\") pod \"test-curl-7342-139753588852320\" (UID: \"beefd8c8-e591-4aaf-aa13-9aebb737a86b\") " pod="opendatahub/test-curl-7342-139753588852320"
Jul 27 20:28:03.776737 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.776717    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:28:03.847075 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.847013    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-dshm\") pod \"d64285f5-13e5-400f-be86-913b84d4e212\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") "
Jul 27 20:28:03.847075 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.847061    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-model-cache\") pod \"d64285f5-13e5-400f-be86-913b84d4e212\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") "
Jul 27 20:28:03.847243 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.847102    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d64285f5-13e5-400f-be86-913b84d4e212-tls-certs\") pod \"d64285f5-13e5-400f-be86-913b84d4e212\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") "
Jul 27 20:28:03.847243 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.847155    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-home\") pod \"d64285f5-13e5-400f-be86-913b84d4e212\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") "
Jul 27 20:28:03.847243 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.847203    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8twk\" (UniqueName: \"kubernetes.io/projected/d64285f5-13e5-400f-be86-913b84d4e212-kube-api-access-j8twk\") pod \"d64285f5-13e5-400f-be86-913b84d4e212\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") "
Jul 27 20:28:03.847243 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.847239    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-tmp-dir\") pod \"d64285f5-13e5-400f-be86-913b84d4e212\" (UID: \"d64285f5-13e5-400f-be86-913b84d4e212\") "
Jul 27 20:28:03.847461 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.847359    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-model-cache" (OuterVolumeSpecName: "model-cache") pod "d64285f5-13e5-400f-be86-913b84d4e212" (UID: "d64285f5-13e5-400f-be86-913b84d4e212"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:28:03.847461 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.847441    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g6kd\" (UniqueName: \"kubernetes.io/projected/beefd8c8-e591-4aaf-aa13-9aebb737a86b-kube-api-access-6g6kd\") pod \"test-curl-7342-139753588852320\" (UID: \"beefd8c8-e591-4aaf-aa13-9aebb737a86b\") " pod="opendatahub/test-curl-7342-139753588852320"
Jul 27 20:28:03.847461 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.847427    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-home" (OuterVolumeSpecName: "home") pod "d64285f5-13e5-400f-be86-913b84d4e212" (UID: "d64285f5-13e5-400f-be86-913b84d4e212"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:28:03.847608 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.847490    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d64285f5-13e5-400f-be86-913b84d4e212" (UID: "d64285f5-13e5-400f-be86-913b84d4e212"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:28:03.847608 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.847580    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-home\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:28:03.847608 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.847600    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-tmp-dir\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:28:03.847950 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.847617    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-model-cache\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:28:03.849247 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.849225    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-dshm" (OuterVolumeSpecName: "dshm") pod "d64285f5-13e5-400f-be86-913b84d4e212" (UID: "d64285f5-13e5-400f-be86-913b84d4e212"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:28:03.849328 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.849308    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d64285f5-13e5-400f-be86-913b84d4e212-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d64285f5-13e5-400f-be86-913b84d4e212" (UID: "d64285f5-13e5-400f-be86-913b84d4e212"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:28:03.849369 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.849329    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d64285f5-13e5-400f-be86-913b84d4e212-kube-api-access-j8twk" (OuterVolumeSpecName: "kube-api-access-j8twk") pod "d64285f5-13e5-400f-be86-913b84d4e212" (UID: "d64285f5-13e5-400f-be86-913b84d4e212"). InnerVolumeSpecName "kube-api-access-j8twk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:28:03.857644 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.857625    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g6kd\" (UniqueName: \"kubernetes.io/projected/beefd8c8-e591-4aaf-aa13-9aebb737a86b-kube-api-access-6g6kd\") pod \"test-curl-7342-139753588852320\" (UID: \"beefd8c8-e591-4aaf-aa13-9aebb737a86b\") " pod="opendatahub/test-curl-7342-139753588852320"
Jul 27 20:28:03.919758 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.919736    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7342-139753588852320"
Jul 27 20:28:03.948016 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.947993    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d64285f5-13e5-400f-be86-913b84d4e212-dshm\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:28:03.948016 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.948015    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d64285f5-13e5-400f-be86-913b84d4e212-tls-certs\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:28:03.948144 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:03.948026    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8twk\" (UniqueName: \"kubernetes.io/projected/d64285f5-13e5-400f-be86-913b84d4e212-kube-api-access-j8twk\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:28:04.049022 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:04.048993    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7342-139753588852320"]
Jul 27 20:28:04.051958 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:28:04.051932    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbeefd8c8_e591_4aaf_aa13_9aebb737a86b.slice/crio-b220640118fc571f503a060ee3e1d82b7868d7564cfbf21423273467f195578c WatchSource:0}: Error finding container b220640118fc571f503a060ee3e1d82b7868d7564cfbf21423273467f195578c: Status 404 returned error can't find the container with id b220640118fc571f503a060ee3e1d82b7868d7564cfbf21423273467f195578c
Jul 27 20:28:04.093488 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:04.093459    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7342-139753588852320" event={"ID":"beefd8c8-e591-4aaf-aa13-9aebb737a86b","Type":"ContainerStarted","Data":"b220640118fc571f503a060ee3e1d82b7868d7564cfbf21423273467f195578c"}
Jul 27 20:28:04.095459 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:04.095438    2579 generic.go:356] "Generic (PLEG): container finished" podID="d64285f5-13e5-400f-be86-913b84d4e212" containerID="5ff422994bdf7bf9d289e6b2d65dc731a999e4e4a571afc4bc731b1f5fa92adf" exitCode=0
Jul 27 20:28:04.095547 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:04.095476    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"
Jul 27 20:28:04.095591 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:04.095471    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l" event={"ID":"d64285f5-13e5-400f-be86-913b84d4e212","Type":"ContainerDied","Data":"5ff422994bdf7bf9d289e6b2d65dc731a999e4e4a571afc4bc731b1f5fa92adf"}
Jul 27 20:28:04.095591 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:04.095585    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l" event={"ID":"d64285f5-13e5-400f-be86-913b84d4e212","Type":"ContainerDied","Data":"a5775fda77c6f287fd5c882a5250d0ed6ac1229a6059a183389bb44f3c63663b"}
Jul 27 20:28:04.095670 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:04.095603    2579 scope.go:117] "RemoveContainer" containerID="5ff422994bdf7bf9d289e6b2d65dc731a999e4e4a571afc4bc731b1f5fa92adf"
Jul 27 20:28:04.104153 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:04.104132    2579 scope.go:117] "RemoveContainer" containerID="5ff422994bdf7bf9d289e6b2d65dc731a999e4e4a571afc4bc731b1f5fa92adf"
Jul 27 20:28:04.104402 ip-10-0-140-36 kubenswrapper[2579]: E0727 20:28:04.104380    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ff422994bdf7bf9d289e6b2d65dc731a999e4e4a571afc4bc731b1f5fa92adf\": container with ID starting with 5ff422994bdf7bf9d289e6b2d65dc731a999e4e4a571afc4bc731b1f5fa92adf not found: ID does not exist" containerID="5ff422994bdf7bf9d289e6b2d65dc731a999e4e4a571afc4bc731b1f5fa92adf"
Jul 27 20:28:04.104477 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:04.104414    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ff422994bdf7bf9d289e6b2d65dc731a999e4e4a571afc4bc731b1f5fa92adf"} err="failed to get container status \"5ff422994bdf7bf9d289e6b2d65dc731a999e4e4a571afc4bc731b1f5fa92adf\": rpc error: code = NotFound desc = could not find container \"5ff422994bdf7bf9d289e6b2d65dc731a999e4e4a571afc4bc731b1f5fa92adf\": container with ID starting with 5ff422994bdf7bf9d289e6b2d65dc731a999e4e4a571afc4bc731b1f5fa92adf not found: ID does not exist"
Jul 27 20:28:04.115738 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:04.115714    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"]
Jul 27 20:28:04.119934 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:04.119911    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f44149/rate-test-model-f44149-kserve-5df88d447f-ng47l"]
Jul 27 20:28:04.261586 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:04.261560    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfa01a19-9b6a-4d62-a335-47e7addade73" path="/var/lib/kubelet/pods/cfa01a19-9b6a-4d62-a335-47e7addade73/volumes"
Jul 27 20:28:04.261977 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:04.261962    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d64285f5-13e5-400f-be86-913b84d4e212" path="/var/lib/kubelet/pods/d64285f5-13e5-400f-be86-913b84d4e212/volumes"
Jul 27 20:28:05.102165 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:05.102129    2579 generic.go:356] "Generic (PLEG): container finished" podID="beefd8c8-e591-4aaf-aa13-9aebb737a86b" containerID="5a895e9fee45314f0e5775208e30a5fc82d21759c577635de493dc833c49c72e" exitCode=0
Jul 27 20:28:05.102565 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:05.102204    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7342-139753588852320" event={"ID":"beefd8c8-e591-4aaf-aa13-9aebb737a86b","Type":"ContainerDied","Data":"5a895e9fee45314f0e5775208e30a5fc82d21759c577635de493dc833c49c72e"}
Jul 27 20:28:06.238088 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:06.238060    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7342-139753588852320"
Jul 27 20:28:06.257178 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:06.257154    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7342-139753588852320_beefd8c8-e591-4aaf-aa13-9aebb737a86b/test-curl-7342-139753588852320/0.log"
Jul 27 20:28:06.283318 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:06.283291    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7342-139753588852320"]
Jul 27 20:28:06.287402 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:06.287381    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7342-139753588852320"]
Jul 27 20:28:06.366118 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:06.366057    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6g6kd\" (UniqueName: \"kubernetes.io/projected/beefd8c8-e591-4aaf-aa13-9aebb737a86b-kube-api-access-6g6kd\") pod \"beefd8c8-e591-4aaf-aa13-9aebb737a86b\" (UID: \"beefd8c8-e591-4aaf-aa13-9aebb737a86b\") "
Jul 27 20:28:06.368180 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:06.368158    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/beefd8c8-e591-4aaf-aa13-9aebb737a86b-kube-api-access-6g6kd" (OuterVolumeSpecName: "kube-api-access-6g6kd") pod "beefd8c8-e591-4aaf-aa13-9aebb737a86b" (UID: "beefd8c8-e591-4aaf-aa13-9aebb737a86b"). InnerVolumeSpecName "kube-api-access-6g6kd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:28:06.467128 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:06.467103    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6g6kd\" (UniqueName: \"kubernetes.io/projected/beefd8c8-e591-4aaf-aa13-9aebb737a86b-kube-api-access-6g6kd\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:28:07.117391 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:07.117361    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7342-139753588852320"
Jul 27 20:28:07.117391 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:07.117390    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b220640118fc571f503a060ee3e1d82b7868d7564cfbf21423273467f195578c"
Jul 27 20:28:07.126826 ip-10-0-140-36 kubenswrapper[2579]: E0727 20:28:07.126802    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7342-139753588852320\" is forbidden: User \"system:node:ip-10-0-140-36.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-36.ec2.internal' and this object" podUID="beefd8c8-e591-4aaf-aa13-9aebb737a86b" pod="opendatahub/test-curl-7342-139753588852320"
Jul 27 20:28:08.260584 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:08.260547    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="beefd8c8-e591-4aaf-aa13-9aebb737a86b" path="/var/lib/kubelet/pods/beefd8c8-e591-4aaf-aa13-9aebb737a86b/volumes"
Jul 27 20:28:26.099990 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.099955    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"]
Jul 27 20:28:26.100436 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.100289    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d64285f5-13e5-400f-be86-913b84d4e212" containerName="main"
Jul 27 20:28:26.100436 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.100300    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d64285f5-13e5-400f-be86-913b84d4e212" containerName="main"
Jul 27 20:28:26.100436 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.100319    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="beefd8c8-e591-4aaf-aa13-9aebb737a86b" containerName="test-curl-7342-139753588852320"
Jul 27 20:28:26.100436 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.100325    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="beefd8c8-e591-4aaf-aa13-9aebb737a86b" containerName="test-curl-7342-139753588852320"
Jul 27 20:28:26.100436 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.100390    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="beefd8c8-e591-4aaf-aa13-9aebb737a86b" containerName="test-curl-7342-139753588852320"
Jul 27 20:28:26.100436 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.100398    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d64285f5-13e5-400f-be86-913b84d4e212" containerName="main"
Jul 27 20:28:26.105672 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.105652    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.107508 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.107486    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-1517d6-openshift-default-dockercfg-n47qm\""
Jul 27 20:28:26.107609 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.107492    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 27 20:28:26.113666 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.113647    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"]
Jul 27 20:28:26.213247 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.213222    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5457c054-c918-4a4e-becc-aaa0892bd712-istio-token\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.213380 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.213254    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-workload-certs\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.213380 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.213278    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-istio-envoy\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.213380 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.213345    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-istio-data\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.213380 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.213372    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5457c054-c918-4a4e-becc-aaa0892bd712-istiod-ca-cert\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.213518 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.213401    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-workload-socket\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.213518 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.213435    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-credential-socket\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.213518 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.213495    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5457c054-c918-4a4e-becc-aaa0892bd712-istio-podinfo\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.213615 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.213524    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glbgk\" (UniqueName: \"kubernetes.io/projected/5457c054-c918-4a4e-becc-aaa0892bd712-kube-api-access-glbgk\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.313911 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.313878    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-credential-socket\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.314010 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.313916    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5457c054-c918-4a4e-becc-aaa0892bd712-istio-podinfo\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.314010 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.313936    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glbgk\" (UniqueName: \"kubernetes.io/projected/5457c054-c918-4a4e-becc-aaa0892bd712-kube-api-access-glbgk\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.314010 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.313976    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5457c054-c918-4a4e-becc-aaa0892bd712-istio-token\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.314010 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.313996    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-workload-certs\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.314237 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.314017    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-istio-envoy\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.314237 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.314039    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-istio-data\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.314237 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.314065    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5457c054-c918-4a4e-becc-aaa0892bd712-istiod-ca-cert\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.314237 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.314102    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-workload-socket\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.314376 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.314297    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-credential-socket\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.314411 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.314370    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-workload-certs\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.314499 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.314480    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-istio-data\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.314542 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.314490    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-workload-socket\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.315038 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.315008    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5457c054-c918-4a4e-becc-aaa0892bd712-istiod-ca-cert\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.316373 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.316331    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-istio-envoy\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.316829 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.316808    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5457c054-c918-4a4e-becc-aaa0892bd712-istio-podinfo\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.322991 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.322972    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glbgk\" (UniqueName: \"kubernetes.io/projected/5457c054-c918-4a4e-becc-aaa0892bd712-kube-api-access-glbgk\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.323263 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.323245    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5457c054-c918-4a4e-becc-aaa0892bd712-istio-token\") pod \"e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") " pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.416730 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.416710    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:26.550260 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:26.550238    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"]
Jul 27 20:28:26.552887 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:28:26.552844    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5457c054_c918_4a4e_becc_aaa0892bd712.slice/crio-9225d3dcf9f370a4b11178b15a119ca0ff589545438dcd7f22d05feff13fe42e WatchSource:0}: Error finding container 9225d3dcf9f370a4b11178b15a119ca0ff589545438dcd7f22d05feff13fe42e: Status 404 returned error can't find the container with id 9225d3dcf9f370a4b11178b15a119ca0ff589545438dcd7f22d05feff13fe42e
Jul 27 20:28:27.208005 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:27.207972    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l" event={"ID":"5457c054-c918-4a4e-becc-aaa0892bd712","Type":"ContainerStarted","Data":"9225d3dcf9f370a4b11178b15a119ca0ff589545438dcd7f22d05feff13fe42e"}
Jul 27 20:28:29.373411 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:29.373367    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 20:28:29.373777 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:29.373462    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 20:28:29.373777 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:29.373508    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 20:28:30.224930 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:30.224894    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l" event={"ID":"5457c054-c918-4a4e-becc-aaa0892bd712","Type":"ContainerStarted","Data":"c9903bed1bbdfb110529ce0280c8604fb52d5c9240bb13674fd720b049a281bb"}
Jul 27 20:28:30.243522 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:30.243471    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l" podStartSLOduration=1.4252978490000001 podStartE2EDuration="4.243457872s" totalImagesPullingTime="2.818160023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:28:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:28:30.24225954 +0000 UTC m=+2938.471501740" watchObservedRunningTime="2026-07-27 20:28:30.243457872 +0000 UTC m=+2938.472700069"
Jul 27 20:28:30.417365 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:30.417334    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:30.422078 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:30.422053    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:31.229930 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:31.229894    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:31.230947 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:31.230928    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:51.321308 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:51.321268    2579 generic.go:356] "Generic (PLEG): container finished" podID="5457c054-c918-4a4e-becc-aaa0892bd712" containerID="c9903bed1bbdfb110529ce0280c8604fb52d5c9240bb13674fd720b049a281bb" exitCode=0
Jul 27 20:28:51.321731 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:51.321340    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l" event={"ID":"5457c054-c918-4a4e-becc-aaa0892bd712","Type":"ContainerDied","Data":"c9903bed1bbdfb110529ce0280c8604fb52d5c9240bb13674fd720b049a281bb"}
Jul 27 20:28:51.321731 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:51.321708    2579 scope.go:117] "RemoveContainer" containerID="c9903bed1bbdfb110529ce0280c8604fb52d5c9240bb13674fd720b049a281bb"
Jul 27 20:28:51.322741 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:51.322712    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 20:28:51.322807 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:51.322780    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 20:28:51.322847 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:51.322810    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 20:28:52.328148 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:52.328118    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l" event={"ID":"5457c054-c918-4a4e-becc-aaa0892bd712","Type":"ContainerStarted","Data":"60f5e1a4c2d85b4b33922d2cda60b7ff21a94ef15114ad6254e4ad34aa9c1991"}
Jul 27 20:28:52.417191 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:52.417150    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:52.417442 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:52.417419    2579 patch_prober.go:28] interesting pod/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 27 20:28:52.417530 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:52.417475    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l" podUID="5457c054-c918-4a4e-becc-aaa0892bd712" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 27 20:28:53.418093 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:53.418058    2579 patch_prober.go:28] interesting pod/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 27 20:28:53.418554 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:53.418118    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l" podUID="5457c054-c918-4a4e-becc-aaa0892bd712" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 27 20:28:54.417875 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:54.417820    2579 patch_prober.go:28] interesting pod/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 27 20:28:54.418066 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:54.417897    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l" podUID="5457c054-c918-4a4e-becc-aaa0892bd712" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 27 20:28:55.417572 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:55.417539    2579 patch_prober.go:28] interesting pod/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 27 20:28:55.417969 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:55.417596    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l" podUID="5457c054-c918-4a4e-becc-aaa0892bd712" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 27 20:28:56.417591 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:56.417556    2579 patch_prober.go:28] interesting pod/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 27 20:28:56.418042 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:56.417610    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l" podUID="5457c054-c918-4a4e-becc-aaa0892bd712" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 27 20:28:57.420779 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:57.420750    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:57.421287 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:57.421174    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:28:57.421701 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:28:57.421681    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:29:00.140590 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.140557    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"]
Jul 27 20:29:00.147340 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.147314    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:00.150136 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.150115    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-555352\"/\"openshift-service-ca.crt\""
Jul 27 20:29:00.150277 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.150247    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-555352\"/\"ipp-route-555352-kserve-self-signed-certs\""
Jul 27 20:29:00.150412 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.150160    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-555352\"/\"kube-root-ca.crt\""
Jul 27 20:29:00.150663 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.150620    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-555352\"/\"default-dockercfg-5kn2p\""
Jul 27 20:29:00.152909 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.152889    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"]
Jul 27 20:29:00.183827 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.183806    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fea2b786-1217-41f6-9751-bd0aae48bbad-tls-certs\") pod \"ipp-route-555352-kserve-59fdb87f4c-m94zz\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") " pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:00.183940 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.183842    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-home\") pod \"ipp-route-555352-kserve-59fdb87f4c-m94zz\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") " pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:00.183940 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.183885    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-tmp-dir\") pod \"ipp-route-555352-kserve-59fdb87f4c-m94zz\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") " pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:00.183940 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.183903    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-dshm\") pod \"ipp-route-555352-kserve-59fdb87f4c-m94zz\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") " pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:00.184077 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.183945    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l7s6\" (UniqueName: \"kubernetes.io/projected/fea2b786-1217-41f6-9751-bd0aae48bbad-kube-api-access-9l7s6\") pod \"ipp-route-555352-kserve-59fdb87f4c-m94zz\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") " pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:00.184077 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.184014    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-model-cache\") pod \"ipp-route-555352-kserve-59fdb87f4c-m94zz\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") " pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:00.284733 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.284708    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l7s6\" (UniqueName: \"kubernetes.io/projected/fea2b786-1217-41f6-9751-bd0aae48bbad-kube-api-access-9l7s6\") pod \"ipp-route-555352-kserve-59fdb87f4c-m94zz\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") " pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:00.284831 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.284753    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-model-cache\") pod \"ipp-route-555352-kserve-59fdb87f4c-m94zz\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") " pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:00.284831 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.284773    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fea2b786-1217-41f6-9751-bd0aae48bbad-tls-certs\") pod \"ipp-route-555352-kserve-59fdb87f4c-m94zz\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") " pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:00.284831 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.284796    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-home\") pod \"ipp-route-555352-kserve-59fdb87f4c-m94zz\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") " pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:00.284831 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.284812    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-tmp-dir\") pod \"ipp-route-555352-kserve-59fdb87f4c-m94zz\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") " pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:00.284831 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.284831    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-dshm\") pod \"ipp-route-555352-kserve-59fdb87f4c-m94zz\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") " pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:00.285155 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.285139    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-model-cache\") pod \"ipp-route-555352-kserve-59fdb87f4c-m94zz\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") " pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:00.285205 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.285190    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-home\") pod \"ipp-route-555352-kserve-59fdb87f4c-m94zz\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") " pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:00.285297 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.285282    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-tmp-dir\") pod \"ipp-route-555352-kserve-59fdb87f4c-m94zz\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") " pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:00.287214 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.287193    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-dshm\") pod \"ipp-route-555352-kserve-59fdb87f4c-m94zz\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") " pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:00.287387 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.287369    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fea2b786-1217-41f6-9751-bd0aae48bbad-tls-certs\") pod \"ipp-route-555352-kserve-59fdb87f4c-m94zz\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") " pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:00.292090 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.292069    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l7s6\" (UniqueName: \"kubernetes.io/projected/fea2b786-1217-41f6-9751-bd0aae48bbad-kube-api-access-9l7s6\") pod \"ipp-route-555352-kserve-59fdb87f4c-m94zz\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") " pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:00.458209 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.458182    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:00.597588 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:00.597554    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"]
Jul 27 20:29:00.599662 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:29:00.599633    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfea2b786_1217_41f6_9751_bd0aae48bbad.slice/crio-f4698cb1028a7c7d4eb5e44e8d56f4815151da763c567bd3e13b101709341844 WatchSource:0}: Error finding container f4698cb1028a7c7d4eb5e44e8d56f4815151da763c567bd3e13b101709341844: Status 404 returned error can't find the container with id f4698cb1028a7c7d4eb5e44e8d56f4815151da763c567bd3e13b101709341844
Jul 27 20:29:01.368878 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:01.368823    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz" event={"ID":"fea2b786-1217-41f6-9751-bd0aae48bbad","Type":"ContainerStarted","Data":"460b7f342a68b16e5246db3576ac6ca1ebd2898c163b0248c973a29b91287735"}
Jul 27 20:29:01.369264 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:01.368892    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz" event={"ID":"fea2b786-1217-41f6-9751-bd0aae48bbad","Type":"ContainerStarted","Data":"f4698cb1028a7c7d4eb5e44e8d56f4815151da763c567bd3e13b101709341844"}
Jul 27 20:29:01.389103 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:01.389060    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz" podStartSLOduration=1.389046403 podStartE2EDuration="1.389046403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:29:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:29:01.387657008 +0000 UTC m=+2969.616899205" watchObservedRunningTime="2026-07-27 20:29:01.389046403 +0000 UTC m=+2969.618288603"
Jul 27 20:29:10.458269 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:10.458233    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:10.458622 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:10.458284    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:10.470766 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:10.470745    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:11.427056 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:11.427031    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:29:32.457537 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:32.457509    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 20:29:32.466712 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:32.466689    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 20:29:54.651930 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.651901    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"]
Jul 27 20:29:54.655228 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.655208    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.657564 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.657542    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-63bd67-openshift-default-dockercfg-6hwxz\""
Jul 27 20:29:54.672667 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.672640    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"]
Jul 27 20:29:54.814723 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.814686    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-workload-socket\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.814898 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.814746    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c478127-9459-4aef-afc4-755f2cc02481-istio-token\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.814898 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.814786    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-istio-data\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.814898 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.814818    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-credential-socket\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.814898 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.814876    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-workload-certs\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.815078 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.814902    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c478127-9459-4aef-afc4-755f2cc02481-istiod-ca-cert\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.815078 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.814925    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdk66\" (UniqueName: \"kubernetes.io/projected/4c478127-9459-4aef-afc4-755f2cc02481-kube-api-access-rdk66\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.815078 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.814997    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-istio-envoy\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.815078 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.815014    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c478127-9459-4aef-afc4-755f2cc02481-istio-podinfo\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.915561 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.915540    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-credential-socket\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.915698 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.915573    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-workload-certs\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.915698 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.915602    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c478127-9459-4aef-afc4-755f2cc02481-istiod-ca-cert\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.915698 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.915627    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdk66\" (UniqueName: \"kubernetes.io/projected/4c478127-9459-4aef-afc4-755f2cc02481-kube-api-access-rdk66\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.915903 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.915703    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-istio-envoy\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.915903 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.915727    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c478127-9459-4aef-afc4-755f2cc02481-istio-podinfo\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.915903 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.915760    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-workload-socket\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.915903 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.915827    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c478127-9459-4aef-afc4-755f2cc02481-istio-token\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.915903 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.915882    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-istio-data\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.916177 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.916003    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-workload-certs\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.916177 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.916029    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-credential-socket\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.916266 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.916241    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-istio-data\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.916364 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.916347    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-workload-socket\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.916533 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.916506    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c478127-9459-4aef-afc4-755f2cc02481-istiod-ca-cert\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.918190 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.918168    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-istio-envoy\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.918578 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.918561    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c478127-9459-4aef-afc4-755f2cc02481-istio-podinfo\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.923309 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.923285    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdk66\" (UniqueName: \"kubernetes.io/projected/4c478127-9459-4aef-afc4-755f2cc02481-kube-api-access-rdk66\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.923422 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.923295    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c478127-9459-4aef-afc4-755f2cc02481-istio-token\") pod \"e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") " pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:54.964971 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:54.964947    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:55.099087 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:55.099047    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"]
Jul 27 20:29:55.101459 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:29:55.101427    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c478127_9459_4aef_afc4_755f2cc02481.slice/crio-b5d5424651b33fc7390468bb213c40160ede4cf66050b68a8b76f5d930c730dd WatchSource:0}: Error finding container b5d5424651b33fc7390468bb213c40160ede4cf66050b68a8b76f5d930c730dd: Status 404 returned error can't find the container with id b5d5424651b33fc7390468bb213c40160ede4cf66050b68a8b76f5d930c730dd
Jul 27 20:29:55.104032 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:55.103991    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 20:29:55.104116 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:55.104080    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 20:29:55.104155 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:55.104123    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 20:29:55.614258 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:55.614220    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp" event={"ID":"4c478127-9459-4aef-afc4-755f2cc02481","Type":"ContainerStarted","Data":"6e2777139be2ae4a676a2a8b81ea31f4f3b286cc76948d4ec6474f8fc1c2c03b"}
Jul 27 20:29:55.614424 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:55.614298    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp" event={"ID":"4c478127-9459-4aef-afc4-755f2cc02481","Type":"ContainerStarted","Data":"b5d5424651b33fc7390468bb213c40160ede4cf66050b68a8b76f5d930c730dd"}
Jul 27 20:29:55.631900 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:55.631833    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp" podStartSLOduration=1.631820248 podStartE2EDuration="1.631820248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:29:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:29:55.630111617 +0000 UTC m=+3023.859353815" watchObservedRunningTime="2026-07-27 20:29:55.631820248 +0000 UTC m=+3023.861062497"
Jul 27 20:29:55.965700 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:55.965671    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:56.970888 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:56.970843    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:57.623786 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:57.623700    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:29:57.624672 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:29:57.624654    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:30:00.134075 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:00.134037    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753070-cxfjr"]
Jul 27 20:30:00.137813 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:00.137790    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753070-cxfjr"
Jul 27 20:30:00.139458 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:00.139431    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x6cbr\""
Jul 27 20:30:00.146138 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:00.145959    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753070-cxfjr"]
Jul 27 20:30:00.148976 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:00.148956    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753070-cxfjr"
Jul 27 20:30:00.289754 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:00.289723    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753070-cxfjr"]
Jul 27 20:30:00.290947 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:30:00.290918    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45882e75_3204_42ad_bef9_b456078ae525.slice/crio-d7c38dcd738d6fbd8e1c4c110000886d35a15624f6b4b0652f217709f5bfa91e WatchSource:0}: Error finding container d7c38dcd738d6fbd8e1c4c110000886d35a15624f6b4b0652f217709f5bfa91e: Status 404 returned error can't find the container with id d7c38dcd738d6fbd8e1c4c110000886d35a15624f6b4b0652f217709f5bfa91e
Jul 27 20:30:00.637790 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:00.637758    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753070-cxfjr" event={"ID":"45882e75-3204-42ad-bef9-b456078ae525","Type":"ContainerStarted","Data":"8e8b134dd53f420c7968f24be1b30c003430dae69c94b2c738137051496eaf8d"}
Jul 27 20:30:00.637956 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:00.637836    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753070-cxfjr" event={"ID":"45882e75-3204-42ad-bef9-b456078ae525","Type":"ContainerStarted","Data":"d7c38dcd738d6fbd8e1c4c110000886d35a15624f6b4b0652f217709f5bfa91e"}
Jul 27 20:30:00.653101 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:00.653059    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753070-cxfjr" podStartSLOduration=0.653045081 podStartE2EDuration="653.045081ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:30:00.651044655 +0000 UTC m=+3028.880286852" watchObservedRunningTime="2026-07-27 20:30:00.653045081 +0000 UTC m=+3028.882287283"
Jul 27 20:30:01.643527 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:01.643489    2579 generic.go:356] "Generic (PLEG): container finished" podID="45882e75-3204-42ad-bef9-b456078ae525" containerID="8e8b134dd53f420c7968f24be1b30c003430dae69c94b2c738137051496eaf8d" exitCode=0
Jul 27 20:30:01.643926 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:01.643573    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753070-cxfjr" event={"ID":"45882e75-3204-42ad-bef9-b456078ae525","Type":"ContainerDied","Data":"8e8b134dd53f420c7968f24be1b30c003430dae69c94b2c738137051496eaf8d"}
Jul 27 20:30:02.775551 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:02.775522    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753070-cxfjr"
Jul 27 20:30:03.654993 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:03.654955    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753070-cxfjr"
Jul 27 20:30:03.655237 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:03.654961    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753070-cxfjr" event={"ID":"45882e75-3204-42ad-bef9-b456078ae525","Type":"ContainerDied","Data":"d7c38dcd738d6fbd8e1c4c110000886d35a15624f6b4b0652f217709f5bfa91e"}
Jul 27 20:30:03.655237 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:03.655102    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d7c38dcd738d6fbd8e1c4c110000886d35a15624f6b4b0652f217709f5bfa91e"
Jul 27 20:30:18.734039 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:18.734001    2579 generic.go:356] "Generic (PLEG): container finished" podID="4c478127-9459-4aef-afc4-755f2cc02481" containerID="6e2777139be2ae4a676a2a8b81ea31f4f3b286cc76948d4ec6474f8fc1c2c03b" exitCode=0
Jul 27 20:30:18.734536 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:18.734075    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp" event={"ID":"4c478127-9459-4aef-afc4-755f2cc02481","Type":"ContainerDied","Data":"6e2777139be2ae4a676a2a8b81ea31f4f3b286cc76948d4ec6474f8fc1c2c03b"}
Jul 27 20:30:18.734609 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:18.734576    2579 scope.go:117] "RemoveContainer" containerID="6e2777139be2ae4a676a2a8b81ea31f4f3b286cc76948d4ec6474f8fc1c2c03b"
Jul 27 20:30:18.735691 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:18.735658    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 20:30:18.735752 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:18.735738    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 20:30:18.735789 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:18.735775    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 20:30:19.741075 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:19.741036    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp" event={"ID":"4c478127-9459-4aef-afc4-755f2cc02481","Type":"ContainerStarted","Data":"269d282d766834b811870f5a249e20cae040f99197da96dc532c32089d0680b4"}
Jul 27 20:30:19.966026 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:19.965992    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:30:19.966256 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:19.966232    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 27 20:30:19.966310 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:19.966291    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp" podUID="4c478127-9459-4aef-afc4-755f2cc02481" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 27 20:30:20.965596 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:20.965562    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 27 20:30:20.966012 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:20.965615    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp" podUID="4c478127-9459-4aef-afc4-755f2cc02481" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 27 20:30:21.966065 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:21.966026    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 27 20:30:21.966474 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:21.966088    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp" podUID="4c478127-9459-4aef-afc4-755f2cc02481" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 27 20:30:22.966187 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:22.966136    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 27 20:30:22.966568 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:22.966208    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp" podUID="4c478127-9459-4aef-afc4-755f2cc02481" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 27 20:30:23.969963 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:23.969881    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:30:23.970384 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:23.970155    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:30:23.970792 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:23.970777    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:30:40.305178 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:40.305148    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"]
Jul 27 20:30:40.305704 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:40.305358    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp" podUID="4c478127-9459-4aef-afc4-755f2cc02481" containerName="istio-proxy" containerID="cri-o://269d282d766834b811870f5a249e20cae040f99197da96dc532c32089d0680b4" gracePeriod=30
Jul 27 20:30:45.550463 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.550441    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:30:45.626495 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.626429    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c478127-9459-4aef-afc4-755f2cc02481-istio-token\") pod \"4c478127-9459-4aef-afc4-755f2cc02481\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") "
Jul 27 20:30:45.626495 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.626459    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-istio-data\") pod \"4c478127-9459-4aef-afc4-755f2cc02481\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") "
Jul 27 20:30:45.626495 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.626490    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-workload-certs\") pod \"4c478127-9459-4aef-afc4-755f2cc02481\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") "
Jul 27 20:30:45.626750 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.626506    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdk66\" (UniqueName: \"kubernetes.io/projected/4c478127-9459-4aef-afc4-755f2cc02481-kube-api-access-rdk66\") pod \"4c478127-9459-4aef-afc4-755f2cc02481\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") "
Jul 27 20:30:45.626750 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.626534    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-credential-socket\") pod \"4c478127-9459-4aef-afc4-755f2cc02481\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") "
Jul 27 20:30:45.626750 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.626556    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c478127-9459-4aef-afc4-755f2cc02481-istiod-ca-cert\") pod \"4c478127-9459-4aef-afc4-755f2cc02481\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") "
Jul 27 20:30:45.626750 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.626621    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c478127-9459-4aef-afc4-755f2cc02481-istio-podinfo\") pod \"4c478127-9459-4aef-afc4-755f2cc02481\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") "
Jul 27 20:30:45.626750 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.626679    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-istio-envoy\") pod \"4c478127-9459-4aef-afc4-755f2cc02481\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") "
Jul 27 20:30:45.626750 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.626728    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-workload-socket\") pod \"4c478127-9459-4aef-afc4-755f2cc02481\" (UID: \"4c478127-9459-4aef-afc4-755f2cc02481\") "
Jul 27 20:30:45.627096 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.626800    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4c478127-9459-4aef-afc4-755f2cc02481" (UID: "4c478127-9459-4aef-afc4-755f2cc02481"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:30:45.627096 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.626824    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4c478127-9459-4aef-afc4-755f2cc02481" (UID: "4c478127-9459-4aef-afc4-755f2cc02481"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:30:45.627096 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.626999    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4c478127-9459-4aef-afc4-755f2cc02481" (UID: "4c478127-9459-4aef-afc4-755f2cc02481"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:30:45.627252 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.627110    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-workload-socket\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:30:45.627252 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.627132    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-workload-certs\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:30:45.627252 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.627146    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-credential-socket\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:30:45.627252 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.627188    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c478127-9459-4aef-afc4-755f2cc02481-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4c478127-9459-4aef-afc4-755f2cc02481" (UID: "4c478127-9459-4aef-afc4-755f2cc02481"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:30:45.627489 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.627467    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-istio-data" (OuterVolumeSpecName: "istio-data") pod "4c478127-9459-4aef-afc4-755f2cc02481" (UID: "4c478127-9459-4aef-afc4-755f2cc02481"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:30:45.629093 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.629069    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4c478127-9459-4aef-afc4-755f2cc02481-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4c478127-9459-4aef-afc4-755f2cc02481" (UID: "4c478127-9459-4aef-afc4-755f2cc02481"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 20:30:45.629093 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.629080    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c478127-9459-4aef-afc4-755f2cc02481-kube-api-access-rdk66" (OuterVolumeSpecName: "kube-api-access-rdk66") pod "4c478127-9459-4aef-afc4-755f2cc02481" (UID: "4c478127-9459-4aef-afc4-755f2cc02481"). InnerVolumeSpecName "kube-api-access-rdk66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:30:45.629366 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.629348    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c478127-9459-4aef-afc4-755f2cc02481-istio-token" (OuterVolumeSpecName: "istio-token") pod "4c478127-9459-4aef-afc4-755f2cc02481" (UID: "4c478127-9459-4aef-afc4-755f2cc02481"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:30:45.629408 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.629391    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4c478127-9459-4aef-afc4-755f2cc02481" (UID: "4c478127-9459-4aef-afc4-755f2cc02481"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:30:45.727922 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.727899    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c478127-9459-4aef-afc4-755f2cc02481-istio-token\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:30:45.727922 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.727921    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-istio-data\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:30:45.728053 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.727933    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdk66\" (UniqueName: \"kubernetes.io/projected/4c478127-9459-4aef-afc4-755f2cc02481-kube-api-access-rdk66\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:30:45.728053 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.727946    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c478127-9459-4aef-afc4-755f2cc02481-istiod-ca-cert\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:30:45.728053 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.727957    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c478127-9459-4aef-afc4-755f2cc02481-istio-podinfo\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:30:45.728053 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.727969    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c478127-9459-4aef-afc4-755f2cc02481-istio-envoy\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:30:45.865374 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.865350    2579 generic.go:356] "Generic (PLEG): container finished" podID="4c478127-9459-4aef-afc4-755f2cc02481" containerID="269d282d766834b811870f5a249e20cae040f99197da96dc532c32089d0680b4" exitCode=0
Jul 27 20:30:45.865465 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.865402    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"
Jul 27 20:30:45.865465 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.865429    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp" event={"ID":"4c478127-9459-4aef-afc4-755f2cc02481","Type":"ContainerDied","Data":"269d282d766834b811870f5a249e20cae040f99197da96dc532c32089d0680b4"}
Jul 27 20:30:45.865553 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.865476    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp" event={"ID":"4c478127-9459-4aef-afc4-755f2cc02481","Type":"ContainerDied","Data":"b5d5424651b33fc7390468bb213c40160ede4cf66050b68a8b76f5d930c730dd"}
Jul 27 20:30:45.865553 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.865508    2579 scope.go:117] "RemoveContainer" containerID="269d282d766834b811870f5a249e20cae040f99197da96dc532c32089d0680b4"
Jul 27 20:30:45.874539 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.874523    2579 scope.go:117] "RemoveContainer" containerID="6e2777139be2ae4a676a2a8b81ea31f4f3b286cc76948d4ec6474f8fc1c2c03b"
Jul 27 20:30:45.882195 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.882179    2579 scope.go:117] "RemoveContainer" containerID="269d282d766834b811870f5a249e20cae040f99197da96dc532c32089d0680b4"
Jul 27 20:30:45.882420 ip-10-0-140-36 kubenswrapper[2579]: E0727 20:30:45.882401    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"269d282d766834b811870f5a249e20cae040f99197da96dc532c32089d0680b4\": container with ID starting with 269d282d766834b811870f5a249e20cae040f99197da96dc532c32089d0680b4 not found: ID does not exist" containerID="269d282d766834b811870f5a249e20cae040f99197da96dc532c32089d0680b4"
Jul 27 20:30:45.882463 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.882429    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"269d282d766834b811870f5a249e20cae040f99197da96dc532c32089d0680b4"} err="failed to get container status \"269d282d766834b811870f5a249e20cae040f99197da96dc532c32089d0680b4\": rpc error: code = NotFound desc = could not find container \"269d282d766834b811870f5a249e20cae040f99197da96dc532c32089d0680b4\": container with ID starting with 269d282d766834b811870f5a249e20cae040f99197da96dc532c32089d0680b4 not found: ID does not exist"
Jul 27 20:30:45.882463 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.882446    2579 scope.go:117] "RemoveContainer" containerID="6e2777139be2ae4a676a2a8b81ea31f4f3b286cc76948d4ec6474f8fc1c2c03b"
Jul 27 20:30:45.882718 ip-10-0-140-36 kubenswrapper[2579]: E0727 20:30:45.882699    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e2777139be2ae4a676a2a8b81ea31f4f3b286cc76948d4ec6474f8fc1c2c03b\": container with ID starting with 6e2777139be2ae4a676a2a8b81ea31f4f3b286cc76948d4ec6474f8fc1c2c03b not found: ID does not exist" containerID="6e2777139be2ae4a676a2a8b81ea31f4f3b286cc76948d4ec6474f8fc1c2c03b"
Jul 27 20:30:45.882772 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.882723    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e2777139be2ae4a676a2a8b81ea31f4f3b286cc76948d4ec6474f8fc1c2c03b"} err="failed to get container status \"6e2777139be2ae4a676a2a8b81ea31f4f3b286cc76948d4ec6474f8fc1c2c03b\": rpc error: code = NotFound desc = could not find container \"6e2777139be2ae4a676a2a8b81ea31f4f3b286cc76948d4ec6474f8fc1c2c03b\": container with ID starting with 6e2777139be2ae4a676a2a8b81ea31f4f3b286cc76948d4ec6474f8fc1c2c03b not found: ID does not exist"
Jul 27 20:30:45.885175 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.885155    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"]
Jul 27 20:30:45.888786 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:45.888769    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-63bd67-openshift-default-786b98449b-z9whp"]
Jul 27 20:30:46.261498 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:30:46.261469    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c478127-9459-4aef-afc4-755f2cc02481" path="/var/lib/kubelet/pods/4c478127-9459-4aef-afc4-755f2cc02481/volumes"
Jul 27 20:31:14.174429 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:14.174389    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"]
Jul 27 20:31:29.193721 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:29.193682    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz" podUID="fea2b786-1217-41f6-9751-bd0aae48bbad" containerName="main" containerID="cri-o://460b7f342a68b16e5246db3576ac6ca1ebd2898c163b0248c973a29b91287735" gracePeriod=45
Jul 27 20:31:29.449181 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:29.449121    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:31:29.572172 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:29.572144    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-model-cache\") pod \"fea2b786-1217-41f6-9751-bd0aae48bbad\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") "
Jul 27 20:31:29.572345 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:29.572194    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-home\") pod \"fea2b786-1217-41f6-9751-bd0aae48bbad\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") "
Jul 27 20:31:29.572345 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:29.572222    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fea2b786-1217-41f6-9751-bd0aae48bbad-tls-certs\") pod \"fea2b786-1217-41f6-9751-bd0aae48bbad\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") "
Jul 27 20:31:29.572345 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:29.572264    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-dshm\") pod \"fea2b786-1217-41f6-9751-bd0aae48bbad\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") "
Jul 27 20:31:29.572345 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:29.572289    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9l7s6\" (UniqueName: \"kubernetes.io/projected/fea2b786-1217-41f6-9751-bd0aae48bbad-kube-api-access-9l7s6\") pod \"fea2b786-1217-41f6-9751-bd0aae48bbad\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") "
Jul 27 20:31:29.572345 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:29.572324    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-tmp-dir\") pod \"fea2b786-1217-41f6-9751-bd0aae48bbad\" (UID: \"fea2b786-1217-41f6-9751-bd0aae48bbad\") "
Jul 27 20:31:29.572600 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:29.572393    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-model-cache" (OuterVolumeSpecName: "model-cache") pod "fea2b786-1217-41f6-9751-bd0aae48bbad" (UID: "fea2b786-1217-41f6-9751-bd0aae48bbad"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:31:29.572600 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:29.572410    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-home" (OuterVolumeSpecName: "home") pod "fea2b786-1217-41f6-9751-bd0aae48bbad" (UID: "fea2b786-1217-41f6-9751-bd0aae48bbad"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:31:29.572600 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:29.572584    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-model-cache\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:31:29.572600 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:29.572602    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-home\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:31:29.572780 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:29.572629    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fea2b786-1217-41f6-9751-bd0aae48bbad" (UID: "fea2b786-1217-41f6-9751-bd0aae48bbad"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:31:29.574624 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:29.574596    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-dshm" (OuterVolumeSpecName: "dshm") pod "fea2b786-1217-41f6-9751-bd0aae48bbad" (UID: "fea2b786-1217-41f6-9751-bd0aae48bbad"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:31:29.574759 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:29.574736    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fea2b786-1217-41f6-9751-bd0aae48bbad-kube-api-access-9l7s6" (OuterVolumeSpecName: "kube-api-access-9l7s6") pod "fea2b786-1217-41f6-9751-bd0aae48bbad" (UID: "fea2b786-1217-41f6-9751-bd0aae48bbad"). InnerVolumeSpecName "kube-api-access-9l7s6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:31:29.574821 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:29.574797    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fea2b786-1217-41f6-9751-bd0aae48bbad-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fea2b786-1217-41f6-9751-bd0aae48bbad" (UID: "fea2b786-1217-41f6-9751-bd0aae48bbad"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:31:29.673576 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:29.673550    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fea2b786-1217-41f6-9751-bd0aae48bbad-tls-certs\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:31:29.673576 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:29.673572    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-dshm\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:31:29.673720 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:29.673583    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9l7s6\" (UniqueName: \"kubernetes.io/projected/fea2b786-1217-41f6-9751-bd0aae48bbad-kube-api-access-9l7s6\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:31:29.673720 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:29.673592    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fea2b786-1217-41f6-9751-bd0aae48bbad-tmp-dir\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:31:30.065613 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:30.065579    2579 generic.go:356] "Generic (PLEG): container finished" podID="fea2b786-1217-41f6-9751-bd0aae48bbad" containerID="460b7f342a68b16e5246db3576ac6ca1ebd2898c163b0248c973a29b91287735" exitCode=0
Jul 27 20:31:30.065778 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:30.065628    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"
Jul 27 20:31:30.065778 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:30.065630    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz" event={"ID":"fea2b786-1217-41f6-9751-bd0aae48bbad","Type":"ContainerDied","Data":"460b7f342a68b16e5246db3576ac6ca1ebd2898c163b0248c973a29b91287735"}
Jul 27 20:31:30.065778 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:30.065760    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz" event={"ID":"fea2b786-1217-41f6-9751-bd0aae48bbad","Type":"ContainerDied","Data":"f4698cb1028a7c7d4eb5e44e8d56f4815151da763c567bd3e13b101709341844"}
Jul 27 20:31:30.065977 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:30.065793    2579 scope.go:117] "RemoveContainer" containerID="460b7f342a68b16e5246db3576ac6ca1ebd2898c163b0248c973a29b91287735"
Jul 27 20:31:30.074795 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:30.074778    2579 scope.go:117] "RemoveContainer" containerID="460b7f342a68b16e5246db3576ac6ca1ebd2898c163b0248c973a29b91287735"
Jul 27 20:31:30.075053 ip-10-0-140-36 kubenswrapper[2579]: E0727 20:31:30.075033    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"460b7f342a68b16e5246db3576ac6ca1ebd2898c163b0248c973a29b91287735\": container with ID starting with 460b7f342a68b16e5246db3576ac6ca1ebd2898c163b0248c973a29b91287735 not found: ID does not exist" containerID="460b7f342a68b16e5246db3576ac6ca1ebd2898c163b0248c973a29b91287735"
Jul 27 20:31:30.075121 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:30.075060    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"460b7f342a68b16e5246db3576ac6ca1ebd2898c163b0248c973a29b91287735"} err="failed to get container status \"460b7f342a68b16e5246db3576ac6ca1ebd2898c163b0248c973a29b91287735\": rpc error: code = NotFound desc = could not find container \"460b7f342a68b16e5246db3576ac6ca1ebd2898c163b0248c973a29b91287735\": container with ID starting with 460b7f342a68b16e5246db3576ac6ca1ebd2898c163b0248c973a29b91287735 not found: ID does not exist"
Jul 27 20:31:30.084622 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:30.084596    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"]
Jul 27 20:31:30.086138 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:30.086120    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-555352/ipp-route-555352-kserve-59fdb87f4c-m94zz"]
Jul 27 20:31:30.261062 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:31:30.261036    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fea2b786-1217-41f6-9751-bd0aae48bbad" path="/var/lib/kubelet/pods/fea2b786-1217-41f6-9751-bd0aae48bbad/volumes"
Jul 27 20:32:35.286064 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:35.286031    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"]
Jul 27 20:32:35.286576 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:35.286321    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l" podUID="5457c054-c918-4a4e-becc-aaa0892bd712" containerName="istio-proxy" containerID="cri-o://60f5e1a4c2d85b4b33922d2cda60b7ff21a94ef15114ad6254e4ad34aa9c1991" gracePeriod=30
Jul 27 20:32:40.532635 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.532603    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:32:40.627401 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.627332    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-istio-envoy\") pod \"5457c054-c918-4a4e-becc-aaa0892bd712\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") "
Jul 27 20:32:40.627401 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.627364    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-istio-data\") pod \"5457c054-c918-4a4e-becc-aaa0892bd712\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") "
Jul 27 20:32:40.627592 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.627414    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5457c054-c918-4a4e-becc-aaa0892bd712-istio-token\") pod \"5457c054-c918-4a4e-becc-aaa0892bd712\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") "
Jul 27 20:32:40.627592 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.627430    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5457c054-c918-4a4e-becc-aaa0892bd712-istio-podinfo\") pod \"5457c054-c918-4a4e-becc-aaa0892bd712\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") "
Jul 27 20:32:40.627592 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.627453    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-workload-socket\") pod \"5457c054-c918-4a4e-becc-aaa0892bd712\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") "
Jul 27 20:32:40.627592 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.627471    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-workload-certs\") pod \"5457c054-c918-4a4e-becc-aaa0892bd712\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") "
Jul 27 20:32:40.627592 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.627517    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-credential-socket\") pod \"5457c054-c918-4a4e-becc-aaa0892bd712\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") "
Jul 27 20:32:40.627592 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.627541    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glbgk\" (UniqueName: \"kubernetes.io/projected/5457c054-c918-4a4e-becc-aaa0892bd712-kube-api-access-glbgk\") pod \"5457c054-c918-4a4e-becc-aaa0892bd712\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") "
Jul 27 20:32:40.627907 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.627595    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5457c054-c918-4a4e-becc-aaa0892bd712-istiod-ca-cert\") pod \"5457c054-c918-4a4e-becc-aaa0892bd712\" (UID: \"5457c054-c918-4a4e-becc-aaa0892bd712\") "
Jul 27 20:32:40.627907 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.627840    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5457c054-c918-4a4e-becc-aaa0892bd712" (UID: "5457c054-c918-4a4e-becc-aaa0892bd712"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:32:40.627907 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.627879    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5457c054-c918-4a4e-becc-aaa0892bd712" (UID: "5457c054-c918-4a4e-becc-aaa0892bd712"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:32:40.628159 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.628136    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5457c054-c918-4a4e-becc-aaa0892bd712-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5457c054-c918-4a4e-becc-aaa0892bd712" (UID: "5457c054-c918-4a4e-becc-aaa0892bd712"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:32:40.628382 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.628339    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5457c054-c918-4a4e-becc-aaa0892bd712" (UID: "5457c054-c918-4a4e-becc-aaa0892bd712"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:32:40.628705 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.628675    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-istio-data" (OuterVolumeSpecName: "istio-data") pod "5457c054-c918-4a4e-becc-aaa0892bd712" (UID: "5457c054-c918-4a4e-becc-aaa0892bd712"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:32:40.629704 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.629685    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5457c054-c918-4a4e-becc-aaa0892bd712-istio-token" (OuterVolumeSpecName: "istio-token") pod "5457c054-c918-4a4e-becc-aaa0892bd712" (UID: "5457c054-c918-4a4e-becc-aaa0892bd712"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:32:40.629909 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.629886    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5457c054-c918-4a4e-becc-aaa0892bd712-kube-api-access-glbgk" (OuterVolumeSpecName: "kube-api-access-glbgk") pod "5457c054-c918-4a4e-becc-aaa0892bd712" (UID: "5457c054-c918-4a4e-becc-aaa0892bd712"). InnerVolumeSpecName "kube-api-access-glbgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:32:40.630082 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.630066    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5457c054-c918-4a4e-becc-aaa0892bd712" (UID: "5457c054-c918-4a4e-becc-aaa0892bd712"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:32:40.630216 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.630202    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5457c054-c918-4a4e-becc-aaa0892bd712-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5457c054-c918-4a4e-becc-aaa0892bd712" (UID: "5457c054-c918-4a4e-becc-aaa0892bd712"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 20:32:40.729070 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.729048    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5457c054-c918-4a4e-becc-aaa0892bd712-istiod-ca-cert\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:32:40.729180 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.729073    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-istio-envoy\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:32:40.729180 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.729086    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-istio-data\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:32:40.729180 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.729097    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5457c054-c918-4a4e-becc-aaa0892bd712-istio-token\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:32:40.729180 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.729109    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5457c054-c918-4a4e-becc-aaa0892bd712-istio-podinfo\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:32:40.729180 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.729120    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-workload-socket\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:32:40.729180 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.729133    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-workload-certs\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:32:40.729180 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.729147    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5457c054-c918-4a4e-becc-aaa0892bd712-credential-socket\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:32:40.729180 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:40.729159    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glbgk\" (UniqueName: \"kubernetes.io/projected/5457c054-c918-4a4e-becc-aaa0892bd712-kube-api-access-glbgk\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:32:41.376272 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:41.376239    2579 generic.go:356] "Generic (PLEG): container finished" podID="5457c054-c918-4a4e-becc-aaa0892bd712" containerID="60f5e1a4c2d85b4b33922d2cda60b7ff21a94ef15114ad6254e4ad34aa9c1991" exitCode=0
Jul 27 20:32:41.376445 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:41.376290    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"
Jul 27 20:32:41.376445 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:41.376317    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l" event={"ID":"5457c054-c918-4a4e-becc-aaa0892bd712","Type":"ContainerDied","Data":"60f5e1a4c2d85b4b33922d2cda60b7ff21a94ef15114ad6254e4ad34aa9c1991"}
Jul 27 20:32:41.376445 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:41.376383    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l" event={"ID":"5457c054-c918-4a4e-becc-aaa0892bd712","Type":"ContainerDied","Data":"9225d3dcf9f370a4b11178b15a119ca0ff589545438dcd7f22d05feff13fe42e"}
Jul 27 20:32:41.376445 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:41.376417    2579 scope.go:117] "RemoveContainer" containerID="60f5e1a4c2d85b4b33922d2cda60b7ff21a94ef15114ad6254e4ad34aa9c1991"
Jul 27 20:32:41.386457 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:41.386440    2579 scope.go:117] "RemoveContainer" containerID="c9903bed1bbdfb110529ce0280c8604fb52d5c9240bb13674fd720b049a281bb"
Jul 27 20:32:41.397791 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:41.397760    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"]
Jul 27 20:32:41.397791 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:41.397771    2579 scope.go:117] "RemoveContainer" containerID="60f5e1a4c2d85b4b33922d2cda60b7ff21a94ef15114ad6254e4ad34aa9c1991"
Jul 27 20:32:41.398133 ip-10-0-140-36 kubenswrapper[2579]: E0727 20:32:41.398106    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60f5e1a4c2d85b4b33922d2cda60b7ff21a94ef15114ad6254e4ad34aa9c1991\": container with ID starting with 60f5e1a4c2d85b4b33922d2cda60b7ff21a94ef15114ad6254e4ad34aa9c1991 not found: ID does not exist" containerID="60f5e1a4c2d85b4b33922d2cda60b7ff21a94ef15114ad6254e4ad34aa9c1991"
Jul 27 20:32:41.398201 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:41.398143    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60f5e1a4c2d85b4b33922d2cda60b7ff21a94ef15114ad6254e4ad34aa9c1991"} err="failed to get container status \"60f5e1a4c2d85b4b33922d2cda60b7ff21a94ef15114ad6254e4ad34aa9c1991\": rpc error: code = NotFound desc = could not find container \"60f5e1a4c2d85b4b33922d2cda60b7ff21a94ef15114ad6254e4ad34aa9c1991\": container with ID starting with 60f5e1a4c2d85b4b33922d2cda60b7ff21a94ef15114ad6254e4ad34aa9c1991 not found: ID does not exist"
Jul 27 20:32:41.398201 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:41.398168    2579 scope.go:117] "RemoveContainer" containerID="c9903bed1bbdfb110529ce0280c8604fb52d5c9240bb13674fd720b049a281bb"
Jul 27 20:32:41.398475 ip-10-0-140-36 kubenswrapper[2579]: E0727 20:32:41.398457    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9903bed1bbdfb110529ce0280c8604fb52d5c9240bb13674fd720b049a281bb\": container with ID starting with c9903bed1bbdfb110529ce0280c8604fb52d5c9240bb13674fd720b049a281bb not found: ID does not exist" containerID="c9903bed1bbdfb110529ce0280c8604fb52d5c9240bb13674fd720b049a281bb"
Jul 27 20:32:41.398543 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:41.398483    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9903bed1bbdfb110529ce0280c8604fb52d5c9240bb13674fd720b049a281bb"} err="failed to get container status \"c9903bed1bbdfb110529ce0280c8604fb52d5c9240bb13674fd720b049a281bb\": rpc error: code = NotFound desc = could not find container \"c9903bed1bbdfb110529ce0280c8604fb52d5c9240bb13674fd720b049a281bb\": container with ID starting with c9903bed1bbdfb110529ce0280c8604fb52d5c9240bb13674fd720b049a281bb not found: ID does not exist"
Jul 27 20:32:41.399436 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:41.399414    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-1517d6-openshift-default-5d56dfdff6-8bf7l"]
Jul 27 20:32:42.261528 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:42.261494    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5457c054-c918-4a4e-becc-aaa0892bd712" path="/var/lib/kubelet/pods/5457c054-c918-4a4e-becc-aaa0892bd712/volumes"
Jul 27 20:32:57.821541 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.821504    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f44149-kvtjr"]
Jul 27 20:32:57.821919 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.821818    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45882e75-3204-42ad-bef9-b456078ae525" containerName="cleanup"
Jul 27 20:32:57.821919 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.821829    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="45882e75-3204-42ad-bef9-b456078ae525" containerName="cleanup"
Jul 27 20:32:57.821919 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.821842    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fea2b786-1217-41f6-9751-bd0aae48bbad" containerName="main"
Jul 27 20:32:57.821919 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.821847    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fea2b786-1217-41f6-9751-bd0aae48bbad" containerName="main"
Jul 27 20:32:57.821919 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.821867    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c478127-9459-4aef-afc4-755f2cc02481" containerName="istio-proxy"
Jul 27 20:32:57.821919 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.821873    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c478127-9459-4aef-afc4-755f2cc02481" containerName="istio-proxy"
Jul 27 20:32:57.821919 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.821883    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5457c054-c918-4a4e-becc-aaa0892bd712" containerName="istio-proxy"
Jul 27 20:32:57.821919 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.821888    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5457c054-c918-4a4e-becc-aaa0892bd712" containerName="istio-proxy"
Jul 27 20:32:57.821919 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.821898    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5457c054-c918-4a4e-becc-aaa0892bd712" containerName="istio-proxy"
Jul 27 20:32:57.821919 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.821904    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5457c054-c918-4a4e-becc-aaa0892bd712" containerName="istio-proxy"
Jul 27 20:32:57.821919 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.821912    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c478127-9459-4aef-afc4-755f2cc02481" containerName="istio-proxy"
Jul 27 20:32:57.821919 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.821917    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c478127-9459-4aef-afc4-755f2cc02481" containerName="istio-proxy"
Jul 27 20:32:57.822273 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.821964    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5457c054-c918-4a4e-becc-aaa0892bd712" containerName="istio-proxy"
Jul 27 20:32:57.822273 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.821973    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c478127-9459-4aef-afc4-755f2cc02481" containerName="istio-proxy"
Jul 27 20:32:57.822273 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.821980    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c478127-9459-4aef-afc4-755f2cc02481" containerName="istio-proxy"
Jul 27 20:32:57.822273 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.821985    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="45882e75-3204-42ad-bef9-b456078ae525" containerName="cleanup"
Jul 27 20:32:57.822273 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.821991    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fea2b786-1217-41f6-9751-bd0aae48bbad" containerName="main"
Jul 27 20:32:57.824920 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.824904    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f44149-kvtjr"
Jul 27 20:32:57.826725 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.826703    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 20:32:57.826813 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.826737    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x6cbr\""
Jul 27 20:32:57.831903 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.831880    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f44149-kvtjr"]
Jul 27 20:32:57.953943 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:57.953916    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5734607-e7b1-4e60-8b79-4da5f792f5f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-f44149-kvtjr\" (UID: \"b5734607-e7b1-4e60-8b79-4da5f792f5f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f44149-kvtjr"
Jul 27 20:32:58.055262 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:58.055236    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5734607-e7b1-4e60-8b79-4da5f792f5f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-f44149-kvtjr\" (UID: \"b5734607-e7b1-4e60-8b79-4da5f792f5f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f44149-kvtjr"
Jul 27 20:32:58.055892 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:58.055850    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5734607-e7b1-4e60-8b79-4da5f792f5f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-f44149-kvtjr\" (UID: \"b5734607-e7b1-4e60-8b79-4da5f792f5f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f44149-kvtjr"
Jul 27 20:32:58.136768 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:58.136713    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f44149-kvtjr"
Jul 27 20:32:58.266759 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:58.266738    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f44149-kvtjr"]
Jul 27 20:32:58.269356 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:32:58.269316    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5734607_e7b1_4e60_8b79_4da5f792f5f7.slice/crio-22f9db63d87935a0d36cca8d4468a45e912119084ba8e8949fe16878b78fc3c7 WatchSource:0}: Error finding container 22f9db63d87935a0d36cca8d4468a45e912119084ba8e8949fe16878b78fc3c7: Status 404 returned error can't find the container with id 22f9db63d87935a0d36cca8d4468a45e912119084ba8e8949fe16878b78fc3c7
Jul 27 20:32:58.459360 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:58.459334    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f44149-kvtjr" event={"ID":"b5734607-e7b1-4e60-8b79-4da5f792f5f7","Type":"ContainerStarted","Data":"22f9db63d87935a0d36cca8d4468a45e912119084ba8e8949fe16878b78fc3c7"}
Jul 27 20:32:59.465771 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:59.465739    2579 generic.go:356] "Generic (PLEG): container finished" podID="b5734607-e7b1-4e60-8b79-4da5f792f5f7" containerID="4dcc75e6a319c2a38bca753a8fa0f0a762215b0c3bbbf11416d8e0e99eea1329" exitCode=0
Jul 27 20:32:59.466146 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:32:59.465833    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f44149-kvtjr" event={"ID":"b5734607-e7b1-4e60-8b79-4da5f792f5f7","Type":"ContainerDied","Data":"4dcc75e6a319c2a38bca753a8fa0f0a762215b0c3bbbf11416d8e0e99eea1329"}
Jul 27 20:33:00.596718 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:00.596688    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f44149-kvtjr"
Jul 27 20:33:00.677224 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:00.677197    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5734607-e7b1-4e60-8b79-4da5f792f5f7-maas-api-service-ca\") pod \"b5734607-e7b1-4e60-8b79-4da5f792f5f7\" (UID: \"b5734607-e7b1-4e60-8b79-4da5f792f5f7\") "
Jul 27 20:33:00.677514 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:00.677492    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5734607-e7b1-4e60-8b79-4da5f792f5f7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b5734607-e7b1-4e60-8b79-4da5f792f5f7" (UID: "b5734607-e7b1-4e60-8b79-4da5f792f5f7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:33:00.778403 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:00.778335    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5734607-e7b1-4e60-8b79-4da5f792f5f7-maas-api-service-ca\") on node \"ip-10-0-140-36.ec2.internal\" DevicePath \"\""
Jul 27 20:33:01.475777 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:01.475739    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f44149-kvtjr" event={"ID":"b5734607-e7b1-4e60-8b79-4da5f792f5f7","Type":"ContainerDied","Data":"22f9db63d87935a0d36cca8d4468a45e912119084ba8e8949fe16878b78fc3c7"}
Jul 27 20:33:01.475969 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:01.475797    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22f9db63d87935a0d36cca8d4468a45e912119084ba8e8949fe16878b78fc3c7"
Jul 27 20:33:01.475969 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:01.475753    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f44149-kvtjr"
Jul 27 20:33:12.976502 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:12.976460    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f44149-kvtjr"]
Jul 27 20:33:12.978426 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:12.978403    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f44149-kvtjr"]
Jul 27 20:33:14.261534 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:14.261502    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5734607-e7b1-4e60-8b79-4da5f792f5f7" path="/var/lib/kubelet/pods/b5734607-e7b1-4e60-8b79-4da5f792f5f7/volumes"
Jul 27 20:33:20.146446 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:20.146409    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-75c9bf7749-5m9gs_7d81d979-b40c-494a-b262-cb16d12581ee/authorino/0.log"
Jul 27 20:33:26.111491 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:26.111402    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7df5588f67-9f6vv_482626a0-f93e-4282-9c33-ee3ea818e5d1/manager/0.log"
Jul 27 20:33:26.117679 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:26.117660    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7df5588f67-9f6vv_482626a0-f93e-4282-9c33-ee3ea818e5d1/copy-manifests/0.log"
Jul 27 20:33:26.923782 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:26.923747    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29753070-cxfjr_45882e75-3204-42ad-bef9-b456078ae525/cleanup/0.log"
Jul 27 20:33:28.265833 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:28.265801    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-75c9bf7749-5m9gs_7d81d979-b40c-494a-b262-cb16d12581ee/authorino/0.log"
Jul 27 20:33:29.614527 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:29.614495    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fd5d97dbc-7xtj6_c6f11af4-1ae4-472e-9383-774dbb37ab62/kube-auth-proxy/0.log"
Jul 27 20:33:30.185473 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:30.185439    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5f5b9f5469-b26r4_c6c35228-a7c8-44a3-9b48-63c27bb72b25/router/0.log"
Jul 27 20:33:30.610444 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:30.610357    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-9f2rd_95f0ed57-2cfd-4942-b707-76b26883ea57/main/0.log"
Jul 27 20:33:30.825034 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:30.825004    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8zgnx_f6b167f4-4de4-4c51-9415-d69b7129eb40/main/0.log"
Jul 27 20:33:31.041891 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:31.041840    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-zcbr5_b6dbec73-ac0a-4f69-9112-36f9b46658d0/main/0.log"
Jul 27 20:33:39.258371 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:39.258338    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7gt55_6407e54e-d37a-4344-91d3-4494c81246c4/global-pull-secret-syncer/0.log"
Jul 27 20:33:39.406572 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:39.406537    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tktlt_967ad4b2-9b0f-4654-be1b-8bcceee7f7c3/konnectivity-agent/0.log"
Jul 27 20:33:39.464874 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:39.464831    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-36.ec2.internal_8f425195a1bad05a939cf1834470c7d1/haproxy/0.log"
Jul 27 20:33:44.548384 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:44.548349    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-75c9bf7749-5m9gs_7d81d979-b40c-494a-b262-cb16d12581ee/authorino/0.log"
Jul 27 20:33:47.024132 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:47.024097    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-kx5wp_0a45576e-6f99-4ff2-bdbe-934a603e6756/cluster-monitoring-operator/0.log"
Jul 27 20:33:47.198920 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:47.198888    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j4lvl_74bf592e-5185-4b72-92f7-a516401d267f/node-exporter/0.log"
Jul 27 20:33:47.220894 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:47.220849    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j4lvl_74bf592e-5185-4b72-92f7-a516401d267f/kube-rbac-proxy/0.log"
Jul 27 20:33:47.249063 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:47.249042    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j4lvl_74bf592e-5185-4b72-92f7-a516401d267f/init-textfile/0.log"
Jul 27 20:33:49.281701 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.281668    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"]
Jul 27 20:33:49.282185 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.282122    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5734607-e7b1-4e60-8b79-4da5f792f5f7" containerName="revoke-keys"
Jul 27 20:33:49.282185 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.282140    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5734607-e7b1-4e60-8b79-4da5f792f5f7" containerName="revoke-keys"
Jul 27 20:33:49.282307 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.282242    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5457c054-c918-4a4e-becc-aaa0892bd712" containerName="istio-proxy"
Jul 27 20:33:49.282307 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.282254    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5734607-e7b1-4e60-8b79-4da5f792f5f7" containerName="revoke-keys"
Jul 27 20:33:49.285243 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.285223    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"
Jul 27 20:33:49.287219 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.287200    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sgjdf\"/\"default-dockercfg-spjbp\""
Jul 27 20:33:49.287315 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.287198    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sgjdf\"/\"openshift-service-ca.crt\""
Jul 27 20:33:49.287841 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.287826    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sgjdf\"/\"kube-root-ca.crt\""
Jul 27 20:33:49.292716 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.292699    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"]
Jul 27 20:33:49.368505 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.368477    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6cded1f4-cf31-4059-84c0-a2267ca5987c-sys\") pod \"perf-node-gather-daemonset-w6btp\" (UID: \"6cded1f4-cf31-4059-84c0-a2267ca5987c\") " pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"
Jul 27 20:33:49.368626 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.368515    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6cded1f4-cf31-4059-84c0-a2267ca5987c-proc\") pod \"perf-node-gather-daemonset-w6btp\" (UID: \"6cded1f4-cf31-4059-84c0-a2267ca5987c\") " pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"
Jul 27 20:33:49.368626 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.368554    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6cded1f4-cf31-4059-84c0-a2267ca5987c-podres\") pod \"perf-node-gather-daemonset-w6btp\" (UID: \"6cded1f4-cf31-4059-84c0-a2267ca5987c\") " pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"
Jul 27 20:33:49.368626 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.368592    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6cded1f4-cf31-4059-84c0-a2267ca5987c-lib-modules\") pod \"perf-node-gather-daemonset-w6btp\" (UID: \"6cded1f4-cf31-4059-84c0-a2267ca5987c\") " pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"
Jul 27 20:33:49.368746 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.368659    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2lc9\" (UniqueName: \"kubernetes.io/projected/6cded1f4-cf31-4059-84c0-a2267ca5987c-kube-api-access-z2lc9\") pod \"perf-node-gather-daemonset-w6btp\" (UID: \"6cded1f4-cf31-4059-84c0-a2267ca5987c\") " pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"
Jul 27 20:33:49.469934 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.469895    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6cded1f4-cf31-4059-84c0-a2267ca5987c-podres\") pod \"perf-node-gather-daemonset-w6btp\" (UID: \"6cded1f4-cf31-4059-84c0-a2267ca5987c\") " pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"
Jul 27 20:33:49.470094 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.469946    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6cded1f4-cf31-4059-84c0-a2267ca5987c-lib-modules\") pod \"perf-node-gather-daemonset-w6btp\" (UID: \"6cded1f4-cf31-4059-84c0-a2267ca5987c\") " pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"
Jul 27 20:33:49.470094 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.469999    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2lc9\" (UniqueName: \"kubernetes.io/projected/6cded1f4-cf31-4059-84c0-a2267ca5987c-kube-api-access-z2lc9\") pod \"perf-node-gather-daemonset-w6btp\" (UID: \"6cded1f4-cf31-4059-84c0-a2267ca5987c\") " pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"
Jul 27 20:33:49.470094 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.470025    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6cded1f4-cf31-4059-84c0-a2267ca5987c-podres\") pod \"perf-node-gather-daemonset-w6btp\" (UID: \"6cded1f4-cf31-4059-84c0-a2267ca5987c\") " pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"
Jul 27 20:33:49.470094 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.470078    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6cded1f4-cf31-4059-84c0-a2267ca5987c-sys\") pod \"perf-node-gather-daemonset-w6btp\" (UID: \"6cded1f4-cf31-4059-84c0-a2267ca5987c\") " pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"
Jul 27 20:33:49.470306 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.470108    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6cded1f4-cf31-4059-84c0-a2267ca5987c-proc\") pod \"perf-node-gather-daemonset-w6btp\" (UID: \"6cded1f4-cf31-4059-84c0-a2267ca5987c\") " pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"
Jul 27 20:33:49.470306 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.470176    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6cded1f4-cf31-4059-84c0-a2267ca5987c-proc\") pod \"perf-node-gather-daemonset-w6btp\" (UID: \"6cded1f4-cf31-4059-84c0-a2267ca5987c\") " pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"
Jul 27 20:33:49.470306 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.470180    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6cded1f4-cf31-4059-84c0-a2267ca5987c-sys\") pod \"perf-node-gather-daemonset-w6btp\" (UID: \"6cded1f4-cf31-4059-84c0-a2267ca5987c\") " pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"
Jul 27 20:33:49.470456 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.470428    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6cded1f4-cf31-4059-84c0-a2267ca5987c-lib-modules\") pod \"perf-node-gather-daemonset-w6btp\" (UID: \"6cded1f4-cf31-4059-84c0-a2267ca5987c\") " pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"
Jul 27 20:33:49.478010 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.477985    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2lc9\" (UniqueName: \"kubernetes.io/projected/6cded1f4-cf31-4059-84c0-a2267ca5987c-kube-api-access-z2lc9\") pod \"perf-node-gather-daemonset-w6btp\" (UID: \"6cded1f4-cf31-4059-84c0-a2267ca5987c\") " pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"
Jul 27 20:33:49.596155 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.596062    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"
Jul 27 20:33:49.740258 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.740220    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"]
Jul 27 20:33:49.743885 ip-10-0-140-36 kubenswrapper[2579]: W0727 20:33:49.743841    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6cded1f4_cf31_4059_84c0_a2267ca5987c.slice/crio-f26597194a0868ab6c6d888b82bdbb801d77f5a7a54668e10fd5884b6aa2bb5a WatchSource:0}: Error finding container f26597194a0868ab6c6d888b82bdbb801d77f5a7a54668e10fd5884b6aa2bb5a: Status 404 returned error can't find the container with id f26597194a0868ab6c6d888b82bdbb801d77f5a7a54668e10fd5884b6aa2bb5a
Jul 27 20:33:49.839748 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.839723    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/1.log"
Jul 27 20:33:49.849368 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:49.849304    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xfztm_e814129b-3b96-4c75-ba5b-480e39d6d71e/console-operator/2.log"
Jul 27 20:33:50.693336 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:50.693302    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp" event={"ID":"6cded1f4-cf31-4059-84c0-a2267ca5987c","Type":"ContainerStarted","Data":"3d737a3d3c0c2571c86e7f2e15bd1e3dbe995f299fe0b589114ea82aadc4f0fe"}
Jul 27 20:33:50.693731 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:50.693352    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp" event={"ID":"6cded1f4-cf31-4059-84c0-a2267ca5987c","Type":"ContainerStarted","Data":"f26597194a0868ab6c6d888b82bdbb801d77f5a7a54668e10fd5884b6aa2bb5a"}
Jul 27 20:33:50.693731 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:50.693390    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"
Jul 27 20:33:50.709400 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:50.709351    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp" podStartSLOduration=1.709335572 podStartE2EDuration="1.709335572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:33:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:33:50.70866856 +0000 UTC m=+3258.937910754" watchObservedRunningTime="2026-07-27 20:33:50.709335572 +0000 UTC m=+3258.938577769"
Jul 27 20:33:51.018830 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:51.018751    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-hx29v_bd0782d2-c168-41d3-bc16-bf48f1453f20/volume-data-source-validator/0.log"
Jul 27 20:33:52.119969 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:52.119928    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mcmp4_66b96198-308c-49c0-bb97-bb8210624656/dns/0.log"
Jul 27 20:33:52.139387 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:52.139361    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mcmp4_66b96198-308c-49c0-bb97-bb8210624656/kube-rbac-proxy/0.log"
Jul 27 20:33:52.206168 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:52.206141    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-grpbk_73b8094b-b342-4acd-9323-a5e71e0c3dfb/dns-node-resolver/0.log"
Jul 27 20:33:52.764398 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:52.764371    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m2pzk_91f1914a-7ad4-45a2-949f-75375e34df1f/node-ca/0.log"
Jul 27 20:33:53.994310 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:53.994279    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fd5d97dbc-7xtj6_c6f11af4-1ae4-472e-9383-774dbb37ab62/kube-auth-proxy/0.log"
Jul 27 20:33:54.177315 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:54.177284    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5f5b9f5469-b26r4_c6c35228-a7c8-44a3-9b48-63c27bb72b25/router/0.log"
Jul 27 20:33:55.005082 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:55.005045    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tct2c_225f0ebd-cdb1-44c5-b705-19e759e4dd61/serve-healthcheck-canary/0.log"
Jul 27 20:33:55.475488 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:55.475452    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-n9pwk_096c978c-0cb6-4ff6-8fba-93b2113f24a4/insights-operator/0.log"
Jul 27 20:33:55.476448 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:55.476428    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-n9pwk_096c978c-0cb6-4ff6-8fba-93b2113f24a4/insights-operator/1.log"
Jul 27 20:33:55.559004 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:55.558975    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lppjn_722f2327-d04b-4470-aea1-be7c383a497a/kube-rbac-proxy/0.log"
Jul 27 20:33:55.577751 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:55.577715    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lppjn_722f2327-d04b-4470-aea1-be7c383a497a/exporter/0.log"
Jul 27 20:33:55.597933 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:55.597905    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lppjn_722f2327-d04b-4470-aea1-be7c383a497a/extractor/0.log"
Jul 27 20:33:56.708535 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:56.708511    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sgjdf/perf-node-gather-daemonset-w6btp"
Jul 27 20:33:58.699974 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:58.699910    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7df5588f67-9f6vv_482626a0-f93e-4282-9c33-ee3ea818e5d1/manager/0.log"
Jul 27 20:33:58.715576 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:33:58.715551    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7df5588f67-9f6vv_482626a0-f93e-4282-9c33-ee3ea818e5d1/copy-manifests/0.log"
Jul 27 20:34:00.058014 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:34:00.057987    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8689f9464-g8plw_e5c5f39c-feaa-4254-b244-5defedc33034/manager/0.log"
Jul 27 20:34:00.366602 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:34:00.366515    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fd5d97dbc-7xtj6_c6f11af4-1ae4-472e-9383-774dbb37ab62/kube-auth-proxy/0.log"
Jul 27 20:34:00.505102 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:34:00.505069    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5f5b9f5469-b26r4_c6c35228-a7c8-44a3-9b48-63c27bb72b25/router/0.log"
Jul 27 20:34:02.181036 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:34:02.181010    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fd5d97dbc-7xtj6_c6f11af4-1ae4-472e-9383-774dbb37ab62/kube-auth-proxy/0.log"
Jul 27 20:34:02.330230 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:34:02.330193    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5f5b9f5469-b26r4_c6c35228-a7c8-44a3-9b48-63c27bb72b25/router/0.log"
Jul 27 20:34:05.310129 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:34:05.310026    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl_b1f4f700-7ce1-4669-a383-793dfbcf4f2e/kube-storage-version-migrator-operator/1.log"
Jul 27 20:34:05.311645 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:34:05.311621    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-rrvsl_b1f4f700-7ce1-4669-a383-793dfbcf4f2e/kube-storage-version-migrator-operator/0.log"
Jul 27 20:34:06.462940 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:34:06.462911    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lwcg2_a76e8a73-6dac-43e0-92ec-6269ccd13593/kube-multus-additional-cni-plugins/0.log"
Jul 27 20:34:06.488200 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:34:06.488177    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lwcg2_a76e8a73-6dac-43e0-92ec-6269ccd13593/egress-router-binary-copy/0.log"
Jul 27 20:34:06.511273 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:34:06.511249    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lwcg2_a76e8a73-6dac-43e0-92ec-6269ccd13593/cni-plugins/0.log"
Jul 27 20:34:06.533232 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:34:06.533206    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lwcg2_a76e8a73-6dac-43e0-92ec-6269ccd13593/bond-cni-plugin/0.log"
Jul 27 20:34:06.554359 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:34:06.554337    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lwcg2_a76e8a73-6dac-43e0-92ec-6269ccd13593/routeoverride-cni/0.log"
Jul 27 20:34:06.575632 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:34:06.575612    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lwcg2_a76e8a73-6dac-43e0-92ec-6269ccd13593/whereabouts-cni-bincopy/0.log"
Jul 27 20:34:06.597947 ip-10-0-140-36 kubenswrapper[2579]: I0727 20:34:06.597926    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lwcg2_a76e8a73-6dac-43e0-92ec-6269ccd13593/whereabouts-cni/0.log"