Jul 27 18:29:23.951007 ip-10-0-132-186 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 27 18:29:23.951039 ip-10-0-132-186 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 27 18:29:23.951048 ip-10-0-132-186 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 27 18:29:23.951329 ip-10-0-132-186 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 27 18:29:34.129452 ip-10-0-132-186 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 27 18:29:34.129476 ip-10-0-132-186 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 22025bb590284fa78c2bb2b3bc423334 --
Jul 27 18:31:31.350158 ip-10-0-132-186 systemd[1]: Starting Kubernetes Kubelet...
Jul 27 18:31:31.672668 ip-10-0-132-186 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 18:31:31.672668 ip-10-0-132-186 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 27 18:31:31.672668 ip-10-0-132-186 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 18:31:31.672668 ip-10-0-132-186 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 27 18:31:31.672668 ip-10-0-132-186 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 18:31:31.674115 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.674024    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 27 18:31:31.679632 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679616    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 18:31:31.679632 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679631    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 18:31:31.679632 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679635    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 18:31:31.679724 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679638    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 18:31:31.679724 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679641    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 18:31:31.679724 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679646    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 18:31:31.679724 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679650    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 18:31:31.679724 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679653    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 18:31:31.679724 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679656    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 18:31:31.679724 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679659    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 18:31:31.679724 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679662    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 18:31:31.679724 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679665    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 18:31:31.679724 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679668    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 18:31:31.679724 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679671    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 18:31:31.679724 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679673    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 18:31:31.679724 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679676    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 18:31:31.679724 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679678    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 18:31:31.679724 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679681    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 18:31:31.679724 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679683    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 18:31:31.679724 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679686    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 18:31:31.679724 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679689    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 18:31:31.679724 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679691    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 18:31:31.679724 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679693    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 18:31:31.680218 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679696    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 18:31:31.680218 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679698    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 18:31:31.680218 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679701    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 18:31:31.680218 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679703    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 18:31:31.680218 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679706    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 18:31:31.680218 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679708    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 18:31:31.680218 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679711    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 18:31:31.680218 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679714    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 18:31:31.680218 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679716    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 18:31:31.680218 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679719    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 18:31:31.680218 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679721    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 18:31:31.680218 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679725    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 18:31:31.680218 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679729    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 18:31:31.680218 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679732    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 18:31:31.680218 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679736    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 18:31:31.680218 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679739    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 18:31:31.680218 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679742    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 18:31:31.680218 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679745    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 18:31:31.680218 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679747    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 18:31:31.680671 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679750    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 18:31:31.680671 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679752    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 18:31:31.680671 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679755    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 18:31:31.680671 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679757    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 18:31:31.680671 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679760    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 18:31:31.680671 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679762    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 18:31:31.680671 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679765    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 18:31:31.680671 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679767    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 18:31:31.680671 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679769    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 18:31:31.680671 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679772    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 18:31:31.680671 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679774    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 18:31:31.680671 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679776    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 18:31:31.680671 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679779    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 18:31:31.680671 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679783    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 18:31:31.680671 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679786    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 18:31:31.680671 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679789    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 18:31:31.680671 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679791    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 18:31:31.680671 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679794    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 18:31:31.680671 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679796    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 18:31:31.681190 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679799    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 18:31:31.681190 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679801    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 18:31:31.681190 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679804    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 18:31:31.681190 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679806    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 18:31:31.681190 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679809    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 18:31:31.681190 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679812    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 18:31:31.681190 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679814    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 18:31:31.681190 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679816    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 18:31:31.681190 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679818    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 18:31:31.681190 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679821    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 18:31:31.681190 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679823    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 18:31:31.681190 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679826    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 18:31:31.681190 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679828    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 18:31:31.681190 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679831    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 18:31:31.681190 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679833    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 18:31:31.681190 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679836    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 18:31:31.681190 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679838    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 18:31:31.681190 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679841    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 18:31:31.681190 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679843    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 18:31:31.681857 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679846    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 18:31:31.681857 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679848    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 18:31:31.681857 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679850    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 18:31:31.681857 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679853    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 18:31:31.681857 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679856    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 18:31:31.681857 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679858    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 18:31:31.681857 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679861    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 18:31:31.681857 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679863    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 18:31:31.681857 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679880    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 18:31:31.681857 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679883    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 18:31:31.681857 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679886    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 18:31:31.681857 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679888    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 18:31:31.681857 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679891    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 18:31:31.681857 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679895    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 18:31:31.681857 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679898    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 18:31:31.681857 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679901    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 18:31:31.681857 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679904    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 18:31:31.681857 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679906    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 18:31:31.681857 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.679909    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 18:31:31.681857 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680633    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 18:31:31.682621 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680640    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 18:31:31.682621 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680644    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 18:31:31.682621 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680647    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 18:31:31.682621 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680649    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 18:31:31.682621 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680652    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 18:31:31.682621 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680654    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 18:31:31.682621 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680658    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 18:31:31.682621 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680661    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 18:31:31.682621 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680664    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 18:31:31.682621 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680666    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 18:31:31.682621 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680669    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 18:31:31.682621 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680673    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 18:31:31.682621 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680676    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 18:31:31.682621 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680678    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 18:31:31.682621 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680681    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 18:31:31.682621 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680685    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 18:31:31.682621 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680689    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 18:31:31.682621 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680692    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 18:31:31.682621 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680694    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 18:31:31.682621 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680697    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 18:31:31.683298 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680700    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 18:31:31.683298 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680703    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 18:31:31.683298 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680706    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 18:31:31.683298 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680708    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 18:31:31.683298 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680711    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 18:31:31.683298 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680714    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 18:31:31.683298 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680716    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 18:31:31.683298 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680720    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 18:31:31.683298 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680722    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 18:31:31.683298 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680725    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 18:31:31.683298 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680727    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 18:31:31.683298 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680730    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 18:31:31.683298 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680732    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 18:31:31.683298 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680734    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 18:31:31.683298 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680737    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 18:31:31.683298 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680739    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 18:31:31.683298 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680743    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 18:31:31.683298 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680747    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 18:31:31.683298 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680750    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 18:31:31.683783 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680752    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 18:31:31.683783 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680755    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 18:31:31.683783 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680758    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 18:31:31.683783 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680760    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 18:31:31.683783 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680763    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 18:31:31.683783 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680766    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 18:31:31.683783 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680768    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 18:31:31.683783 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680771    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 18:31:31.683783 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680773    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 18:31:31.683783 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680776    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 18:31:31.683783 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680778    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 18:31:31.683783 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680780    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 18:31:31.683783 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680783    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 18:31:31.683783 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680786    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 18:31:31.683783 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680788    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 18:31:31.683783 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680791    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 18:31:31.683783 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680793    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 18:31:31.683783 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680796    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 18:31:31.683783 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680799    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 18:31:31.683783 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680801    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 18:31:31.684292 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680804    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 18:31:31.684292 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680806    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 18:31:31.684292 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680809    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 18:31:31.684292 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680811    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 18:31:31.684292 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680814    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 18:31:31.684292 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680816    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 18:31:31.684292 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680819    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 18:31:31.684292 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680821    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 18:31:31.684292 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680824    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 18:31:31.684292 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680827    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 18:31:31.684292 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680830    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 18:31:31.684292 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680832    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 18:31:31.684292 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680834    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 18:31:31.684292 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680837    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 18:31:31.684292 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680839    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 18:31:31.684292 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680842    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 18:31:31.684292 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680844    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 18:31:31.684292 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680846    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 18:31:31.684292 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680850    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 18:31:31.684292 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680852    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 18:31:31.684790 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680855    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 18:31:31.684790 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680857    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 18:31:31.684790 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680860    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 18:31:31.684790 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680862    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 18:31:31.684790 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680881    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 18:31:31.684790 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680885    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 18:31:31.684790 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680887    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 18:31:31.684790 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680890    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 18:31:31.684790 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680893    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 18:31:31.684790 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680895    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 18:31:31.684790 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680898    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 18:31:31.684790 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680900    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 18:31:31.684790 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680903    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 18:31:31.684790 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680906    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 18:31:31.684790 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680909    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 18:31:31.684790 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680912    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 18:31:31.684790 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680916    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 18:31:31.684790 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680919    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 18:31:31.684790 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.680922    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 18:31:31.684790 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682914    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 18:31:31.685305 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682926    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 18:31:31.685305 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682931    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 18:31:31.685305 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682934    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 18:31:31.685305 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682937    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 18:31:31.685305 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682940    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 18:31:31.685305 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682943    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 18:31:31.685305 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682946    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 18:31:31.685305 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682948    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 18:31:31.685305 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682951    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 18:31:31.685305 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682954    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 18:31:31.685305 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682957    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 18:31:31.685305 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682960    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 18:31:31.685305 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682963    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 18:31:31.685305 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682965    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 18:31:31.685305 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682968    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 18:31:31.685305 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682970    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 18:31:31.685305 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682973    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 18:31:31.685305 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682976    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 18:31:31.685305 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682978    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 18:31:31.685305 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682980    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 18:31:31.685777 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682983    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 18:31:31.685777 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682985    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 18:31:31.685777 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.682988    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 18:31:31.685777 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683004    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 18:31:31.685777 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683008    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 18:31:31.685777 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683010    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 18:31:31.685777 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683013    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 18:31:31.685777 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683015    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 18:31:31.685777 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683018    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 18:31:31.685777 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683021    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 18:31:31.685777 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683023    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 18:31:31.685777 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683026    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 18:31:31.685777 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683029    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 18:31:31.685777 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683032    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 18:31:31.685777 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683034    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 18:31:31.685777 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683037    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 18:31:31.685777 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683040    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 18:31:31.685777 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683043    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 18:31:31.685777 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683046    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 18:31:31.685777 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683050    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 18:31:31.686280 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683053    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 18:31:31.686280 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683057    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 18:31:31.686280 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683067    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 18:31:31.686280 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683070    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 18:31:31.686280 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683072    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 18:31:31.686280 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683075    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 18:31:31.686280 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683077    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 18:31:31.686280 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683080    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 18:31:31.686280 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683083    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 18:31:31.686280 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683088    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 18:31:31.686280 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683090    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 18:31:31.686280 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683093    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 18:31:31.686280 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683095    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 18:31:31.686280 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683098    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 18:31:31.686280 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683100    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 18:31:31.686280 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683103    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 18:31:31.686280 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683105    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 18:31:31.686280 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683108    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 18:31:31.686280 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683110    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 18:31:31.686732 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683113    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 18:31:31.686732 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683115    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 18:31:31.686732 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683117    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 18:31:31.686732 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683120    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 18:31:31.686732 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683123    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 18:31:31.686732 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683126    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 18:31:31.686732 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683129    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 18:31:31.686732 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683133    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 18:31:31.686732 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683137    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 18:31:31.686732 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683141    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 18:31:31.686732 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683144    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 18:31:31.686732 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683147    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 18:31:31.686732 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683150    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 18:31:31.686732 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683153    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 18:31:31.686732 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683156    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 18:31:31.686732 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683159    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 18:31:31.686732 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683162    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 18:31:31.686732 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683164    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 18:31:31.686732 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683168    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 18:31:31.687258 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683181    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 18:31:31.687258 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683184    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 18:31:31.687258 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683187    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 18:31:31.687258 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683190    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 18:31:31.687258 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683194    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 18:31:31.687258 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683196    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 18:31:31.687258 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683199    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 18:31:31.687258 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683202    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 18:31:31.687258 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683204    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 18:31:31.687258 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683207    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 18:31:31.687258 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683209    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 18:31:31.687258 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683212    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 18:31:31.687258 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683214    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 18:31:31.687258 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683217    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 18:31:31.687258 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683219    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 18:31:31.687258 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683222    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 18:31:31.687258 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683224    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 18:31:31.687258 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683227    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 18:31:31.687258 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683231    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 18:31:31.687258 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683233    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683302    2562 flags.go:64] FLAG: --address="0.0.0.0"
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683310    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683318    2562 flags.go:64] FLAG: --anonymous-auth="true"
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683324    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683331    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683336    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683343    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683349    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683352    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683355    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683358    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683362    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683365    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683368    2562 flags.go:64] FLAG: --cgroup-root=""
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683371    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683374    2562 flags.go:64] FLAG: --client-ca-file=""
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683377    2562 flags.go:64] FLAG: --cloud-provider="external"
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683380    2562 flags.go:64] FLAG: --cluster-dns="[]"
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683385    2562 flags.go:64] FLAG: --cluster-domain=""
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683388    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683391    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683394    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683397    2562 flags.go:64] FLAG: --container-log-max-files="5"
Jul 27 18:31:31.687752 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683401    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683404    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683407    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683410    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683413    2562 flags.go:64] FLAG: --contention-profiling="false"
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683416    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683419    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683423    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683426    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683431    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683434    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683437    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683440    2562 flags.go:64] FLAG: --enable-load-reader="false"
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683442    2562 flags.go:64] FLAG: --enable-server="true"
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683445    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683450    2562 flags.go:64] FLAG: --event-burst="100"
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683453    2562 flags.go:64] FLAG: --event-qps="50"
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683456    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683459    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683461    2562 flags.go:64] FLAG: --eviction-hard=""
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683465    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683468    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683470    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683473    2562 flags.go:64] FLAG: --eviction-soft=""
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683476    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 27 18:31:31.688345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683479    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683482    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683486    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683489    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683491    2562 flags.go:64] FLAG: --fail-swap-on="true"
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683494    2562 flags.go:64] FLAG: --feature-gates=""
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683498    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683501    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683504    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683507    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683510    2562 flags.go:64] FLAG: --healthz-port="10248"
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683513    2562 flags.go:64] FLAG: --help="false"
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683516    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-132-186.ec2.internal"
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683519    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683526    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683529    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683532    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683535    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683538    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683541    2562 flags.go:64] FLAG: --image-service-endpoint=""
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683544    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683547    2562 flags.go:64] FLAG: --kube-api-burst="100"
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683550    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683553    2562 flags.go:64] FLAG: --kube-api-qps="50"
Jul 27 18:31:31.689004 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683556    2562 flags.go:64] FLAG: --kube-reserved=""
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683558    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683561    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683564    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683567    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683575    2562 flags.go:64] FLAG: --lock-file=""
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683578    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683581    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683584    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683590    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683592    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683596    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683599    2562 flags.go:64] FLAG: --logging-format="text"
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683602    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683605    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683608    2562 flags.go:64] FLAG: --manifest-url=""
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683611    2562 flags.go:64] FLAG: --manifest-url-header=""
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683615    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683618    2562 flags.go:64] FLAG: --max-open-files="1000000"
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683622    2562 flags.go:64] FLAG: --max-pods="110"
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683625    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683628    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683632    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683635    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683639    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 27 18:31:31.689604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683642    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683645    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683654    2562 flags.go:64] FLAG: --node-status-max-images="50"
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683657    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683660    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683663    2562 flags.go:64] FLAG: --pod-cidr=""
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683666    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683672    2562 flags.go:64] FLAG: --pod-manifest-path=""
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683675    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683678    2562 flags.go:64] FLAG: --pods-per-core="0"
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683681    2562 flags.go:64] FLAG: --port="10250"
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683684    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683687    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a06654c95fdd3f4f"
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683690    2562 flags.go:64] FLAG: --qos-reserved=""
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683693    2562 flags.go:64] FLAG: --read-only-port="10255"
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683696    2562 flags.go:64] FLAG: --register-node="true"
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683699    2562 flags.go:64] FLAG: --register-with-taints=""
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683703    2562 flags.go:64] FLAG: --registry-burst="10"
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683706    2562 flags.go:64] FLAG: --registry-qps="5"
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683709    2562 flags.go:64] FLAG: --reserved-cpus=""
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683712    2562 flags.go:64] FLAG: --reserved-memory=""
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683716    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683719    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683722    2562 flags.go:64] FLAG: --rotate-certificates="false"
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683728    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 27 18:31:31.690244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683731    2562 flags.go:64] FLAG: --runonce="false"
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683734    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683737    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683740    2562 flags.go:64] FLAG: --seccomp-default="false"
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683742    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683746    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683749    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683752    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683755    2562 flags.go:64] FLAG: --storage-driver-password="root"
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683758    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683761    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683764    2562 flags.go:64] FLAG: --storage-driver-user="root"
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683767    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683770    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683773    2562 flags.go:64] FLAG: --system-cgroups=""
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683776    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683781    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683784    2562 flags.go:64] FLAG: --tls-cert-file=""
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683787    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683791    2562 flags.go:64] FLAG: --tls-min-version=""
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683795    2562 flags.go:64] FLAG: --tls-private-key-file=""
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683799    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683802    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683805    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683808    2562 flags.go:64] FLAG: --v="2"
Jul 27 18:31:31.690923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683812    2562 flags.go:64] FLAG: --version="false"
Jul 27 18:31:31.691542 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683816    2562 flags.go:64] FLAG: --vmodule=""
Jul 27 18:31:31.691542 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683820    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 27 18:31:31.691542 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.683823    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 27 18:31:31.691542 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683956    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 18:31:31.691542 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683960    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 18:31:31.691542 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683963    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 18:31:31.691542 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683966    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 18:31:31.691542 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683969    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 18:31:31.691542 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683971    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 18:31:31.691542 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683976    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 18:31:31.691542 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683980    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 18:31:31.691542 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683985    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 18:31:31.691542 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683988    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 18:31:31.691542 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683991    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 18:31:31.691542 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683994    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 18:31:31.691542 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.683997    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 18:31:31.691542 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684000    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 18:31:31.691542 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684003    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 18:31:31.691542 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684006    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 18:31:31.692023 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684008    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 18:31:31.692023 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684011    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 18:31:31.692023 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684014    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 18:31:31.692023 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684016    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 18:31:31.692023 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684019    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 18:31:31.692023 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684021    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 18:31:31.692023 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684024    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 18:31:31.692023 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684028    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 18:31:31.692023 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684030    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 18:31:31.692023 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684033    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 18:31:31.692023 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684035    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 18:31:31.692023 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684037    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 18:31:31.692023 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684040    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 18:31:31.692023 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684042    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 18:31:31.692023 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684045    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 18:31:31.692023 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684048    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 18:31:31.692023 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684050    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 18:31:31.692023 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684053    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 18:31:31.692023 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684055    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 18:31:31.692023 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684057    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 18:31:31.692510 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684060    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 18:31:31.692510 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684062    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 18:31:31.692510 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684065    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 18:31:31.692510 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684067    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 18:31:31.692510 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684071    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 18:31:31.692510 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684073    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 18:31:31.692510 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684076    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 18:31:31.692510 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684078    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 18:31:31.692510 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684081    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 18:31:31.692510 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684083    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 18:31:31.692510 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684085    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 18:31:31.692510 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684089    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 18:31:31.692510 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684093    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 18:31:31.692510 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684096    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 18:31:31.692510 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684098    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 18:31:31.692510 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684101    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 18:31:31.692510 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684104    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 18:31:31.692510 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684106    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 18:31:31.692510 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684109    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 18:31:31.692998 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684112    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 18:31:31.692998 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684115    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 18:31:31.692998 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684117    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 18:31:31.692998 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684120    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 18:31:31.692998 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684122    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 18:31:31.692998 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684125    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 18:31:31.692998 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684127    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 18:31:31.692998 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684130    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 18:31:31.692998 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684133    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 18:31:31.692998 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684137    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 18:31:31.692998 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684140    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 18:31:31.692998 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684143    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 18:31:31.692998 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684146    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 18:31:31.692998 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684148    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 18:31:31.692998 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684151    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 18:31:31.692998 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684154    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 18:31:31.692998 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684156    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 18:31:31.692998 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684160    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 18:31:31.692998 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684162    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 18:31:31.692998 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684165    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 18:31:31.693480 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684167    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 18:31:31.693480 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684170    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 18:31:31.693480 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684172    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 18:31:31.693480 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684175    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 18:31:31.693480 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684177    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 18:31:31.693480 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684180    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 18:31:31.693480 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684182    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 18:31:31.693480 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684185    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 18:31:31.693480 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684187    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 18:31:31.693480 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684190    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 18:31:31.693480 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684192    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 18:31:31.693480 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684195    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 18:31:31.693480 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684198    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 18:31:31.693480 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684201    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 18:31:31.693480 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684204    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 18:31:31.693480 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684206    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 18:31:31.693480 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684208    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 18:31:31.693480 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684211    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 18:31:31.693480 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684213    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 18:31:31.693480 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684216    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 18:31:31.693983 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684218    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 18:31:31.693983 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684220    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 18:31:31.693983 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684223    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 18:31:31.693983 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.684225    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 18:31:31.693983 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.684861    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 18:31:31.693983 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.691858    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 27 18:31:31.693983 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.691890    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 27 18:31:31.693983 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.692648    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 27 18:31:31.693983 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.692655    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 27 18:31:31.693983 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692717    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 18:31:31.693983 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692724    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 18:31:31.693983 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692729    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 18:31:31.693983 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692732    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 18:31:31.693983 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692736    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 18:31:31.694382 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692739    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 18:31:31.694382 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692742    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 18:31:31.694382 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692745    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 18:31:31.694382 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692747    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 18:31:31.694382 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692750    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 18:31:31.694382 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692753    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 18:31:31.694382 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692755    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 18:31:31.694382 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692758    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 18:31:31.694382 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692760    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 18:31:31.694382 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692763    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 18:31:31.694382 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692765    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 18:31:31.694382 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692768    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 18:31:31.694382 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692770    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 18:31:31.694382 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692773    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 18:31:31.694382 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692776    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 18:31:31.694382 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692780    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 18:31:31.694382 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692784    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 18:31:31.694382 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692787    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 18:31:31.694382 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692790    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 18:31:31.694382 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692793    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 18:31:31.694885 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692800    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 18:31:31.694885 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692803    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 18:31:31.694885 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692806    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 18:31:31.694885 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692809    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 18:31:31.694885 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692811    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 18:31:31.694885 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692814    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 18:31:31.694885 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692817    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 18:31:31.694885 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692819    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 18:31:31.694885 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692821    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 18:31:31.694885 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692824    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 18:31:31.694885 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692827    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 18:31:31.694885 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692829    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 18:31:31.694885 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692832    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 18:31:31.694885 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692835    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 18:31:31.694885 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692837    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 18:31:31.694885 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692840    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 18:31:31.694885 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692842    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 18:31:31.694885 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692845    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 18:31:31.694885 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692847    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 18:31:31.694885 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692850    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 18:31:31.695377 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692852    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 18:31:31.695377 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692854    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 18:31:31.695377 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692857    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 18:31:31.695377 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692860    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 18:31:31.695377 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692862    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 18:31:31.695377 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692878    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 18:31:31.695377 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692881    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 18:31:31.695377 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692884    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 18:31:31.695377 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692887    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 18:31:31.695377 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692889    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 18:31:31.695377 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692892    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 18:31:31.695377 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692895    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 18:31:31.695377 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692897    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 18:31:31.695377 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692901    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 18:31:31.695377 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692903    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 18:31:31.695377 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692906    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 18:31:31.695377 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692909    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 18:31:31.695377 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692912    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 18:31:31.695377 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692914    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 18:31:31.695377 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692917    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 18:31:31.695880 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692919    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 18:31:31.695880 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692922    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 18:31:31.695880 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692924    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 18:31:31.695880 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692927    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 18:31:31.695880 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692929    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 18:31:31.695880 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692932    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 18:31:31.695880 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692934    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 18:31:31.695880 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692937    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 18:31:31.695880 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692940    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 18:31:31.695880 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692943    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 18:31:31.695880 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692945    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 18:31:31.695880 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692948    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 18:31:31.695880 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692951    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 18:31:31.695880 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692953    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 18:31:31.695880 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692956    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 18:31:31.695880 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692958    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 18:31:31.695880 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692961    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 18:31:31.695880 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692963    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 18:31:31.695880 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692966    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 18:31:31.696349 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692968    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 18:31:31.696349 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692971    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 18:31:31.696349 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692974    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 18:31:31.696349 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692976    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 18:31:31.696349 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692979    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 18:31:31.696349 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692982    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 18:31:31.696349 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692984    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 18:31:31.696349 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692987    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 18:31:31.696349 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692990    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 18:31:31.696349 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692992    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 18:31:31.696349 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692995    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 18:31:31.696349 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.692997    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 18:31:31.696349 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693000    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 18:31:31.696349 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693003    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 18:31:31.696349 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693006    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 18:31:31.696721 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.693012    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 18:31:31.696721 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693131    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 18:31:31.696721 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693136    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 18:31:31.696721 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693139    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 18:31:31.696721 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693141    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 18:31:31.696721 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693144    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 18:31:31.696721 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693146    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 18:31:31.696721 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693149    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 18:31:31.696721 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693151    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 18:31:31.696721 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693154    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 18:31:31.696721 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693156    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 18:31:31.696721 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693159    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 18:31:31.696721 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693161    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 18:31:31.696721 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693164    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 18:31:31.696721 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693166    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 18:31:31.697103 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693169    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 18:31:31.697103 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693171    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 18:31:31.697103 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693175    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 18:31:31.697103 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693177    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 18:31:31.697103 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693180    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 18:31:31.697103 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693183    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 18:31:31.697103 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693186    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 18:31:31.697103 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693188    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 18:31:31.697103 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693191    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 18:31:31.697103 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693194    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 18:31:31.697103 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693196    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 18:31:31.697103 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693199    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 18:31:31.697103 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693202    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 18:31:31.697103 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693205    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 18:31:31.697103 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693207    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 18:31:31.697103 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693210    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 18:31:31.697103 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693212    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 18:31:31.697103 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693215    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 18:31:31.697103 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693217    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 18:31:31.697103 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693219    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 18:31:31.697587 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693222    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 18:31:31.697587 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693224    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 18:31:31.697587 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693227    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 18:31:31.697587 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693229    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 18:31:31.697587 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693232    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 18:31:31.697587 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693235    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 18:31:31.697587 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693239    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 18:31:31.697587 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693242    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 18:31:31.697587 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693244    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 18:31:31.697587 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693247    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 18:31:31.697587 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693249    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 18:31:31.697587 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693253    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 18:31:31.697587 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693256    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 18:31:31.697587 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693258    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 18:31:31.697587 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693261    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 18:31:31.697587 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693264    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 18:31:31.697587 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693268    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 18:31:31.697587 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693271    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 18:31:31.697587 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693273    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 18:31:31.698072 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693276    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 18:31:31.698072 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693279    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 18:31:31.698072 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693281    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 18:31:31.698072 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693284    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 18:31:31.698072 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693287    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 18:31:31.698072 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693290    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 18:31:31.698072 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693292    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 18:31:31.698072 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693295    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 18:31:31.698072 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693297    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 18:31:31.698072 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693299    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 18:31:31.698072 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693302    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 18:31:31.698072 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693305    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 18:31:31.698072 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693308    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 18:31:31.698072 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693311    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 18:31:31.698072 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693314    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 18:31:31.698072 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693317    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 18:31:31.698072 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693319    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 18:31:31.698072 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693322    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 18:31:31.698072 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693324    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 18:31:31.698072 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693327    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 18:31:31.698559 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693330    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 18:31:31.698559 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693332    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 18:31:31.698559 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693335    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 18:31:31.698559 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693337    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 18:31:31.698559 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693340    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 18:31:31.698559 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693343    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 18:31:31.698559 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693345    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 18:31:31.698559 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693348    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 18:31:31.698559 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693350    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 18:31:31.698559 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693355    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 18:31:31.698559 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693358    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 18:31:31.698559 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693360    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 18:31:31.698559 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693363    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 18:31:31.698559 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693365    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 18:31:31.698559 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693368    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 18:31:31.698559 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693370    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 18:31:31.698559 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693373    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 18:31:31.698559 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693376    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 18:31:31.698559 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693379    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 18:31:31.699192 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693381    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 18:31:31.699192 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693384    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 18:31:31.699192 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693386    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 18:31:31.699192 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693389    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 18:31:31.699192 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693391    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 18:31:31.699192 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693393    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 18:31:31.699192 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:31.693396    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 18:31:31.699192 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.693401    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 18:31:31.699192 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.693538    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 27 18:31:31.699192 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.695748    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 27 18:31:31.699192 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.696972    2562 server.go:1019] "Starting client certificate rotation"
Jul 27 18:31:31.699192 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.697078    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 18:31:31.699192 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.697121    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 18:31:31.699543 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.699045    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 18:31:31.700374 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.700337    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 18:31:31.709232 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.709215    2562 log.go:25] "Validated CRI v1 runtime API"
Jul 27 18:31:31.712931 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.712917    2562 log.go:25] "Validated CRI v1 image API"
Jul 27 18:31:31.713796 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.713776    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 27 18:31:31.716005 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.715981    2562 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a15f595b-d80b-4ec4-aa9f-54c6d2baee33:/dev/nvme0n1p4 a82ca965-3cc7-491d-ba04-34896cd2c242:/dev/nvme0n1p3]
Jul 27 18:31:31.716071 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.716002    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 27 18:31:31.722777 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.722659    2562 manager.go:216] Machine: {Timestamp:2026-07-27 18:31:31.721543407 +0000 UTC m=+0.271837030 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099344 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec228be15abd19d02acbde26a13089da SystemUUID:ec228be1-5abd-19d0-2acb-de26a13089da BootID:22025bb5-9028-4fa7-8c2b-b2b3bc423334 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6f:3b:c7:dc:4d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6f:3b:c7:dc:4d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:c4:0d:c2:38:9e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 27 18:31:31.722777 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.722767    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 27 18:31:31.722936 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.722851    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 27 18:31:31.725471 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.725442    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 27 18:31:31.725611 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.725472    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-186.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 27 18:31:31.726111 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.726101    2562 topology_manager.go:138] "Creating topology manager with none policy"
Jul 27 18:31:31.726149 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.726114    2562 container_manager_linux.go:309] "Creating device plugin manager"
Jul 27 18:31:31.726149 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.726122    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 18:31:31.726658 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.726648    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 18:31:31.726904 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.726888    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 18:31:31.727351 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.727340    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 27 18:31:31.729086 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.729073    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 18:31:31.729201 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.729192    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 27 18:31:31.729246 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.729221    2562 kubelet.go:486] "Attempting to sync node with API server"
Jul 27 18:31:31.729246 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.729237    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 27 18:31:31.729316 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.729252    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 27 18:31:31.729316 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.729262    2562 kubelet.go:398] "Adding apiserver pod source"
Jul 27 18:31:31.729316 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.729272    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 27 18:31:31.729652 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.729639    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 18:31:31.729697 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.729660    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 18:31:31.732085 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.732068    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 27 18:31:31.733850 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.733836    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 27 18:31:31.733925 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.733885    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 27 18:31:31.735284 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.735271    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 27 18:31:31.735344 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.735287    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 27 18:31:31.735344 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.735294    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 27 18:31:31.735344 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.735299    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 27 18:31:31.735344 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.735309    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 27 18:31:31.735344 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.735314    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 27 18:31:31.735344 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.735323    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 27 18:31:31.735344 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.735328    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 27 18:31:31.735344 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.735335    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 27 18:31:31.735344 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.735340    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 27 18:31:31.735576 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.735354    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 27 18:31:31.735576 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.735359    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 27 18:31:31.736087 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.736078    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 27 18:31:31.736087 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.736087    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 27 18:31:31.739263 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.739248    2562 server.go:1268] "Started kubelet"
Jul 27 18:31:31.739434 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.739324    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 27 18:31:31.739434 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.739338    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 27 18:31:31.739529 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.739438    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 27 18:31:31.740120 ip-10-0-132-186 systemd[1]: Started Kubernetes Kubelet.
Jul 27 18:31:31.740371 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:31.740345    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 18:31:31.740463 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:31.740344    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-186.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 18:31:31.740761 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.740619    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-186.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 18:31:31.741242 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.741206    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 27 18:31:31.742003 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.741985    2562 server.go:310] "Adding debug handlers to kubelet server"
Jul 27 18:31:31.744689 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:31.744669    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 27 18:31:31.745840 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:31.745041    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-186.ec2.internal.18c6391b4e480d55  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-186.ec2.internal,UID:ip-10-0-132-186.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-186.ec2.internal,},FirstTimestamp:2026-07-27 18:31:31.739225429 +0000 UTC m=+0.289519052,LastTimestamp:2026-07-27 18:31:31.739225429 +0000 UTC m=+0.289519052,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-186.ec2.internal,}"
Jul 27 18:31:31.747566 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.747547    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 27 18:31:31.747658 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.747552    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 27 18:31:31.747716 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:31.747692    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-186.ec2.internal\" not found"
Jul 27 18:31:31.748168 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.748116    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 27 18:31:31.748168 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.748129    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 27 18:31:31.748168 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.748147    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 27 18:31:31.748411 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.748395    2562 reconstruct.go:97] "Volume reconstruction finished"
Jul 27 18:31:31.748411 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.748408    2562 reconciler.go:29] "Reconciler: start to sync state"
Jul 27 18:31:31.748547 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.748453    2562 factory.go:55] Registering systemd factory
Jul 27 18:31:31.748547 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.748470    2562 factory.go:223] Registration of the systemd container factory successfully
Jul 27 18:31:31.748724 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.748711    2562 factory.go:153] Registering CRI-O factory
Jul 27 18:31:31.748762 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.748726    2562 factory.go:223] Registration of the crio container factory successfully
Jul 27 18:31:31.748791 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.748768    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 27 18:31:31.748823 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.748786    2562 factory.go:103] Registering Raw factory
Jul 27 18:31:31.748823 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.748822    2562 manager.go:1196] Started watching for new ooms in manager
Jul 27 18:31:31.749746 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.749723    2562 manager.go:349] Starting recovery of all containers
Jul 27 18:31:31.753479 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.753449    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gj7st"
Jul 27 18:31:31.754550 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:31.754514    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-186.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 27 18:31:31.754719 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:31.754680    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 27 18:31:31.762392 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.762348    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gj7st"
Jul 27 18:31:31.763051 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.763029    2562 manager.go:354] Recovery completed
Jul 27 18:31:31.764475 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:31.764457    2562 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 27 18:31:31.767751 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.767735    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 18:31:31.769811 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.769793    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 18:31:31.769933 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.769823    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 18:31:31.769933 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.769833    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeHasSufficientPID"
Jul 27 18:31:31.770497 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.770478    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 27 18:31:31.770497 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.770492    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 27 18:31:31.770607 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.770514    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 18:31:31.770607 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.770493    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 18:31:31.771397 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.771379    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 18:31:31.771484 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.771411    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 18:31:31.771484 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.771424    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeHasSufficientPID"
Jul 27 18:31:31.772071 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:31.771999    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-186.ec2.internal.18c6391b501ab837  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-186.ec2.internal,UID:ip-10-0-132-186.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-186.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-186.ec2.internal,},FirstTimestamp:2026-07-27 18:31:31.769808951 +0000 UTC m=+0.320102575,LastTimestamp:2026-07-27 18:31:31.769808951 +0000 UTC m=+0.320102575,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-186.ec2.internal,}"
Jul 27 18:31:31.772172 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.772100    2562 policy_none.go:49] "None policy: Start"
Jul 27 18:31:31.772172 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.772115    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 27 18:31:31.772172 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.772125    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 27 18:31:31.773130 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.773118    2562 policy_none.go:47] "Start"
Jul 27 18:31:31.810541 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.810518    2562 manager.go:337] "Starting Device Plugin manager"
Jul 27 18:31:31.810675 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:31.810555    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 27 18:31:31.810675 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.810566    2562 server.go:87] "Starting device plugin registration server"
Jul 27 18:31:31.811110 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.810890    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 27 18:31:31.811149 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.811122    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 27 18:31:31.811259 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.811229    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 27 18:31:31.811353 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.811318    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 27 18:31:31.811353 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.811328    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 27 18:31:31.811935 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:31.811911    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 27 18:31:31.812008 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:31.811963    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-186.ec2.internal\" not found"
Jul 27 18:31:31.912149 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.912111    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 18:31:31.913236 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.913166    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 18:31:31.913236 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.913197    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 18:31:31.913236 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.913210    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeHasSufficientPID"
Jul 27 18:31:31.913375 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.913245    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-186.ec2.internal"
Jul 27 18:31:31.921426 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.921407    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-186.ec2.internal"
Jul 27 18:31:31.921481 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:31.921433    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-186.ec2.internal\": node \"ip-10-0-132-186.ec2.internal\" not found"
Jul 27 18:31:31.928947 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.928858    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 27 18:31:31.930326 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.930309    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 27 18:31:31.930390 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.930357    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 27 18:31:31.930390 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.930383    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 27 18:31:31.930479 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:31.930421    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 27 18:31:31.932716 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:31.932697    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 18:31:31.935247 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:31.935231    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-186.ec2.internal\" not found"
Jul 27 18:31:32.031444 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.031403    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-186.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-186.ec2.internal"]
Jul 27 18:31:32.031607 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.031511    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 18:31:32.033142 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.033124    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 18:31:32.033216 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.033156    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 18:31:32.033216 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.033170    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeHasSufficientPID"
Jul 27 18:31:32.034426 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.034414    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 18:31:32.034546 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.034529    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-186.ec2.internal"
Jul 27 18:31:32.034590 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.034569    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 18:31:32.035160 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.035143    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 18:31:32.035244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.035175    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 18:31:32.035244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.035186    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeHasSufficientPID"
Jul 27 18:31:32.035244 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.035143    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 18:31:32.035327 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.035250    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 18:31:32.035327 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.035264    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeHasSufficientPID"
Jul 27 18:31:32.035399 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:32.035342    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-186.ec2.internal\" not found"
Jul 27 18:31:32.036359 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.036341    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-186.ec2.internal"
Jul 27 18:31:32.036410 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.036368    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 18:31:32.037067 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.037050    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 18:31:32.037140 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.037080    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 18:31:32.037140 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.037095    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeHasSufficientPID"
Jul 27 18:31:32.050333 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.050292    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/129a2108b5a3803a480ffe6e1a6b9b55-config\") pod \"kube-apiserver-proxy-ip-10-0-132-186.ec2.internal\" (UID: \"129a2108b5a3803a480ffe6e1a6b9b55\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-186.ec2.internal"
Jul 27 18:31:32.050405 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.050347    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1a59ab1fcabaf70a08c93371f33a8e3a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-186.ec2.internal\" (UID: \"1a59ab1fcabaf70a08c93371f33a8e3a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-186.ec2.internal"
Jul 27 18:31:32.050405 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.050375    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a59ab1fcabaf70a08c93371f33a8e3a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-186.ec2.internal\" (UID: \"1a59ab1fcabaf70a08c93371f33a8e3a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-186.ec2.internal"
Jul 27 18:31:32.067941 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:32.067913    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-186.ec2.internal\" not found" node="ip-10-0-132-186.ec2.internal"
Jul 27 18:31:32.072656 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:32.072637    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-186.ec2.internal\" not found" node="ip-10-0-132-186.ec2.internal"
Jul 27 18:31:32.136284 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:32.136256    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-186.ec2.internal\" not found"
Jul 27 18:31:32.150907 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.150881    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1a59ab1fcabaf70a08c93371f33a8e3a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-186.ec2.internal\" (UID: \"1a59ab1fcabaf70a08c93371f33a8e3a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-186.ec2.internal"
Jul 27 18:31:32.151020 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.150911    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a59ab1fcabaf70a08c93371f33a8e3a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-186.ec2.internal\" (UID: \"1a59ab1fcabaf70a08c93371f33a8e3a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-186.ec2.internal"
Jul 27 18:31:32.151020 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.150935    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/129a2108b5a3803a480ffe6e1a6b9b55-config\") pod \"kube-apiserver-proxy-ip-10-0-132-186.ec2.internal\" (UID: \"129a2108b5a3803a480ffe6e1a6b9b55\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-186.ec2.internal"
Jul 27 18:31:32.151020 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.150954    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1a59ab1fcabaf70a08c93371f33a8e3a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-186.ec2.internal\" (UID: \"1a59ab1fcabaf70a08c93371f33a8e3a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-186.ec2.internal"
Jul 27 18:31:32.151020 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.150979    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/129a2108b5a3803a480ffe6e1a6b9b55-config\") pod \"kube-apiserver-proxy-ip-10-0-132-186.ec2.internal\" (UID: \"129a2108b5a3803a480ffe6e1a6b9b55\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-186.ec2.internal"
Jul 27 18:31:32.151020 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.150981    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a59ab1fcabaf70a08c93371f33a8e3a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-186.ec2.internal\" (UID: \"1a59ab1fcabaf70a08c93371f33a8e3a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-186.ec2.internal"
Jul 27 18:31:32.237137 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:32.237040    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-186.ec2.internal\" not found"
Jul 27 18:31:32.337614 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:32.337576    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-186.ec2.internal\" not found"
Jul 27 18:31:32.371098 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.371066    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-186.ec2.internal"
Jul 27 18:31:32.376157 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.376130    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-186.ec2.internal"
Jul 27 18:31:32.438393 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:32.438344    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-186.ec2.internal\" not found"
Jul 27 18:31:32.538943 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:32.538854    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-186.ec2.internal\" not found"
Jul 27 18:31:32.639496 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:32.639457    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-186.ec2.internal\" not found"
Jul 27 18:31:32.648189 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.648169    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 18:31:32.696353 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.696324    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 27 18:31:32.696748 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.696472    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 27 18:31:32.696748 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.696520    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 27 18:31:32.729656 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.729618    2562 apiserver.go:52] "Watching apiserver"
Jul 27 18:31:32.745205 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.745173    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 27 18:31:32.745514 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.745489    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-qhfd6","openshift-image-registry/node-ca-jz5n4","openshift-multus/network-metrics-daemon-mhtmr","openshift-network-diagnostics/network-check-target-zqgfw","openshift-network-operator/iptables-alerter-nphwb","kube-system/global-pull-secret-syncer-rmf9d","openshift-dns/node-resolver-8kqdg","openshift-multus/multus-9gvlm","openshift-multus/multus-additional-cni-plugins-q9k9p","openshift-ovn-kubernetes/ovnkube-node-5wkjf","kube-system/konnectivity-agent-scl2m","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"]
Jul 27 18:31:32.746936 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.746916    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.747711 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.747688    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 27 18:31:32.747912 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.747894    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-186.ec2.internal"
Jul 27 18:31:32.748466 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.748445    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jz5n4"
Jul 27 18:31:32.748885 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.748861    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 27 18:31:32.749045 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.749023    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 27 18:31:32.749197 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.749180    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jh7mt\""
Jul 27 18:31:32.749660 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.749644    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:32.749745 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:32.749695    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mhtmr" podUID="88f67acb-91af-4850-bb4d-81ff499bc672"
Jul 27 18:31:32.750810 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.750788    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:32.750932 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.750887    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 27 18:31:32.750932 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:32.750866    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zqgfw" podUID="8e6d520b-feb0-4b85-921f-7fe3f1ab3549"
Jul 27 18:31:32.751351 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.751334    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 27 18:31:32.751439 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.751424    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-99mjx\""
Jul 27 18:31:32.751493 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.751441    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 27 18:31:32.752119 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.752103    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nphwb"
Jul 27 18:31:32.753202 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.753185    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8kqdg"
Jul 27 18:31:32.754069 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754053    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 27 18:31:32.754135 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754095    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7r7qn\""
Jul 27 18:31:32.754200 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754185    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 27 18:31:32.754246 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754231    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2efd4785-cf26-4815-86af-86dd8dbd5108-host\") pod \"node-ca-jz5n4\" (UID: \"2efd4785-cf26-4815-86af-86dd8dbd5108\") " pod="openshift-image-registry/node-ca-jz5n4"
Jul 27 18:31:32.754290 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754259    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/846631a2-6652-4ae8-9d36-d60069b1321c-etc-tuned\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.754290 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754277    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/846631a2-6652-4ae8-9d36-d60069b1321c-tmp\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.754390 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754301    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-etc-modprobe-d\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.754390 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754317    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-etc-systemd\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.754390 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754333    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn2cx\" (UniqueName: \"kubernetes.io/projected/846631a2-6652-4ae8-9d36-d60069b1321c-kube-api-access-nn2cx\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.754390 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754356    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2efd4785-cf26-4815-86af-86dd8dbd5108-serviceca\") pod \"node-ca-jz5n4\" (UID: \"2efd4785-cf26-4815-86af-86dd8dbd5108\") " pod="openshift-image-registry/node-ca-jz5n4"
Jul 27 18:31:32.754538 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754409    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6hsn\" (UniqueName: \"kubernetes.io/projected/2efd4785-cf26-4815-86af-86dd8dbd5108-kube-api-access-g6hsn\") pod \"node-ca-jz5n4\" (UID: \"2efd4785-cf26-4815-86af-86dd8dbd5108\") " pod="openshift-image-registry/node-ca-jz5n4"
Jul 27 18:31:32.754538 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754437    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-lib-modules\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.754538 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754456    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-host\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.754538 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754476    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-etc-sysconfig\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.754538 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754502    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-etc-sysctl-d\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.754538 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754517    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.754778 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754519    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 27 18:31:32.754778 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754522    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-etc-sysctl-conf\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.754778 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754698    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-run\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.754778 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754737    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-sys\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.754991 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754775    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-etc-kubernetes\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.754991 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.754801    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-var-lib-kubelet\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.755095 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.755045    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hz5m7\""
Jul 27 18:31:32.755095 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.755082    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 27 18:31:32.755183 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.755081    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 27 18:31:32.755985 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.755963    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.757310 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.757283    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.758067 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.758048    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 27 18:31:32.758138 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.758084    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 27 18:31:32.758326 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.758312    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 27 18:31:32.758420 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.758401    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 27 18:31:32.758556 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.758541    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-n7qqs\""
Jul 27 18:31:32.758646 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.758632    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-scl2m"
Jul 27 18:31:32.758803 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.758784    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 27 18:31:32.759427 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.759412    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 27 18:31:32.760022 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.760004    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.761631 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.761263    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 27 18:31:32.761631 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.761357    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8jjp2\""
Jul 27 18:31:32.761631 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.761421    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:32.761631 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.761422    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 27 18:31:32.761631 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.761360    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-186.ec2.internal"]
Jul 27 18:31:32.762020 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.761998    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 27 18:31:32.762241 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.761357    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 27 18:31:32.762302 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.762239    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 18:31:32.762684 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.762667    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8b957\""
Jul 27 18:31:32.762737 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.762720    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 27 18:31:32.763084 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.763066    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-186.ec2.internal"
Jul 27 18:31:32.763284 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:32.763129    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmf9d" podUID="914d3bcb-4f6c-4153-aec8-e4547ff69068"
Jul 27 18:31:32.763505 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.763486    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jwjqh\""
Jul 27 18:31:32.763564 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.763547    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 27 18:31:32.763644 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.763625    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 27 18:31:32.763721 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.763707    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 27 18:31:32.763888 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.763872    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 27 18:31:32.764141 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.764124    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 27 18:31:32.764231 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.764214    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xdwfh\""
Jul 27 18:31:32.764564 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.764548    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 27 18:31:32.766706 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.766662    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-26 18:26:32 +0000 UTC" deadline="2028-01-02 16:17:02.400430077 +0000 UTC"
Jul 27 18:31:32.766774 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.766707    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12573h45m29.633725947s"
Jul 27 18:31:32.766774 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.766712    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 18:31:32.771503 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.771484    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-186.ec2.internal"]
Jul 27 18:31:32.771738 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.771724    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 18:31:32.789822 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.789763    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-sb49g"
Jul 27 18:31:32.800258 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.800232    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-sb49g"
Jul 27 18:31:32.831423 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.831397    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 18:31:32.849337 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.849320    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 27 18:31:32.855019 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.854998    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs\") pod \"network-metrics-daemon-mhtmr\" (UID: \"88f67acb-91af-4850-bb4d-81ff499bc672\") " pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:32.855091 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855028    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-cnibin\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.855091 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855049    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a56a2199-4863-403e-bc3e-b601728e5406-konnectivity-ca\") pod \"konnectivity-agent-scl2m\" (UID: \"a56a2199-4863-403e-bc3e-b601728e5406\") " pod="kube-system/konnectivity-agent-scl2m"
Jul 27 18:31:32.855091 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855070    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/846631a2-6652-4ae8-9d36-d60069b1321c-tmp\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.855091 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855086    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-run-ovn\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.855282 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855131    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-multus-socket-dir-parent\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.855282 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855186    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-hostroot\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.855282 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855216    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-multus-daemon-config\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.855282 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855246    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-run-openvswitch\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.855282 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855271    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret\") pod \"global-pull-secret-syncer-rmf9d\" (UID: \"914d3bcb-4f6c-4153-aec8-e4547ff69068\") " pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:32.855505 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855289    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-host-var-lib-kubelet\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.855505 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855324    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/be23ce78-126d-46a9-95d2-40d68c20f6f2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.855505 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855366    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-lib-modules\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.855505 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855423    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 27 18:31:32.855505 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855437    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6ac2accf-aafd-4d06-8d73-230036eea5a4-ovn-node-metrics-cert\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.855505 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855466    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b54bfb21-6fdf-49b2-8615-5b48d84edd66-device-dir\") pod \"aws-ebs-csi-driver-node-q5cgq\" (UID: \"b54bfb21-6fdf-49b2-8615-5b48d84edd66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.855505 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855492    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brczk\" (UniqueName: \"kubernetes.io/projected/88f67acb-91af-4850-bb4d-81ff499bc672-kube-api-access-brczk\") pod \"network-metrics-daemon-mhtmr\" (UID: \"88f67acb-91af-4850-bb4d-81ff499bc672\") " pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:32.855505 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855497    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-lib-modules\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.855839 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855515    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-host-run-netns\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.855839 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855541    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-host-var-lib-cni-multus\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.855839 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855589    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-host-run-multus-certs\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.855839 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855616    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6ac2accf-aafd-4d06-8d73-230036eea5a4-ovnkube-script-lib\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.855839 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855656    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bklv\" (UniqueName: \"kubernetes.io/projected/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-kube-api-access-4bklv\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.855839 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855690    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be23ce78-126d-46a9-95d2-40d68c20f6f2-os-release\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.855839 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855724    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-host-kubelet\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.855839 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855757    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-host-run-ovn-kubernetes\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.855839 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855778    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mxhd\" (UniqueName: \"kubernetes.io/projected/b54bfb21-6fdf-49b2-8615-5b48d84edd66-kube-api-access-6mxhd\") pod \"aws-ebs-csi-driver-node-q5cgq\" (UID: \"b54bfb21-6fdf-49b2-8615-5b48d84edd66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.855839 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855794    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgmtn\" (UniqueName: \"kubernetes.io/projected/f74b3920-3180-47a1-9895-d7f14e32d952-kube-api-access-vgmtn\") pod \"iptables-alerter-nphwb\" (UID: \"f74b3920-3180-47a1-9895-d7f14e32d952\") " pod="openshift-network-operator/iptables-alerter-nphwb"
Jul 27 18:31:32.855839 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855811    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-multus-conf-dir\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.856289 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855841    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn2cx\" (UniqueName: \"kubernetes.io/projected/846631a2-6652-4ae8-9d36-d60069b1321c-kube-api-access-nn2cx\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.856289 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855888    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-run-systemd\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.856289 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855911    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-host-cni-bin\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.856289 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.855999    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-multus-cni-dir\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.856289 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.856031    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-host-run-k8s-cni-cncf-io\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.856289 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.856053    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-host-var-lib-cni-bin\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.856289 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.856072    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-etc-sysconfig\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.856289 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.856090    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6ac2accf-aafd-4d06-8d73-230036eea5a4-env-overrides\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.856289 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.856105    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-host-slash\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.856289 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.856122    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-host-run-netns\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.856289 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.856140    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-node-log\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.856289 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.856155    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-host-cni-netd\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.856289 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.856173    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-etc-sysconfig\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.856289 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.856186    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6ac2accf-aafd-4d06-8d73-230036eea5a4-ovnkube-config\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.856289 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.856213    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-systemd-units\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.856289 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.856272    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/914d3bcb-4f6c-4153-aec8-e4547ff69068-kubelet-config\") pod \"global-pull-secret-syncer-rmf9d\" (UID: \"914d3bcb-4f6c-4153-aec8-e4547ff69068\") " pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:32.856979 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.856305    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-etc-kubernetes\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.856979 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.856510    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be23ce78-126d-46a9-95d2-40d68c20f6f2-cni-binary-copy\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.856979 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.856558    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-etc-systemd\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.856979 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.856582    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2efd4785-cf26-4815-86af-86dd8dbd5108-serviceca\") pod \"node-ca-jz5n4\" (UID: \"2efd4785-cf26-4815-86af-86dd8dbd5108\") " pod="openshift-image-registry/node-ca-jz5n4"
Jul 27 18:31:32.856979 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.856605    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrbmb\" (UniqueName: \"kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb\") pod \"network-check-target-zqgfw\" (UID: \"8e6d520b-feb0-4b85-921f-7fe3f1ab3549\") " pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:32.856979 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.856686    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-etc-systemd\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.857335 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.856628    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-etc-openvswitch\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.857335 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857203    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2efd4785-cf26-4815-86af-86dd8dbd5108-serviceca\") pod \"node-ca-jz5n4\" (UID: \"2efd4785-cf26-4815-86af-86dd8dbd5108\") " pod="openshift-image-registry/node-ca-jz5n4"
Jul 27 18:31:32.857335 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857211    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b54bfb21-6fdf-49b2-8615-5b48d84edd66-socket-dir\") pod \"aws-ebs-csi-driver-node-q5cgq\" (UID: \"b54bfb21-6fdf-49b2-8615-5b48d84edd66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.857335 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857299    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f74b3920-3180-47a1-9895-d7f14e32d952-host-slash\") pod \"iptables-alerter-nphwb\" (UID: \"f74b3920-3180-47a1-9895-d7f14e32d952\") " pod="openshift-network-operator/iptables-alerter-nphwb"
Jul 27 18:31:32.857335 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857327    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/05f049d3-8c6b-4155-8586-be26e91c5bb9-hosts-file\") pod \"node-resolver-8kqdg\" (UID: \"05f049d3-8c6b-4155-8586-be26e91c5bb9\") " pod="openshift-dns/node-resolver-8kqdg"
Jul 27 18:31:32.857596 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857352    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-system-cni-dir\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.857596 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857388    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-host\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.857596 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857415    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-etc-sysctl-d\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.857596 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857442    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-sys\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.857596 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857469    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.857596 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857489    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-host\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.857596 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857498    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b54bfb21-6fdf-49b2-8615-5b48d84edd66-sys-fs\") pod \"aws-ebs-csi-driver-node-q5cgq\" (UID: \"b54bfb21-6fdf-49b2-8615-5b48d84edd66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.857596 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857525    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05f049d3-8c6b-4155-8586-be26e91c5bb9-tmp-dir\") pod \"node-resolver-8kqdg\" (UID: \"05f049d3-8c6b-4155-8586-be26e91c5bb9\") " pod="openshift-dns/node-resolver-8kqdg"
Jul 27 18:31:32.857596 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857529    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-sys\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.857596 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857527    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-etc-sysctl-d\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.857596 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857554    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6gsq\" (UniqueName: \"kubernetes.io/projected/be23ce78-126d-46a9-95d2-40d68c20f6f2-kube-api-access-q6gsq\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.858019 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857608    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a56a2199-4863-403e-bc3e-b601728e5406-agent-certs\") pod \"konnectivity-agent-scl2m\" (UID: \"a56a2199-4863-403e-bc3e-b601728e5406\") " pod="kube-system/konnectivity-agent-scl2m"
Jul 27 18:31:32.858019 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857643    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-etc-kubernetes\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.858019 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857682    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-var-lib-kubelet\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.858019 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857698    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-etc-kubernetes\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.858019 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857719    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b54bfb21-6fdf-49b2-8615-5b48d84edd66-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q5cgq\" (UID: \"b54bfb21-6fdf-49b2-8615-5b48d84edd66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.858019 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857732    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-var-lib-kubelet\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.858019 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857746    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdn6f\" (UniqueName: \"kubernetes.io/projected/05f049d3-8c6b-4155-8586-be26e91c5bb9-kube-api-access-qdn6f\") pod \"node-resolver-8kqdg\" (UID: \"05f049d3-8c6b-4155-8586-be26e91c5bb9\") " pod="openshift-dns/node-resolver-8kqdg"
Jul 27 18:31:32.858019 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857773    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be23ce78-126d-46a9-95d2-40d68c20f6f2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.858019 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857798    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/846631a2-6652-4ae8-9d36-d60069b1321c-etc-tuned\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.858019 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857828    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-var-lib-openvswitch\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.858019 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857851    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be23ce78-126d-46a9-95d2-40d68c20f6f2-system-cni-dir\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.858019 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857874    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-etc-modprobe-d\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.858019 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857915    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6hsn\" (UniqueName: \"kubernetes.io/projected/2efd4785-cf26-4815-86af-86dd8dbd5108-kube-api-access-g6hsn\") pod \"node-ca-jz5n4\" (UID: \"2efd4785-cf26-4815-86af-86dd8dbd5108\") " pod="openshift-image-registry/node-ca-jz5n4"
Jul 27 18:31:32.858019 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857936    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmn4b\" (UniqueName: \"kubernetes.io/projected/6ac2accf-aafd-4d06-8d73-230036eea5a4-kube-api-access-vmn4b\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.858019 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857961    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b54bfb21-6fdf-49b2-8615-5b48d84edd66-registration-dir\") pod \"aws-ebs-csi-driver-node-q5cgq\" (UID: \"b54bfb21-6fdf-49b2-8615-5b48d84edd66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.858019 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.857983    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f74b3920-3180-47a1-9895-d7f14e32d952-iptables-alerter-script\") pod \"iptables-alerter-nphwb\" (UID: \"f74b3920-3180-47a1-9895-d7f14e32d952\") " pod="openshift-network-operator/iptables-alerter-nphwb"
Jul 27 18:31:32.858558 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.858006    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-os-release\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.858558 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.858026    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-cni-binary-copy\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.858558 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.858043    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-etc-modprobe-d\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.858558 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.858050    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-etc-sysctl-conf\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.858558 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.858079    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-run\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.858558 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.858170    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-log-socket\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.858558 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.858182    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-run\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.858558 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.858183    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/846631a2-6652-4ae8-9d36-d60069b1321c-etc-sysctl-conf\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.858558 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.858196    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b54bfb21-6fdf-49b2-8615-5b48d84edd66-etc-selinux\") pod \"aws-ebs-csi-driver-node-q5cgq\" (UID: \"b54bfb21-6fdf-49b2-8615-5b48d84edd66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.858558 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.858222    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be23ce78-126d-46a9-95d2-40d68c20f6f2-cnibin\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.858558 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.858269    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/be23ce78-126d-46a9-95d2-40d68c20f6f2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.858558 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.858292    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2efd4785-cf26-4815-86af-86dd8dbd5108-host\") pod \"node-ca-jz5n4\" (UID: \"2efd4785-cf26-4815-86af-86dd8dbd5108\") " pod="openshift-image-registry/node-ca-jz5n4"
Jul 27 18:31:32.858558 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.858312    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/914d3bcb-4f6c-4153-aec8-e4547ff69068-dbus\") pod \"global-pull-secret-syncer-rmf9d\" (UID: \"914d3bcb-4f6c-4153-aec8-e4547ff69068\") " pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:32.858558 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.858367    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2efd4785-cf26-4815-86af-86dd8dbd5108-host\") pod \"node-ca-jz5n4\" (UID: \"2efd4785-cf26-4815-86af-86dd8dbd5108\") " pod="openshift-image-registry/node-ca-jz5n4"
Jul 27 18:31:32.859239 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.859223    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/846631a2-6652-4ae8-9d36-d60069b1321c-tmp\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.859964 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.859933    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/846631a2-6652-4ae8-9d36-d60069b1321c-etc-tuned\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.865038 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.865009    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn2cx\" (UniqueName: \"kubernetes.io/projected/846631a2-6652-4ae8-9d36-d60069b1321c-kube-api-access-nn2cx\") pod \"tuned-qhfd6\" (UID: \"846631a2-6652-4ae8-9d36-d60069b1321c\") " pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:32.865809 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.865783    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6hsn\" (UniqueName: \"kubernetes.io/projected/2efd4785-cf26-4815-86af-86dd8dbd5108-kube-api-access-g6hsn\") pod \"node-ca-jz5n4\" (UID: \"2efd4785-cf26-4815-86af-86dd8dbd5108\") " pod="openshift-image-registry/node-ca-jz5n4"
Jul 27 18:31:32.959045 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959007    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6ac2accf-aafd-4d06-8d73-230036eea5a4-ovn-node-metrics-cert\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.959045 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959050    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b54bfb21-6fdf-49b2-8615-5b48d84edd66-device-dir\") pod \"aws-ebs-csi-driver-node-q5cgq\" (UID: \"b54bfb21-6fdf-49b2-8615-5b48d84edd66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.959300 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959079    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brczk\" (UniqueName: \"kubernetes.io/projected/88f67acb-91af-4850-bb4d-81ff499bc672-kube-api-access-brczk\") pod \"network-metrics-daemon-mhtmr\" (UID: \"88f67acb-91af-4850-bb4d-81ff499bc672\") " pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:32.959300 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959103    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-host-run-netns\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.959300 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959156    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-host-run-netns\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.959300 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959189    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b54bfb21-6fdf-49b2-8615-5b48d84edd66-device-dir\") pod \"aws-ebs-csi-driver-node-q5cgq\" (UID: \"b54bfb21-6fdf-49b2-8615-5b48d84edd66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.959300 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959215    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-host-var-lib-cni-multus\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.959300 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959251    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-host-run-multus-certs\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.959300 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959257    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-host-var-lib-cni-multus\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.959300 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959287    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6ac2accf-aafd-4d06-8d73-230036eea5a4-ovnkube-script-lib\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.959300 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959293    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-host-run-multus-certs\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.959727 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959369    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bklv\" (UniqueName: \"kubernetes.io/projected/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-kube-api-access-4bklv\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.959727 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959401    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be23ce78-126d-46a9-95d2-40d68c20f6f2-os-release\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.959727 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959429    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-host-kubelet\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.959727 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959480    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-host-kubelet\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.959727 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959499    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be23ce78-126d-46a9-95d2-40d68c20f6f2-os-release\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.959727 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959529    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-host-run-ovn-kubernetes\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.959727 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959549    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mxhd\" (UniqueName: \"kubernetes.io/projected/b54bfb21-6fdf-49b2-8615-5b48d84edd66-kube-api-access-6mxhd\") pod \"aws-ebs-csi-driver-node-q5cgq\" (UID: \"b54bfb21-6fdf-49b2-8615-5b48d84edd66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.959727 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959572    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgmtn\" (UniqueName: \"kubernetes.io/projected/f74b3920-3180-47a1-9895-d7f14e32d952-kube-api-access-vgmtn\") pod \"iptables-alerter-nphwb\" (UID: \"f74b3920-3180-47a1-9895-d7f14e32d952\") " pod="openshift-network-operator/iptables-alerter-nphwb"
Jul 27 18:31:32.959727 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959596    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-multus-conf-dir\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.959727 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959614    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-run-systemd\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.959727 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959625    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-host-run-ovn-kubernetes\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.959727 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959635    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-host-cni-bin\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.959727 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959673    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-host-cni-bin\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.959727 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959671    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-multus-conf-dir\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.959727 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959685    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-multus-cni-dir\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.959727 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959712    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-host-run-k8s-cni-cncf-io\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.959727 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959703    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-run-systemd\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.960793 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959733    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-host-var-lib-cni-bin\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.960793 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959766    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6ac2accf-aafd-4d06-8d73-230036eea5a4-env-overrides\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.960793 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959779    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-host-run-k8s-cni-cncf-io\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.960793 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959768    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-host-var-lib-cni-bin\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.960793 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959796    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-host-slash\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.960793 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959819    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-multus-cni-dir\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.960793 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959830    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-host-run-netns\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.960793 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959822    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6ac2accf-aafd-4d06-8d73-230036eea5a4-ovnkube-script-lib\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.960793 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959831    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-host-slash\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.960793 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959859    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-host-run-netns\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.960793 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959927    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-node-log\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.960793 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959966    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-host-cni-netd\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.960793 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959990    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6ac2accf-aafd-4d06-8d73-230036eea5a4-ovnkube-config\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.960793 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960004    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-host-cni-netd\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.960793 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960013    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-systemd-units\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.960793 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.959964    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-node-log\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.960793 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960053    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-systemd-units\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.960793 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960077    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/914d3bcb-4f6c-4153-aec8-e4547ff69068-kubelet-config\") pod \"global-pull-secret-syncer-rmf9d\" (UID: \"914d3bcb-4f6c-4153-aec8-e4547ff69068\") " pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:32.961718 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960139    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/914d3bcb-4f6c-4153-aec8-e4547ff69068-kubelet-config\") pod \"global-pull-secret-syncer-rmf9d\" (UID: \"914d3bcb-4f6c-4153-aec8-e4547ff69068\") " pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:32.961718 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960182    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-etc-kubernetes\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.961718 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960190    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6ac2accf-aafd-4d06-8d73-230036eea5a4-env-overrides\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.961718 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960217    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-etc-kubernetes\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.961718 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960227    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be23ce78-126d-46a9-95d2-40d68c20f6f2-cni-binary-copy\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.961718 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960257    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrbmb\" (UniqueName: \"kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb\") pod \"network-check-target-zqgfw\" (UID: \"8e6d520b-feb0-4b85-921f-7fe3f1ab3549\") " pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:32.961718 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960274    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-etc-openvswitch\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.961718 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960291    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b54bfb21-6fdf-49b2-8615-5b48d84edd66-socket-dir\") pod \"aws-ebs-csi-driver-node-q5cgq\" (UID: \"b54bfb21-6fdf-49b2-8615-5b48d84edd66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.961718 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960316    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f74b3920-3180-47a1-9895-d7f14e32d952-host-slash\") pod \"iptables-alerter-nphwb\" (UID: \"f74b3920-3180-47a1-9895-d7f14e32d952\") " pod="openshift-network-operator/iptables-alerter-nphwb"
Jul 27 18:31:32.961718 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960345    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/05f049d3-8c6b-4155-8586-be26e91c5bb9-hosts-file\") pod \"node-resolver-8kqdg\" (UID: \"05f049d3-8c6b-4155-8586-be26e91c5bb9\") " pod="openshift-dns/node-resolver-8kqdg"
Jul 27 18:31:32.961718 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960391    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-system-cni-dir\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.961718 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960422    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.961718 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960451    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b54bfb21-6fdf-49b2-8615-5b48d84edd66-sys-fs\") pod \"aws-ebs-csi-driver-node-q5cgq\" (UID: \"b54bfb21-6fdf-49b2-8615-5b48d84edd66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.961718 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960486    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05f049d3-8c6b-4155-8586-be26e91c5bb9-tmp-dir\") pod \"node-resolver-8kqdg\" (UID: \"05f049d3-8c6b-4155-8586-be26e91c5bb9\") " pod="openshift-dns/node-resolver-8kqdg"
Jul 27 18:31:32.961718 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960503    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-etc-openvswitch\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.961718 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960515    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6gsq\" (UniqueName: \"kubernetes.io/projected/be23ce78-126d-46a9-95d2-40d68c20f6f2-kube-api-access-q6gsq\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.961718 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960540    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a56a2199-4863-403e-bc3e-b601728e5406-agent-certs\") pod \"konnectivity-agent-scl2m\" (UID: \"a56a2199-4863-403e-bc3e-b601728e5406\") " pod="kube-system/konnectivity-agent-scl2m"
Jul 27 18:31:32.962595 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960569    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b54bfb21-6fdf-49b2-8615-5b48d84edd66-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q5cgq\" (UID: \"b54bfb21-6fdf-49b2-8615-5b48d84edd66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.962595 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960577    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b54bfb21-6fdf-49b2-8615-5b48d84edd66-socket-dir\") pod \"aws-ebs-csi-driver-node-q5cgq\" (UID: \"b54bfb21-6fdf-49b2-8615-5b48d84edd66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.962595 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960443    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6ac2accf-aafd-4d06-8d73-230036eea5a4-ovnkube-config\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.962595 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960595    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdn6f\" (UniqueName: \"kubernetes.io/projected/05f049d3-8c6b-4155-8586-be26e91c5bb9-kube-api-access-qdn6f\") pod \"node-resolver-8kqdg\" (UID: \"05f049d3-8c6b-4155-8586-be26e91c5bb9\") " pod="openshift-dns/node-resolver-8kqdg"
Jul 27 18:31:32.962595 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960622    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be23ce78-126d-46a9-95d2-40d68c20f6f2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.962595 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960649    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-var-lib-openvswitch\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.962595 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960657    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-system-cni-dir\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.962595 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960686    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.962595 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960686    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be23ce78-126d-46a9-95d2-40d68c20f6f2-system-cni-dir\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.962595 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960719    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmn4b\" (UniqueName: \"kubernetes.io/projected/6ac2accf-aafd-4d06-8d73-230036eea5a4-kube-api-access-vmn4b\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.962595 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960728    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be23ce78-126d-46a9-95d2-40d68c20f6f2-system-cni-dir\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.962595 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960739    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b54bfb21-6fdf-49b2-8615-5b48d84edd66-registration-dir\") pod \"aws-ebs-csi-driver-node-q5cgq\" (UID: \"b54bfb21-6fdf-49b2-8615-5b48d84edd66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.962595 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960757    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f74b3920-3180-47a1-9895-d7f14e32d952-iptables-alerter-script\") pod \"iptables-alerter-nphwb\" (UID: \"f74b3920-3180-47a1-9895-d7f14e32d952\") " pod="openshift-network-operator/iptables-alerter-nphwb"
Jul 27 18:31:32.962595 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960774    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-os-release\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.962595 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960791    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-cni-binary-copy\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.962595 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960811    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-log-socket\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.962595 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960827    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b54bfb21-6fdf-49b2-8615-5b48d84edd66-etc-selinux\") pod \"aws-ebs-csi-driver-node-q5cgq\" (UID: \"b54bfb21-6fdf-49b2-8615-5b48d84edd66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.963183 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960844    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be23ce78-126d-46a9-95d2-40d68c20f6f2-cnibin\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.963183 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960862    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/be23ce78-126d-46a9-95d2-40d68c20f6f2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.963183 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960879    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/914d3bcb-4f6c-4153-aec8-e4547ff69068-dbus\") pod \"global-pull-secret-syncer-rmf9d\" (UID: \"914d3bcb-4f6c-4153-aec8-e4547ff69068\") " pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:32.963183 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960896    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs\") pod \"network-metrics-daemon-mhtmr\" (UID: \"88f67acb-91af-4850-bb4d-81ff499bc672\") " pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:32.963183 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960912    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-cnibin\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.963183 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960928    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a56a2199-4863-403e-bc3e-b601728e5406-konnectivity-ca\") pod \"konnectivity-agent-scl2m\" (UID: \"a56a2199-4863-403e-bc3e-b601728e5406\") " pod="kube-system/konnectivity-agent-scl2m"
Jul 27 18:31:32.963183 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960966    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-run-ovn\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.963183 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960982    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-multus-socket-dir-parent\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.963183 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961000    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-hostroot\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.963183 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961023    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-multus-daemon-config\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.963183 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961049    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-run-openvswitch\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.963183 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961077    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret\") pod \"global-pull-secret-syncer-rmf9d\" (UID: \"914d3bcb-4f6c-4153-aec8-e4547ff69068\") " pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:32.963183 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961087    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be23ce78-126d-46a9-95d2-40d68c20f6f2-cni-binary-copy\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.963183 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961096    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05f049d3-8c6b-4155-8586-be26e91c5bb9-tmp-dir\") pod \"node-resolver-8kqdg\" (UID: \"05f049d3-8c6b-4155-8586-be26e91c5bb9\") " pod="openshift-dns/node-resolver-8kqdg"
Jul 27 18:31:32.963183 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961104    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-host-var-lib-kubelet\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.963183 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961141    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/be23ce78-126d-46a9-95d2-40d68c20f6f2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.963183 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961152    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-host-var-lib-kubelet\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.963726 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961176    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-os-release\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.963726 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961229    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-log-socket\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.963726 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961275    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-multus-socket-dir-parent\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.963726 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961365    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f74b3920-3180-47a1-9895-d7f14e32d952-iptables-alerter-script\") pod \"iptables-alerter-nphwb\" (UID: \"f74b3920-3180-47a1-9895-d7f14e32d952\") " pod="openshift-network-operator/iptables-alerter-nphwb"
Jul 27 18:31:32.963726 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961386    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b54bfb21-6fdf-49b2-8615-5b48d84edd66-etc-selinux\") pod \"aws-ebs-csi-driver-node-q5cgq\" (UID: \"b54bfb21-6fdf-49b2-8615-5b48d84edd66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.963726 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961431    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b54bfb21-6fdf-49b2-8615-5b48d84edd66-sys-fs\") pod \"aws-ebs-csi-driver-node-q5cgq\" (UID: \"b54bfb21-6fdf-49b2-8615-5b48d84edd66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.963726 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.960626    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/05f049d3-8c6b-4155-8586-be26e91c5bb9-hosts-file\") pod \"node-resolver-8kqdg\" (UID: \"05f049d3-8c6b-4155-8586-be26e91c5bb9\") " pod="openshift-dns/node-resolver-8kqdg"
Jul 27 18:31:32.963726 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961483    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f74b3920-3180-47a1-9895-d7f14e32d952-host-slash\") pod \"iptables-alerter-nphwb\" (UID: \"f74b3920-3180-47a1-9895-d7f14e32d952\") " pod="openshift-network-operator/iptables-alerter-nphwb"
Jul 27 18:31:32.963726 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961518    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b54bfb21-6fdf-49b2-8615-5b48d84edd66-registration-dir\") pod \"aws-ebs-csi-driver-node-q5cgq\" (UID: \"b54bfb21-6fdf-49b2-8615-5b48d84edd66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.963726 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961542    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be23ce78-126d-46a9-95d2-40d68c20f6f2-cnibin\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.963726 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961586    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be23ce78-126d-46a9-95d2-40d68c20f6f2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.963726 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961660    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-var-lib-openvswitch\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.963726 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961715    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-run-openvswitch\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.963726 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961738    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-multus-daemon-config\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.963726 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:32.961838    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 18:31:32.963726 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:32.961928    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret podName:914d3bcb-4f6c-4153-aec8-e4547ff69068 nodeName:}" failed. No retries permitted until 2026-07-27 18:31:33.461892396 +0000 UTC m=+2.012186009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret") pod "global-pull-secret-syncer-rmf9d" (UID: "914d3bcb-4f6c-4153-aec8-e4547ff69068") : object "kube-system"/"original-pull-secret" not registered
Jul 27 18:31:32.963726 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.961938    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/be23ce78-126d-46a9-95d2-40d68c20f6f2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.964252 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.962006    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-hostroot\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.964252 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.962059    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b54bfb21-6fdf-49b2-8615-5b48d84edd66-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q5cgq\" (UID: \"b54bfb21-6fdf-49b2-8615-5b48d84edd66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.964252 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:32.962203    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 18:31:32.964252 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:32.962265    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs podName:88f67acb-91af-4850-bb4d-81ff499bc672 nodeName:}" failed. No retries permitted until 2026-07-27 18:31:33.462248508 +0000 UTC m=+2.012542124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs") pod "network-metrics-daemon-mhtmr" (UID: "88f67acb-91af-4850-bb4d-81ff499bc672") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 18:31:32.964252 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.962288    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/914d3bcb-4f6c-4153-aec8-e4547ff69068-dbus\") pod \"global-pull-secret-syncer-rmf9d\" (UID: \"914d3bcb-4f6c-4153-aec8-e4547ff69068\") " pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:32.964252 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.962314    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a56a2199-4863-403e-bc3e-b601728e5406-konnectivity-ca\") pod \"konnectivity-agent-scl2m\" (UID: \"a56a2199-4863-403e-bc3e-b601728e5406\") " pod="kube-system/konnectivity-agent-scl2m"
Jul 27 18:31:32.964252 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.962324    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/be23ce78-126d-46a9-95d2-40d68c20f6f2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.964252 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.962352    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-cnibin\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.964252 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.962412    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6ac2accf-aafd-4d06-8d73-230036eea5a4-run-ovn\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.964252 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.962472    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6ac2accf-aafd-4d06-8d73-230036eea5a4-ovn-node-metrics-cert\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.964252 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.962773    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-cni-binary-copy\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.964623 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.964584    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a56a2199-4863-403e-bc3e-b601728e5406-agent-certs\") pod \"konnectivity-agent-scl2m\" (UID: \"a56a2199-4863-403e-bc3e-b601728e5406\") " pod="kube-system/konnectivity-agent-scl2m"
Jul 27 18:31:32.968507 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.968483    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brczk\" (UniqueName: \"kubernetes.io/projected/88f67acb-91af-4850-bb4d-81ff499bc672-kube-api-access-brczk\") pod \"network-metrics-daemon-mhtmr\" (UID: \"88f67acb-91af-4850-bb4d-81ff499bc672\") " pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:32.970085 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.970067    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bklv\" (UniqueName: \"kubernetes.io/projected/5cf7f534-634d-4eab-9577-7d03c7cdb9cb-kube-api-access-4bklv\") pod \"multus-9gvlm\" (UID: \"5cf7f534-634d-4eab-9577-7d03c7cdb9cb\") " pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:32.971384 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:32.971367    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 18:31:32.971430 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:32.971391    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 18:31:32.971430 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:32.971407    2562 projected.go:196] Error preparing data for projected volume kube-api-access-mrbmb for pod openshift-network-diagnostics/network-check-target-zqgfw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 18:31:32.971501 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:32.971483    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb podName:8e6d520b-feb0-4b85-921f-7fe3f1ab3549 nodeName:}" failed. No retries permitted until 2026-07-27 18:31:33.471464534 +0000 UTC m=+2.021758173 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mrbmb" (UniqueName: "kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb") pod "network-check-target-zqgfw" (UID: "8e6d520b-feb0-4b85-921f-7fe3f1ab3549") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 18:31:32.973879 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.973855    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgmtn\" (UniqueName: \"kubernetes.io/projected/f74b3920-3180-47a1-9895-d7f14e32d952-kube-api-access-vgmtn\") pod \"iptables-alerter-nphwb\" (UID: \"f74b3920-3180-47a1-9895-d7f14e32d952\") " pod="openshift-network-operator/iptables-alerter-nphwb"
Jul 27 18:31:32.973991 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.973972    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6gsq\" (UniqueName: \"kubernetes.io/projected/be23ce78-126d-46a9-95d2-40d68c20f6f2-kube-api-access-q6gsq\") pod \"multus-additional-cni-plugins-q9k9p\" (UID: \"be23ce78-126d-46a9-95d2-40d68c20f6f2\") " pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:32.974417 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.974382    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mxhd\" (UniqueName: \"kubernetes.io/projected/b54bfb21-6fdf-49b2-8615-5b48d84edd66-kube-api-access-6mxhd\") pod \"aws-ebs-csi-driver-node-q5cgq\" (UID: \"b54bfb21-6fdf-49b2-8615-5b48d84edd66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:32.975190 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.975172    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmn4b\" (UniqueName: \"kubernetes.io/projected/6ac2accf-aafd-4d06-8d73-230036eea5a4-kube-api-access-vmn4b\") pod \"ovnkube-node-5wkjf\" (UID: \"6ac2accf-aafd-4d06-8d73-230036eea5a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:32.976210 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:32.976189    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdn6f\" (UniqueName: \"kubernetes.io/projected/05f049d3-8c6b-4155-8586-be26e91c5bb9-kube-api-access-qdn6f\") pod \"node-resolver-8kqdg\" (UID: \"05f049d3-8c6b-4155-8586-be26e91c5bb9\") " pod="openshift-dns/node-resolver-8kqdg"
Jul 27 18:31:33.074363 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.074263    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qhfd6"
Jul 27 18:31:33.087648 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.087594    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jz5n4"
Jul 27 18:31:33.095714 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.095688    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nphwb"
Jul 27 18:31:33.107665 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.107640    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8kqdg"
Jul 27 18:31:33.123055 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.123023    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9gvlm"
Jul 27 18:31:33.145126 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.145103    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q9k9p"
Jul 27 18:31:33.172843 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.172803    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:31:33.187761 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.187732    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-scl2m"
Jul 27 18:31:33.192477 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.192457    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq"
Jul 27 18:31:33.207313 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.207291    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 18:31:33.225811 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:33.225778    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod129a2108b5a3803a480ffe6e1a6b9b55.slice/crio-666188d4344a849e802445e5075f49864485b63350f085ea1a7a06022a41b8c5 WatchSource:0}: Error finding container 666188d4344a849e802445e5075f49864485b63350f085ea1a7a06022a41b8c5: Status 404 returned error can't find the container with id 666188d4344a849e802445e5075f49864485b63350f085ea1a7a06022a41b8c5
Jul 27 18:31:33.232489 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.232472    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 18:31:33.394166 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:33.394126    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a59ab1fcabaf70a08c93371f33a8e3a.slice/crio-e7603e0814ce35d96a85b4e13a60bface0050388fca0ed930acdf6dab47ab5b6 WatchSource:0}: Error finding container e7603e0814ce35d96a85b4e13a60bface0050388fca0ed930acdf6dab47ab5b6: Status 404 returned error can't find the container with id e7603e0814ce35d96a85b4e13a60bface0050388fca0ed930acdf6dab47ab5b6
Jul 27 18:31:33.449253 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:33.449219    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cf7f534_634d_4eab_9577_7d03c7cdb9cb.slice/crio-03237caaf8000ce0efafaf66bd59d35a4f4624469357138b4649521b12d581f6 WatchSource:0}: Error finding container 03237caaf8000ce0efafaf66bd59d35a4f4624469357138b4649521b12d581f6: Status 404 returned error can't find the container with id 03237caaf8000ce0efafaf66bd59d35a4f4624469357138b4649521b12d581f6
Jul 27 18:31:33.464841 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.464812    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs\") pod \"network-metrics-daemon-mhtmr\" (UID: \"88f67acb-91af-4850-bb4d-81ff499bc672\") " pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:33.465000 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.464849    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret\") pod \"global-pull-secret-syncer-rmf9d\" (UID: \"914d3bcb-4f6c-4153-aec8-e4547ff69068\") " pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:33.465000 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:33.464978    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 18:31:33.465094 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:33.465029    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 18:31:33.465094 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:33.465047    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret podName:914d3bcb-4f6c-4153-aec8-e4547ff69068 nodeName:}" failed. No retries permitted until 2026-07-27 18:31:34.465027679 +0000 UTC m=+3.015321294 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret") pod "global-pull-secret-syncer-rmf9d" (UID: "914d3bcb-4f6c-4153-aec8-e4547ff69068") : object "kube-system"/"original-pull-secret" not registered
Jul 27 18:31:33.465094 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:33.465066    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs podName:88f67acb-91af-4850-bb4d-81ff499bc672 nodeName:}" failed. No retries permitted until 2026-07-27 18:31:34.465055664 +0000 UTC m=+3.015349277 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs") pod "network-metrics-daemon-mhtmr" (UID: "88f67acb-91af-4850-bb4d-81ff499bc672") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 18:31:33.512730 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:33.512685    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe23ce78_126d_46a9_95d2_40d68c20f6f2.slice/crio-6e14951fc3cf504ec91e19954b68b9aca819a7067d5ae19c0b81ff7e8fac9ba1 WatchSource:0}: Error finding container 6e14951fc3cf504ec91e19954b68b9aca819a7067d5ae19c0b81ff7e8fac9ba1: Status 404 returned error can't find the container with id 6e14951fc3cf504ec91e19954b68b9aca819a7067d5ae19c0b81ff7e8fac9ba1
Jul 27 18:31:33.536069 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:33.536035    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05f049d3_8c6b_4155_8586_be26e91c5bb9.slice/crio-22928317c2617b83bf4f71b5a68acfe4ff8bca26bf98eddb0f371161aa5d70d0 WatchSource:0}: Error finding container 22928317c2617b83bf4f71b5a68acfe4ff8bca26bf98eddb0f371161aa5d70d0: Status 404 returned error can't find the container with id 22928317c2617b83bf4f71b5a68acfe4ff8bca26bf98eddb0f371161aa5d70d0
Jul 27 18:31:33.536678 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:33.536645    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod846631a2_6652_4ae8_9d36_d60069b1321c.slice/crio-005ceffb80fe1f341dc9c8a50fe66aaa1d224b777d0baccb6631d8fa71a7432f WatchSource:0}: Error finding container 005ceffb80fe1f341dc9c8a50fe66aaa1d224b777d0baccb6631d8fa71a7432f: Status 404 returned error can't find the container with id 005ceffb80fe1f341dc9c8a50fe66aaa1d224b777d0baccb6631d8fa71a7432f
Jul 27 18:31:33.566219 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.566178    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrbmb\" (UniqueName: \"kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb\") pod \"network-check-target-zqgfw\" (UID: \"8e6d520b-feb0-4b85-921f-7fe3f1ab3549\") " pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:33.566416 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:33.566372    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 18:31:33.566416 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:33.566398    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 18:31:33.566416 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:33.566411    2562 projected.go:196] Error preparing data for projected volume kube-api-access-mrbmb for pod openshift-network-diagnostics/network-check-target-zqgfw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 18:31:33.566590 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:33.566482    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb podName:8e6d520b-feb0-4b85-921f-7fe3f1ab3549 nodeName:}" failed. No retries permitted until 2026-07-27 18:31:34.566460873 +0000 UTC m=+3.116754508 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mrbmb" (UniqueName: "kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb") pod "network-check-target-zqgfw" (UID: "8e6d520b-feb0-4b85-921f-7fe3f1ab3549") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 18:31:33.589680 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:33.589642    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ac2accf_aafd_4d06_8d73_230036eea5a4.slice/crio-09e483f73111c9c1ecb3bda36fa4d21bc059f86915c544a80f84b055ae92c2f1 WatchSource:0}: Error finding container 09e483f73111c9c1ecb3bda36fa4d21bc059f86915c544a80f84b055ae92c2f1: Status 404 returned error can't find the container with id 09e483f73111c9c1ecb3bda36fa4d21bc059f86915c544a80f84b055ae92c2f1
Jul 27 18:31:33.597348 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:33.597318    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda56a2199_4863_403e_bc3e_b601728e5406.slice/crio-d5c7d1552ddd8c85dd951d2bd3622825183f20fc3c303c5806ada88cf7072da6 WatchSource:0}: Error finding container d5c7d1552ddd8c85dd951d2bd3622825183f20fc3c303c5806ada88cf7072da6: Status 404 returned error can't find the container with id d5c7d1552ddd8c85dd951d2bd3622825183f20fc3c303c5806ada88cf7072da6
Jul 27 18:31:33.724540 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:33.724498    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf74b3920_3180_47a1_9895_d7f14e32d952.slice/crio-ee4a9d5eb0a50b4373a6d517a49a4fc904244e19f6b4f1860b05109f019c4241 WatchSource:0}: Error finding container ee4a9d5eb0a50b4373a6d517a49a4fc904244e19f6b4f1860b05109f019c4241: Status 404 returned error can't find the container with id ee4a9d5eb0a50b4373a6d517a49a4fc904244e19f6b4f1860b05109f019c4241
Jul 27 18:31:33.803269 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.803185    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 18:26:33 +0000 UTC" deadline="2028-01-01 22:32:57.984312335 +0000 UTC"
Jul 27 18:31:33.803269 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.803258    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12556h1m24.181057352s"
Jul 27 18:31:33.849879 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.849843    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 18:31:33.931875 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.931663    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:33.931875 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:33.931818    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zqgfw" podUID="8e6d520b-feb0-4b85-921f-7fe3f1ab3549"
Jul 27 18:31:33.938011 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.937959    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf" event={"ID":"6ac2accf-aafd-4d06-8d73-230036eea5a4","Type":"ContainerStarted","Data":"09e483f73111c9c1ecb3bda36fa4d21bc059f86915c544a80f84b055ae92c2f1"}
Jul 27 18:31:33.939571 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.939535    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qhfd6" event={"ID":"846631a2-6652-4ae8-9d36-d60069b1321c","Type":"ContainerStarted","Data":"005ceffb80fe1f341dc9c8a50fe66aaa1d224b777d0baccb6631d8fa71a7432f"}
Jul 27 18:31:33.941301 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.941238    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-186.ec2.internal" event={"ID":"1a59ab1fcabaf70a08c93371f33a8e3a","Type":"ContainerStarted","Data":"e7603e0814ce35d96a85b4e13a60bface0050388fca0ed930acdf6dab47ab5b6"}
Jul 27 18:31:33.943052 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.943022    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-186.ec2.internal" event={"ID":"129a2108b5a3803a480ffe6e1a6b9b55","Type":"ContainerStarted","Data":"666188d4344a849e802445e5075f49864485b63350f085ea1a7a06022a41b8c5"}
Jul 27 18:31:33.944831 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.944676    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nphwb" event={"ID":"f74b3920-3180-47a1-9895-d7f14e32d952","Type":"ContainerStarted","Data":"ee4a9d5eb0a50b4373a6d517a49a4fc904244e19f6b4f1860b05109f019c4241"}
Jul 27 18:31:33.947827 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.947761    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8kqdg" event={"ID":"05f049d3-8c6b-4155-8586-be26e91c5bb9","Type":"ContainerStarted","Data":"22928317c2617b83bf4f71b5a68acfe4ff8bca26bf98eddb0f371161aa5d70d0"}
Jul 27 18:31:33.949476 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.949442    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q9k9p" event={"ID":"be23ce78-126d-46a9-95d2-40d68c20f6f2","Type":"ContainerStarted","Data":"6e14951fc3cf504ec91e19954b68b9aca819a7067d5ae19c0b81ff7e8fac9ba1"}
Jul 27 18:31:33.954487 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.954242    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9gvlm" event={"ID":"5cf7f534-634d-4eab-9577-7d03c7cdb9cb","Type":"ContainerStarted","Data":"03237caaf8000ce0efafaf66bd59d35a4f4624469357138b4649521b12d581f6"}
Jul 27 18:31:33.955856 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:33.955830    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-scl2m" event={"ID":"a56a2199-4863-403e-bc3e-b601728e5406","Type":"ContainerStarted","Data":"d5c7d1552ddd8c85dd951d2bd3622825183f20fc3c303c5806ada88cf7072da6"}
Jul 27 18:31:34.475101 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:34.475051    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs\") pod \"network-metrics-daemon-mhtmr\" (UID: \"88f67acb-91af-4850-bb4d-81ff499bc672\") " pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:34.475317 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:34.475114    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret\") pod \"global-pull-secret-syncer-rmf9d\" (UID: \"914d3bcb-4f6c-4153-aec8-e4547ff69068\") " pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:34.475393 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:34.475313    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 18:31:34.475393 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:34.475381    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret podName:914d3bcb-4f6c-4153-aec8-e4547ff69068 nodeName:}" failed. No retries permitted until 2026-07-27 18:31:36.475361173 +0000 UTC m=+5.025654788 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret") pod "global-pull-secret-syncer-rmf9d" (UID: "914d3bcb-4f6c-4153-aec8-e4547ff69068") : object "kube-system"/"original-pull-secret" not registered
Jul 27 18:31:34.475875 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:34.475854    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 18:31:34.475998 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:34.475919    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs podName:88f67acb-91af-4850-bb4d-81ff499bc672 nodeName:}" failed. No retries permitted until 2026-07-27 18:31:36.475900197 +0000 UTC m=+5.026193813 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs") pod "network-metrics-daemon-mhtmr" (UID: "88f67acb-91af-4850-bb4d-81ff499bc672") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 18:31:34.576489 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:34.576448    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrbmb\" (UniqueName: \"kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb\") pod \"network-check-target-zqgfw\" (UID: \"8e6d520b-feb0-4b85-921f-7fe3f1ab3549\") " pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:34.576701 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:34.576679    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 18:31:34.576774 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:34.576710    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 18:31:34.576774 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:34.576739    2562 projected.go:196] Error preparing data for projected volume kube-api-access-mrbmb for pod openshift-network-diagnostics/network-check-target-zqgfw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 18:31:34.576899 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:34.576805    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb podName:8e6d520b-feb0-4b85-921f-7fe3f1ab3549 nodeName:}" failed. No retries permitted until 2026-07-27 18:31:36.576784213 +0000 UTC m=+5.127077831 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mrbmb" (UniqueName: "kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb") pod "network-check-target-zqgfw" (UID: "8e6d520b-feb0-4b85-921f-7fe3f1ab3549") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 18:31:34.703084 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:31:34.703045    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb54bfb21_6fdf_49b2_8615_5b48d84edd66.slice/crio-b60f590d77895a2c87b991eceb0fff13636fd615262d1033a5e8b89f2d8c986c WatchSource:0}: Error finding container b60f590d77895a2c87b991eceb0fff13636fd615262d1033a5e8b89f2d8c986c: Status 404 returned error can't find the container with id b60f590d77895a2c87b991eceb0fff13636fd615262d1033a5e8b89f2d8c986c
Jul 27 18:31:34.803523 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:34.803368    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 18:26:33 +0000 UTC" deadline="2028-01-05 10:11:23.345377343 +0000 UTC"
Jul 27 18:31:34.803523 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:34.803411    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12639h39m48.541971285s"
Jul 27 18:31:34.931898 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:34.931177    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:34.931898 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:34.931337    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mhtmr" podUID="88f67acb-91af-4850-bb4d-81ff499bc672"
Jul 27 18:31:34.931898 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:34.931751    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:34.931898 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:34.931850    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmf9d" podUID="914d3bcb-4f6c-4153-aec8-e4547ff69068"
Jul 27 18:31:34.963620 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:34.963518    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq" event={"ID":"b54bfb21-6fdf-49b2-8615-5b48d84edd66","Type":"ContainerStarted","Data":"b60f590d77895a2c87b991eceb0fff13636fd615262d1033a5e8b89f2d8c986c"}
Jul 27 18:31:34.968264 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:34.968184    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jz5n4" event={"ID":"2efd4785-cf26-4815-86af-86dd8dbd5108","Type":"ContainerStarted","Data":"53a7774fc073987abdcc703320662507e11274fee8e2fb7e11c5260d7b65b98d"}
Jul 27 18:31:35.786254 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:35.786218    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 18:31:35.934092 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:35.934056    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:35.934579 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:35.934200    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zqgfw" podUID="8e6d520b-feb0-4b85-921f-7fe3f1ab3549"
Jul 27 18:31:36.492520 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:36.492475    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs\") pod \"network-metrics-daemon-mhtmr\" (UID: \"88f67acb-91af-4850-bb4d-81ff499bc672\") " pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:36.492733 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:36.492538    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret\") pod \"global-pull-secret-syncer-rmf9d\" (UID: \"914d3bcb-4f6c-4153-aec8-e4547ff69068\") " pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:36.492921 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:36.492861    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 18:31:36.493036 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:36.492960    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret podName:914d3bcb-4f6c-4153-aec8-e4547ff69068 nodeName:}" failed. No retries permitted until 2026-07-27 18:31:40.492938864 +0000 UTC m=+9.043232494 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret") pod "global-pull-secret-syncer-rmf9d" (UID: "914d3bcb-4f6c-4153-aec8-e4547ff69068") : object "kube-system"/"original-pull-secret" not registered
Jul 27 18:31:36.495902 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:36.493420    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 18:31:36.495902 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:36.493493    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs podName:88f67acb-91af-4850-bb4d-81ff499bc672 nodeName:}" failed. No retries permitted until 2026-07-27 18:31:40.493474993 +0000 UTC m=+9.043768605 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs") pod "network-metrics-daemon-mhtmr" (UID: "88f67acb-91af-4850-bb4d-81ff499bc672") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 18:31:36.593593 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:36.593534    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrbmb\" (UniqueName: \"kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb\") pod \"network-check-target-zqgfw\" (UID: \"8e6d520b-feb0-4b85-921f-7fe3f1ab3549\") " pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:36.593796 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:36.593769    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 18:31:36.593880 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:36.593800    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 18:31:36.593880 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:36.593817    2562 projected.go:196] Error preparing data for projected volume kube-api-access-mrbmb for pod openshift-network-diagnostics/network-check-target-zqgfw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 18:31:36.594047 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:36.593976    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb podName:8e6d520b-feb0-4b85-921f-7fe3f1ab3549 nodeName:}" failed. No retries permitted until 2026-07-27 18:31:40.593876406 +0000 UTC m=+9.144170044 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mrbmb" (UniqueName: "kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb") pod "network-check-target-zqgfw" (UID: "8e6d520b-feb0-4b85-921f-7fe3f1ab3549") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 18:31:36.931882 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:36.931263    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:36.931882 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:36.931314    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:36.931882 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:36.931438    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mhtmr" podUID="88f67acb-91af-4850-bb4d-81ff499bc672"
Jul 27 18:31:36.931882 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:36.931559    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmf9d" podUID="914d3bcb-4f6c-4153-aec8-e4547ff69068"
Jul 27 18:31:37.933621 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:37.933582    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:37.934155 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:37.933733    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zqgfw" podUID="8e6d520b-feb0-4b85-921f-7fe3f1ab3549"
Jul 27 18:31:38.931182 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:38.931146    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:38.931400 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:38.931307    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mhtmr" podUID="88f67acb-91af-4850-bb4d-81ff499bc672"
Jul 27 18:31:38.931781 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:38.931756    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:38.931914 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:38.931890    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmf9d" podUID="914d3bcb-4f6c-4153-aec8-e4547ff69068"
Jul 27 18:31:39.931113 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:39.930895    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:39.931113 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:39.931059    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zqgfw" podUID="8e6d520b-feb0-4b85-921f-7fe3f1ab3549"
Jul 27 18:31:40.528371 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:40.528334    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs\") pod \"network-metrics-daemon-mhtmr\" (UID: \"88f67acb-91af-4850-bb4d-81ff499bc672\") " pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:40.528371 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:40.528378    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret\") pod \"global-pull-secret-syncer-rmf9d\" (UID: \"914d3bcb-4f6c-4153-aec8-e4547ff69068\") " pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:40.528638 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:40.528516    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 18:31:40.528638 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:40.528566    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret podName:914d3bcb-4f6c-4153-aec8-e4547ff69068 nodeName:}" failed. No retries permitted until 2026-07-27 18:31:48.528551984 +0000 UTC m=+17.078845597 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret") pod "global-pull-secret-syncer-rmf9d" (UID: "914d3bcb-4f6c-4153-aec8-e4547ff69068") : object "kube-system"/"original-pull-secret" not registered
Jul 27 18:31:40.528933 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:40.528898    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 18:31:40.529036 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:40.528966    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs podName:88f67acb-91af-4850-bb4d-81ff499bc672 nodeName:}" failed. No retries permitted until 2026-07-27 18:31:48.528952323 +0000 UTC m=+17.079245952 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs") pod "network-metrics-daemon-mhtmr" (UID: "88f67acb-91af-4850-bb4d-81ff499bc672") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 18:31:40.629480 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:40.629435    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrbmb\" (UniqueName: \"kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb\") pod \"network-check-target-zqgfw\" (UID: \"8e6d520b-feb0-4b85-921f-7fe3f1ab3549\") " pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:40.629672 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:40.629633    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 18:31:40.629672 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:40.629658    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 18:31:40.629672 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:40.629669    2562 projected.go:196] Error preparing data for projected volume kube-api-access-mrbmb for pod openshift-network-diagnostics/network-check-target-zqgfw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 18:31:40.629807 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:40.629720    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb podName:8e6d520b-feb0-4b85-921f-7fe3f1ab3549 nodeName:}" failed. No retries permitted until 2026-07-27 18:31:48.629703831 +0000 UTC m=+17.179997461 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mrbmb" (UniqueName: "kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb") pod "network-check-target-zqgfw" (UID: "8e6d520b-feb0-4b85-921f-7fe3f1ab3549") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 18:31:40.931211 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:40.931117    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:40.931729 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:40.931117    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:40.931729 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:40.931280    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmf9d" podUID="914d3bcb-4f6c-4153-aec8-e4547ff69068"
Jul 27 18:31:40.931729 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:40.931321    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mhtmr" podUID="88f67acb-91af-4850-bb4d-81ff499bc672"
Jul 27 18:31:41.932606 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:41.932566    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:41.933136 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:41.932690    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zqgfw" podUID="8e6d520b-feb0-4b85-921f-7fe3f1ab3549"
Jul 27 18:31:42.931308 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:42.931220    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:42.931308 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:42.931257    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:42.931598 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:42.931366    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmf9d" podUID="914d3bcb-4f6c-4153-aec8-e4547ff69068"
Jul 27 18:31:42.931893 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:42.931833    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mhtmr" podUID="88f67acb-91af-4850-bb4d-81ff499bc672"
Jul 27 18:31:43.931330 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:43.931295    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:43.931759 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:43.931435    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zqgfw" podUID="8e6d520b-feb0-4b85-921f-7fe3f1ab3549"
Jul 27 18:31:44.931413 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:44.931373    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:44.931413 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:44.931414    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:44.931855 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:44.931503    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mhtmr" podUID="88f67acb-91af-4850-bb4d-81ff499bc672"
Jul 27 18:31:44.931855 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:44.931632    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmf9d" podUID="914d3bcb-4f6c-4153-aec8-e4547ff69068"
Jul 27 18:31:45.930697 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:45.930659    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:45.930900 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:45.930825    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zqgfw" podUID="8e6d520b-feb0-4b85-921f-7fe3f1ab3549"
Jul 27 18:31:46.931104 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:46.931010    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:46.931104 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:46.931060    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:46.931676 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:46.931149    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mhtmr" podUID="88f67acb-91af-4850-bb4d-81ff499bc672"
Jul 27 18:31:46.931676 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:46.931233    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmf9d" podUID="914d3bcb-4f6c-4153-aec8-e4547ff69068"
Jul 27 18:31:47.931284 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:47.931242    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:47.931702 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:47.931369    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zqgfw" podUID="8e6d520b-feb0-4b85-921f-7fe3f1ab3549"
Jul 27 18:31:48.588580 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:48.588538    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs\") pod \"network-metrics-daemon-mhtmr\" (UID: \"88f67acb-91af-4850-bb4d-81ff499bc672\") " pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:48.588580 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:48.588582    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret\") pod \"global-pull-secret-syncer-rmf9d\" (UID: \"914d3bcb-4f6c-4153-aec8-e4547ff69068\") " pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:48.588836 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:48.588702    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 18:31:48.588836 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:48.588701    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 18:31:48.588836 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:48.588764    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs podName:88f67acb-91af-4850-bb4d-81ff499bc672 nodeName:}" failed. No retries permitted until 2026-07-27 18:32:04.588744629 +0000 UTC m=+33.139038242 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs") pod "network-metrics-daemon-mhtmr" (UID: "88f67acb-91af-4850-bb4d-81ff499bc672") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 18:31:48.588836 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:48.588780    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret podName:914d3bcb-4f6c-4153-aec8-e4547ff69068 nodeName:}" failed. No retries permitted until 2026-07-27 18:32:04.588772479 +0000 UTC m=+33.139066090 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret") pod "global-pull-secret-syncer-rmf9d" (UID: "914d3bcb-4f6c-4153-aec8-e4547ff69068") : object "kube-system"/"original-pull-secret" not registered
Jul 27 18:31:48.689676 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:48.689637    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrbmb\" (UniqueName: \"kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb\") pod \"network-check-target-zqgfw\" (UID: \"8e6d520b-feb0-4b85-921f-7fe3f1ab3549\") " pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:48.689851 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:48.689811    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 18:31:48.689851 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:48.689841    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 18:31:48.689982 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:48.689853    2562 projected.go:196] Error preparing data for projected volume kube-api-access-mrbmb for pod openshift-network-diagnostics/network-check-target-zqgfw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 18:31:48.689982 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:48.689934    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb podName:8e6d520b-feb0-4b85-921f-7fe3f1ab3549 nodeName:}" failed. No retries permitted until 2026-07-27 18:32:04.689914969 +0000 UTC m=+33.240208582 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mrbmb" (UniqueName: "kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb") pod "network-check-target-zqgfw" (UID: "8e6d520b-feb0-4b85-921f-7fe3f1ab3549") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 18:31:48.930910 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:48.930815    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:48.930910 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:48.930849    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:48.931143 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:48.930962    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mhtmr" podUID="88f67acb-91af-4850-bb4d-81ff499bc672"
Jul 27 18:31:48.931143 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:48.931091    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmf9d" podUID="914d3bcb-4f6c-4153-aec8-e4547ff69068"
Jul 27 18:31:49.930926 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:49.930885    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:49.931367 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:49.931026    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zqgfw" podUID="8e6d520b-feb0-4b85-921f-7fe3f1ab3549"
Jul 27 18:31:50.931495 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:50.931450    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:50.931495 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:50.931494    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:50.931947 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:50.931597    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmf9d" podUID="914d3bcb-4f6c-4153-aec8-e4547ff69068"
Jul 27 18:31:50.931947 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:50.931703    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mhtmr" podUID="88f67acb-91af-4850-bb4d-81ff499bc672"
Jul 27 18:31:51.931579 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:51.931540    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:51.932069 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:51.931661    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zqgfw" podUID="8e6d520b-feb0-4b85-921f-7fe3f1ab3549"
Jul 27 18:31:52.931203 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:52.931165    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:52.931384 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:52.931166    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:52.931384 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:52.931319    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mhtmr" podUID="88f67acb-91af-4850-bb4d-81ff499bc672"
Jul 27 18:31:52.931508 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:52.931378    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmf9d" podUID="914d3bcb-4f6c-4153-aec8-e4547ff69068"
Jul 27 18:31:53.931241 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:53.931205    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:53.931771 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:53.931344    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zqgfw" podUID="8e6d520b-feb0-4b85-921f-7fe3f1ab3549"
Jul 27 18:31:54.930955 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:54.930577    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:54.931129 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:54.930968    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mhtmr" podUID="88f67acb-91af-4850-bb4d-81ff499bc672"
Jul 27 18:31:54.931129 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:54.930645    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:54.931129 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:54.931078    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmf9d" podUID="914d3bcb-4f6c-4153-aec8-e4547ff69068"
Jul 27 18:31:55.020968 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.020943    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jz5n4" event={"ID":"2efd4785-cf26-4815-86af-86dd8dbd5108","Type":"ContainerStarted","Data":"5d1bf649f5ecc71b4f9bd1314f747628a01df2a229e2d4ac70a5e6984aacd7c7"}
Jul 27 18:31:55.022438 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.022411    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-scl2m" event={"ID":"a56a2199-4863-403e-bc3e-b601728e5406","Type":"ContainerStarted","Data":"046af1fd87af75856e442c5fe239d80a6222475d139121fdc17a3809701947be"}
Jul 27 18:31:55.025081 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.025058    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf" event={"ID":"6ac2accf-aafd-4d06-8d73-230036eea5a4","Type":"ContainerStarted","Data":"b4d489a6eb072e5ee0fded18aa32d29cbd12bfa837bbd6fcdddf71238ac37c9d"}
Jul 27 18:31:55.025171 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.025137    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf" event={"ID":"6ac2accf-aafd-4d06-8d73-230036eea5a4","Type":"ContainerStarted","Data":"8d818e0cfb6f07d8128eed7dfa43bb6c4bcf9c1ef609569c261abae16b4e7ebb"}
Jul 27 18:31:55.025221 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.025188    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf" event={"ID":"6ac2accf-aafd-4d06-8d73-230036eea5a4","Type":"ContainerStarted","Data":"047417403e3091b94ec474b86dd89043955bf0e852d416fa8a6a42923a8fc876"}
Jul 27 18:31:55.025260 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.025238    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf" event={"ID":"6ac2accf-aafd-4d06-8d73-230036eea5a4","Type":"ContainerStarted","Data":"8bc4ba10997db98392624e60ceb3f2bbc02da75481425c9b2b25730b224cb2b9"}
Jul 27 18:31:55.026406 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.026381    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qhfd6" event={"ID":"846631a2-6652-4ae8-9d36-d60069b1321c","Type":"ContainerStarted","Data":"019371b3efd19c4af6718115f62543131ba5b83604362893019960b8f2a9f680"}
Jul 27 18:31:55.027576 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.027553    2562 generic.go:356] "Generic (PLEG): container finished" podID="1a59ab1fcabaf70a08c93371f33a8e3a" containerID="758e7db26611fe908aa473f7a1a15f52db57c9c3763efa29c34cf9f4c636274d" exitCode=0
Jul 27 18:31:55.027641 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.027623    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-186.ec2.internal" event={"ID":"1a59ab1fcabaf70a08c93371f33a8e3a","Type":"ContainerDied","Data":"758e7db26611fe908aa473f7a1a15f52db57c9c3763efa29c34cf9f4c636274d"}
Jul 27 18:31:55.028810 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.028790    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-186.ec2.internal" event={"ID":"129a2108b5a3803a480ffe6e1a6b9b55","Type":"ContainerStarted","Data":"46f8f9856c53953f8357a096ec367680d7df53b5e23838453146eee0a949cc3c"}
Jul 27 18:31:55.030055 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.030025    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8kqdg" event={"ID":"05f049d3-8c6b-4155-8586-be26e91c5bb9","Type":"ContainerStarted","Data":"668349440abf7b1acf6a33754f2936b13349aee1878a285a43a0414c4a58894a"}
Jul 27 18:31:55.031530 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.031506    2562 generic.go:356] "Generic (PLEG): container finished" podID="be23ce78-126d-46a9-95d2-40d68c20f6f2" containerID="5b671a9d7666e74eda0020f2d19e6bb691cc41b8d98e33ba0369aa62876387b1" exitCode=0
Jul 27 18:31:55.031597 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.031582    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q9k9p" event={"ID":"be23ce78-126d-46a9-95d2-40d68c20f6f2","Type":"ContainerDied","Data":"5b671a9d7666e74eda0020f2d19e6bb691cc41b8d98e33ba0369aa62876387b1"}
Jul 27 18:31:55.033144 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.033123    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9gvlm" event={"ID":"5cf7f534-634d-4eab-9577-7d03c7cdb9cb","Type":"ContainerStarted","Data":"4f2ffbe8c620267b4f1f45c91e887306956d98b855f31e19c5904c780ab3e352"}
Jul 27 18:31:55.034684 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.034665    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq" event={"ID":"b54bfb21-6fdf-49b2-8615-5b48d84edd66","Type":"ContainerStarted","Data":"f1b954a5dbacbb7c1d8796012fb20f81011cd4f4b1ee613888cde2d99ac5436a"}
Jul 27 18:31:55.036687 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.036628    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jz5n4" podStartSLOduration=3.152622345 podStartE2EDuration="23.036610348s" totalImagesPullingTime="19.883988003s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 18:31:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:31:55.036128694 +0000 UTC m=+23.586422326" watchObservedRunningTime="2026-07-27 18:31:55.036610348 +0000 UTC m=+23.586903982"
Jul 27 18:31:55.055357 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.055310    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8kqdg" podStartSLOduration=2.104564653 podStartE2EDuration="23.055293839s" totalImagesPullingTime="20.950729186s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 18:31:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:31:55.055044411 +0000 UTC m=+23.605338044" watchObservedRunningTime="2026-07-27 18:31:55.055293839 +0000 UTC m=+23.605587472"
Jul 27 18:31:55.075695 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.075644    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9gvlm" podStartSLOduration=1.991306646 podStartE2EDuration="23.075623962s" totalImagesPullingTime="21.084317316s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 18:31:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:31:55.075555196 +0000 UTC m=+23.625848847" watchObservedRunningTime="2026-07-27 18:31:55.075623962 +0000 UTC m=+23.625917596"
Jul 27 18:31:55.111112 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.111070    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-scl2m" podStartSLOduration=6.010806447 podStartE2EDuration="23.111053949s" totalImagesPullingTime="17.100247502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:31:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:31:55.110840419 +0000 UTC m=+23.661134053" watchObservedRunningTime="2026-07-27 18:31:55.111053949 +0000 UTC m=+23.661347581"
Jul 27 18:31:55.126152 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.126100    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-186.ec2.internal" podStartSLOduration=22.12609004 podStartE2EDuration="22.12609004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 18:31:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:31:55.125710073 +0000 UTC m=+23.676003706" watchObservedRunningTime="2026-07-27 18:31:55.12609004 +0000 UTC m=+23.676383679"
Jul 27 18:31:55.143927 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.143861    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qhfd6" podStartSLOduration=2.193084465 podStartE2EDuration="23.143846888s" totalImagesPullingTime="20.950762423s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 18:31:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:31:55.143628422 +0000 UTC m=+23.693922066" watchObservedRunningTime="2026-07-27 18:31:55.143846888 +0000 UTC m=+23.694140521"
Jul 27 18:31:55.932041 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:55.932008    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:55.932227 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:55.932142    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zqgfw" podUID="8e6d520b-feb0-4b85-921f-7fe3f1ab3549"
Jul 27 18:31:56.039392 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:56.039349    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nphwb" event={"ID":"f74b3920-3180-47a1-9895-d7f14e32d952","Type":"ContainerStarted","Data":"b752d6993420a2e95dd9a805d9342910295f1855f941da9d83602672d02142a6"}
Jul 27 18:31:56.043221 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:56.043190    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf" event={"ID":"6ac2accf-aafd-4d06-8d73-230036eea5a4","Type":"ContainerStarted","Data":"614221e2016afcd69c153e6229ca05f93fce5f3a2ba980063bce9664f77ab639"}
Jul 27 18:31:56.043374 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:56.043274    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf" event={"ID":"6ac2accf-aafd-4d06-8d73-230036eea5a4","Type":"ContainerStarted","Data":"a8eea0d313dd0448e8516cde5eac7f947f4aec56202362cefacb3fbd2197f968"}
Jul 27 18:31:56.074589 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:56.074536    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nphwb" podStartSLOduration=3.326524235 podStartE2EDuration="24.074520674s" totalImagesPullingTime="20.747996439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 18:31:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:31:56.055652954 +0000 UTC m=+24.605946586" watchObservedRunningTime="2026-07-27 18:31:56.074520674 +0000 UTC m=+24.624814306"
Jul 27 18:31:56.484071 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:56.483842    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 27 18:31:56.827165 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:56.827009    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-27T18:31:56.484064233Z","UUID":"049441fe-1837-490d-b757-33e9d82083f9","Handler":null,"Name":"","Endpoint":""}
Jul 27 18:31:56.828795 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:56.828773    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 27 18:31:56.828952 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:56.828814    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 27 18:31:56.931081 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:56.931039    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:56.931081 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:56.931080    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:56.931337 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:56.931192    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mhtmr" podUID="88f67acb-91af-4850-bb4d-81ff499bc672"
Jul 27 18:31:56.931337 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:56.931289    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmf9d" podUID="914d3bcb-4f6c-4153-aec8-e4547ff69068"
Jul 27 18:31:57.048515 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:57.048474    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-186.ec2.internal" event={"ID":"1a59ab1fcabaf70a08c93371f33a8e3a","Type":"ContainerStarted","Data":"526314aad28d94901aa81d8585fc9cf999762997dc4d1b845e99f235aaf14067"}
Jul 27 18:31:57.051569 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:57.051532    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq" event={"ID":"b54bfb21-6fdf-49b2-8615-5b48d84edd66","Type":"ContainerStarted","Data":"f5918b1cc4aab0ff86b45e57bf7acea25f97f33fb00de4cc78d43e2517f70a09"}
Jul 27 18:31:57.064773 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:57.064719    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-186.ec2.internal" podStartSLOduration=24.06470347 podStartE2EDuration="24.06470347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 18:31:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:31:57.064590846 +0000 UTC m=+25.614884499" watchObservedRunningTime="2026-07-27 18:31:57.06470347 +0000 UTC m=+25.614997103"
Jul 27 18:31:57.931235 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:57.931160    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:57.931505 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:57.931278    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zqgfw" podUID="8e6d520b-feb0-4b85-921f-7fe3f1ab3549"
Jul 27 18:31:58.058200 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:58.058169    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf" event={"ID":"6ac2accf-aafd-4d06-8d73-230036eea5a4","Type":"ContainerStarted","Data":"a9f72f58c07fbe339d55f82f994c7b54d073ecddbc9832de999504807d8c73f7"}
Jul 27 18:31:58.930729 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:58.930690    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:31:58.930966 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:58.930737    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:31:58.930966 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:58.930829    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mhtmr" podUID="88f67acb-91af-4850-bb4d-81ff499bc672"
Jul 27 18:31:58.931070 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:58.930998    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmf9d" podUID="914d3bcb-4f6c-4153-aec8-e4547ff69068"
Jul 27 18:31:59.934642 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:31:59.934465    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:31:59.935128 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:31:59.934712    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zqgfw" podUID="8e6d520b-feb0-4b85-921f-7fe3f1ab3549"
Jul 27 18:32:00.010747 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:00.010712    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-scl2m"
Jul 27 18:32:00.011360 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:00.011342    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-scl2m"
Jul 27 18:32:00.066610 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:00.066571    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf" event={"ID":"6ac2accf-aafd-4d06-8d73-230036eea5a4","Type":"ContainerStarted","Data":"f643ec3e42addf3ead118af83379a2a75ed0d16d6f4cc03454b586c87a50540a"}
Jul 27 18:32:00.066834 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:00.066812    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:32:00.069054 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:00.069024    2562 generic.go:356] "Generic (PLEG): container finished" podID="be23ce78-126d-46a9-95d2-40d68c20f6f2" containerID="224cc1d64979dc9781df0a1401995ec142cce0bb3b6af58bed38fa9442006e28" exitCode=0
Jul 27 18:32:00.069205 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:00.069110    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q9k9p" event={"ID":"be23ce78-126d-46a9-95d2-40d68c20f6f2","Type":"ContainerDied","Data":"224cc1d64979dc9781df0a1401995ec142cce0bb3b6af58bed38fa9442006e28"}
Jul 27 18:32:00.071515 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:00.071487    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq" event={"ID":"b54bfb21-6fdf-49b2-8615-5b48d84edd66","Type":"ContainerStarted","Data":"b09388f82161284791d01c9f1294311df9d0ccea044061b38fb39562c5bf8828"}
Jul 27 18:32:00.071683 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:00.071666    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-scl2m"
Jul 27 18:32:00.072208 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:00.072191    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-scl2m"
Jul 27 18:32:00.084699 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:00.084613    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:32:00.090907 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:00.090847    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf" podStartSLOduration=7.152357715 podStartE2EDuration="28.090831467s" totalImagesPullingTime="20.938473752s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 18:31:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:32:00.090200233 +0000 UTC m=+28.640493866" watchObservedRunningTime="2026-07-27 18:32:00.090831467 +0000 UTC m=+28.641125099"
Jul 27 18:32:00.183520 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:00.183471    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5cgq" podStartSLOduration=4.208733597 podStartE2EDuration="28.183455099s" totalImagesPullingTime="23.974721502s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 18:31:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:32:00.183195836 +0000 UTC m=+28.733489469" watchObservedRunningTime="2026-07-27 18:32:00.183455099 +0000 UTC m=+28.733748732"
Jul 27 18:32:00.931445 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:00.931410    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:32:00.931688 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:00.931416    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:32:00.931688 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:00.931527    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmf9d" podUID="914d3bcb-4f6c-4153-aec8-e4547ff69068"
Jul 27 18:32:00.931688 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:00.931603    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mhtmr" podUID="88f67acb-91af-4850-bb4d-81ff499bc672"
Jul 27 18:32:01.074260 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:01.074229    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:32:01.082375 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:01.082240    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:32:01.094474 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:01.094443    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:32:01.650714 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:01.650376    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zqgfw"]
Jul 27 18:32:01.650954 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:01.650748    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:32:01.650954 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:01.650841    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zqgfw" podUID="8e6d520b-feb0-4b85-921f-7fe3f1ab3549"
Jul 27 18:32:01.653845 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:01.653819    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rmf9d"]
Jul 27 18:32:01.653997 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:01.653959    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:32:01.654094 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:01.654072    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmf9d" podUID="914d3bcb-4f6c-4153-aec8-e4547ff69068"
Jul 27 18:32:01.654465 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:01.654443    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mhtmr"]
Jul 27 18:32:01.654577 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:01.654542    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:32:01.654639 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:01.654627    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mhtmr" podUID="88f67acb-91af-4850-bb4d-81ff499bc672"
Jul 27 18:32:02.078577 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:02.078494    2562 generic.go:356] "Generic (PLEG): container finished" podID="be23ce78-126d-46a9-95d2-40d68c20f6f2" containerID="116408a3131546b87a51701c437aaf7edfd89560f5c3a59fbfcb909d53cc9c58" exitCode=0
Jul 27 18:32:02.079144 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:02.078577    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q9k9p" event={"ID":"be23ce78-126d-46a9-95d2-40d68c20f6f2","Type":"ContainerDied","Data":"116408a3131546b87a51701c437aaf7edfd89560f5c3a59fbfcb909d53cc9c58"}
Jul 27 18:32:02.931131 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:02.931094    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:32:02.931299 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:02.931095    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:32:02.931299 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:02.931207    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmf9d" podUID="914d3bcb-4f6c-4153-aec8-e4547ff69068"
Jul 27 18:32:02.931370 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:02.931297    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zqgfw" podUID="8e6d520b-feb0-4b85-921f-7fe3f1ab3549"
Jul 27 18:32:03.931209 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:03.931135    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:32:03.931553 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:03.931256    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mhtmr" podUID="88f67acb-91af-4850-bb4d-81ff499bc672"
Jul 27 18:32:04.612739 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:04.612642    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs\") pod \"network-metrics-daemon-mhtmr\" (UID: \"88f67acb-91af-4850-bb4d-81ff499bc672\") " pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:32:04.612739 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:04.612687    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret\") pod \"global-pull-secret-syncer-rmf9d\" (UID: \"914d3bcb-4f6c-4153-aec8-e4547ff69068\") " pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:32:04.613001 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:04.612804    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 18:32:04.613001 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:04.612804    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 18:32:04.613001 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:04.612890    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs podName:88f67acb-91af-4850-bb4d-81ff499bc672 nodeName:}" failed. No retries permitted until 2026-07-27 18:32:36.612855233 +0000 UTC m=+65.163148844 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs") pod "network-metrics-daemon-mhtmr" (UID: "88f67acb-91af-4850-bb4d-81ff499bc672") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 18:32:04.613001 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:04.612912    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret podName:914d3bcb-4f6c-4153-aec8-e4547ff69068 nodeName:}" failed. No retries permitted until 2026-07-27 18:32:36.612900887 +0000 UTC m=+65.163194498 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret") pod "global-pull-secret-syncer-rmf9d" (UID: "914d3bcb-4f6c-4153-aec8-e4547ff69068") : object "kube-system"/"original-pull-secret" not registered
Jul 27 18:32:04.713348 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:04.713315    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrbmb\" (UniqueName: \"kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb\") pod \"network-check-target-zqgfw\" (UID: \"8e6d520b-feb0-4b85-921f-7fe3f1ab3549\") " pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:32:04.713508 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:04.713476    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 18:32:04.713508 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:04.713497    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 18:32:04.713508 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:04.713506    2562 projected.go:196] Error preparing data for projected volume kube-api-access-mrbmb for pod openshift-network-diagnostics/network-check-target-zqgfw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 18:32:04.713649 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:04.713559    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb podName:8e6d520b-feb0-4b85-921f-7fe3f1ab3549 nodeName:}" failed. No retries permitted until 2026-07-27 18:32:36.713545822 +0000 UTC m=+65.263839433 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mrbmb" (UniqueName: "kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb") pod "network-check-target-zqgfw" (UID: "8e6d520b-feb0-4b85-921f-7fe3f1ab3549") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 18:32:04.930932 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:04.930814    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:32:04.930932 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:04.930841    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:32:04.931152 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:04.930967    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmf9d" podUID="914d3bcb-4f6c-4153-aec8-e4547ff69068"
Jul 27 18:32:04.931152 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:04.931102    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zqgfw" podUID="8e6d520b-feb0-4b85-921f-7fe3f1ab3549"
Jul 27 18:32:05.087662 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:05.087630    2562 generic.go:356] "Generic (PLEG): container finished" podID="be23ce78-126d-46a9-95d2-40d68c20f6f2" containerID="52917a65d057fa7018ce8152624c04aaa3c80f8b8a4510bb6be7e2630fe83232" exitCode=0
Jul 27 18:32:05.088113 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:05.087694    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q9k9p" event={"ID":"be23ce78-126d-46a9-95d2-40d68c20f6f2","Type":"ContainerDied","Data":"52917a65d057fa7018ce8152624c04aaa3c80f8b8a4510bb6be7e2630fe83232"}
Jul 27 18:32:05.934447 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:05.934420    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:32:05.934602 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:05.934537    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mhtmr" podUID="88f67acb-91af-4850-bb4d-81ff499bc672"
Jul 27 18:32:06.930815 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:06.930767    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:32:06.931262 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:06.930779    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:32:06.931262 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:06.930913    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmf9d" podUID="914d3bcb-4f6c-4153-aec8-e4547ff69068"
Jul 27 18:32:06.931262 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:06.930992    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zqgfw" podUID="8e6d520b-feb0-4b85-921f-7fe3f1ab3549"
Jul 27 18:32:07.345232 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.345158    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-186.ec2.internal" event="NodeReady"
Jul 27 18:32:07.345395 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.345311    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 27 18:32:07.392060 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.392026    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wzhvk"]
Jul 27 18:32:07.394116 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.394092    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:32:07.396157 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.396133    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xjh28"]
Jul 27 18:32:07.396909 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.396890    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 27 18:32:07.397450 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.397424    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m8x25\""
Jul 27 18:32:07.397450 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.397441    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 27 18:32:07.397975 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.397952    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xjh28"
Jul 27 18:32:07.400692 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.400669    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 27 18:32:07.400692 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.400686    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 27 18:32:07.400838 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.400716    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bghlc\""
Jul 27 18:32:07.400838 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.400750    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 27 18:32:07.406886 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.406846    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wzhvk"]
Jul 27 18:32:07.410048 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.410028    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xjh28"]
Jul 27 18:32:07.533789 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.533757    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/243eab96-1170-4ea0-98fa-d9cb4a68adb2-cert\") pod \"ingress-canary-xjh28\" (UID: \"243eab96-1170-4ea0-98fa-d9cb4a68adb2\") " pod="openshift-ingress-canary/ingress-canary-xjh28"
Jul 27 18:32:07.533999 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.533814    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2ea9768a-db15-4a87-b27c-1c5e69af52b8-config-volume\") pod \"dns-default-wzhvk\" (UID: \"2ea9768a-db15-4a87-b27c-1c5e69af52b8\") " pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:32:07.533999 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.533927    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls\") pod \"dns-default-wzhvk\" (UID: \"2ea9768a-db15-4a87-b27c-1c5e69af52b8\") " pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:32:07.533999 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.533965    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cbfv\" (UniqueName: \"kubernetes.io/projected/243eab96-1170-4ea0-98fa-d9cb4a68adb2-kube-api-access-6cbfv\") pod \"ingress-canary-xjh28\" (UID: \"243eab96-1170-4ea0-98fa-d9cb4a68adb2\") " pod="openshift-ingress-canary/ingress-canary-xjh28"
Jul 27 18:32:07.533999 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.533993    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4jkv\" (UniqueName: \"kubernetes.io/projected/2ea9768a-db15-4a87-b27c-1c5e69af52b8-kube-api-access-q4jkv\") pod \"dns-default-wzhvk\" (UID: \"2ea9768a-db15-4a87-b27c-1c5e69af52b8\") " pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:32:07.534186 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.534012    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ea9768a-db15-4a87-b27c-1c5e69af52b8-tmp-dir\") pod \"dns-default-wzhvk\" (UID: \"2ea9768a-db15-4a87-b27c-1c5e69af52b8\") " pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:32:07.635216 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.635121    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4jkv\" (UniqueName: \"kubernetes.io/projected/2ea9768a-db15-4a87-b27c-1c5e69af52b8-kube-api-access-q4jkv\") pod \"dns-default-wzhvk\" (UID: \"2ea9768a-db15-4a87-b27c-1c5e69af52b8\") " pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:32:07.635216 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.635169    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ea9768a-db15-4a87-b27c-1c5e69af52b8-tmp-dir\") pod \"dns-default-wzhvk\" (UID: \"2ea9768a-db15-4a87-b27c-1c5e69af52b8\") " pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:32:07.635446 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.635237    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/243eab96-1170-4ea0-98fa-d9cb4a68adb2-cert\") pod \"ingress-canary-xjh28\" (UID: \"243eab96-1170-4ea0-98fa-d9cb4a68adb2\") " pod="openshift-ingress-canary/ingress-canary-xjh28"
Jul 27 18:32:07.635446 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.635280    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2ea9768a-db15-4a87-b27c-1c5e69af52b8-config-volume\") pod \"dns-default-wzhvk\" (UID: \"2ea9768a-db15-4a87-b27c-1c5e69af52b8\") " pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:32:07.635446 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.635337    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls\") pod \"dns-default-wzhvk\" (UID: \"2ea9768a-db15-4a87-b27c-1c5e69af52b8\") " pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:32:07.635446 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.635365    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cbfv\" (UniqueName: \"kubernetes.io/projected/243eab96-1170-4ea0-98fa-d9cb4a68adb2-kube-api-access-6cbfv\") pod \"ingress-canary-xjh28\" (UID: \"243eab96-1170-4ea0-98fa-d9cb4a68adb2\") " pod="openshift-ingress-canary/ingress-canary-xjh28"
Jul 27 18:32:07.636129 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.636108    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2ea9768a-db15-4a87-b27c-1c5e69af52b8-config-volume\") pod \"dns-default-wzhvk\" (UID: \"2ea9768a-db15-4a87-b27c-1c5e69af52b8\") " pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:32:07.636251 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:07.636208    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 18:32:07.636298 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:07.636287    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls podName:2ea9768a-db15-4a87-b27c-1c5e69af52b8 nodeName:}" failed. No retries permitted until 2026-07-27 18:32:08.136264564 +0000 UTC m=+36.686558176 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls") pod "dns-default-wzhvk" (UID: "2ea9768a-db15-4a87-b27c-1c5e69af52b8") : secret "dns-default-metrics-tls" not found
Jul 27 18:32:07.636441 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.636401    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ea9768a-db15-4a87-b27c-1c5e69af52b8-tmp-dir\") pod \"dns-default-wzhvk\" (UID: \"2ea9768a-db15-4a87-b27c-1c5e69af52b8\") " pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:32:07.641120 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.641092    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/243eab96-1170-4ea0-98fa-d9cb4a68adb2-cert\") pod \"ingress-canary-xjh28\" (UID: \"243eab96-1170-4ea0-98fa-d9cb4a68adb2\") " pod="openshift-ingress-canary/ingress-canary-xjh28"
Jul 27 18:32:07.645411 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.645231    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4jkv\" (UniqueName: \"kubernetes.io/projected/2ea9768a-db15-4a87-b27c-1c5e69af52b8-kube-api-access-q4jkv\") pod \"dns-default-wzhvk\" (UID: \"2ea9768a-db15-4a87-b27c-1c5e69af52b8\") " pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:32:07.646619 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.646565    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cbfv\" (UniqueName: \"kubernetes.io/projected/243eab96-1170-4ea0-98fa-d9cb4a68adb2-kube-api-access-6cbfv\") pod \"ingress-canary-xjh28\" (UID: \"243eab96-1170-4ea0-98fa-d9cb4a68adb2\") " pod="openshift-ingress-canary/ingress-canary-xjh28"
Jul 27 18:32:07.714321 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.714283    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xjh28"
Jul 27 18:32:07.864428 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.864394    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xjh28"]
Jul 27 18:32:07.871210 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:32:07.871175    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod243eab96_1170_4ea0_98fa_d9cb4a68adb2.slice/crio-1b84ea814391087e0925b779f4e1f681b940e0310cd338c2cb10f8423c3ffec5 WatchSource:0}: Error finding container 1b84ea814391087e0925b779f4e1f681b940e0310cd338c2cb10f8423c3ffec5: Status 404 returned error can't find the container with id 1b84ea814391087e0925b779f4e1f681b940e0310cd338c2cb10f8423c3ffec5
Jul 27 18:32:07.931537 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.931459    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:32:07.934373 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.934346    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 18:32:07.934594 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:07.934577    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j4wf8\""
Jul 27 18:32:08.096625 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:08.096584    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xjh28" event={"ID":"243eab96-1170-4ea0-98fa-d9cb4a68adb2","Type":"ContainerStarted","Data":"1b84ea814391087e0925b779f4e1f681b940e0310cd338c2cb10f8423c3ffec5"}
Jul 27 18:32:08.139035 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:08.139001    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls\") pod \"dns-default-wzhvk\" (UID: \"2ea9768a-db15-4a87-b27c-1c5e69af52b8\") " pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:32:08.139227 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:08.139182    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 18:32:08.139294 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:08.139261    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls podName:2ea9768a-db15-4a87-b27c-1c5e69af52b8 nodeName:}" failed. No retries permitted until 2026-07-27 18:32:09.139234834 +0000 UTC m=+37.689528487 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls") pod "dns-default-wzhvk" (UID: "2ea9768a-db15-4a87-b27c-1c5e69af52b8") : secret "dns-default-metrics-tls" not found
Jul 27 18:32:08.930983 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:08.930946    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:32:08.931168 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:08.930946    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:32:08.933837 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:08.933811    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 18:32:08.934354 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:08.933815    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 18:32:08.934354 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:08.934349    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 18:32:08.934530 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:08.934423    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nv59m\""
Jul 27 18:32:09.145705 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:09.145668    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls\") pod \"dns-default-wzhvk\" (UID: \"2ea9768a-db15-4a87-b27c-1c5e69af52b8\") " pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:32:09.145926 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:09.145897    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 18:32:09.146006 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:09.145984    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls podName:2ea9768a-db15-4a87-b27c-1c5e69af52b8 nodeName:}" failed. No retries permitted until 2026-07-27 18:32:11.145960752 +0000 UTC m=+39.696254367 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls") pod "dns-default-wzhvk" (UID: "2ea9768a-db15-4a87-b27c-1c5e69af52b8") : secret "dns-default-metrics-tls" not found
Jul 27 18:32:10.107443 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:10.107406    2562 generic.go:356] "Generic (PLEG): container finished" podID="be23ce78-126d-46a9-95d2-40d68c20f6f2" containerID="d337fefe85a431ad58c673f522f4018c7aa44f9732e01ccab655316ab40977c5" exitCode=0
Jul 27 18:32:10.108017 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:10.107465    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q9k9p" event={"ID":"be23ce78-126d-46a9-95d2-40d68c20f6f2","Type":"ContainerDied","Data":"d337fefe85a431ad58c673f522f4018c7aa44f9732e01ccab655316ab40977c5"}
Jul 27 18:32:11.115071 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:11.115035    2562 generic.go:356] "Generic (PLEG): container finished" podID="be23ce78-126d-46a9-95d2-40d68c20f6f2" containerID="8a1c3e147baeb851dd746a5d8511defeb60d330c4197029fcb37b8b1cac3def4" exitCode=0
Jul 27 18:32:11.115529 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:11.115123    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q9k9p" event={"ID":"be23ce78-126d-46a9-95d2-40d68c20f6f2","Type":"ContainerDied","Data":"8a1c3e147baeb851dd746a5d8511defeb60d330c4197029fcb37b8b1cac3def4"}
Jul 27 18:32:11.159939 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:11.159902    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls\") pod \"dns-default-wzhvk\" (UID: \"2ea9768a-db15-4a87-b27c-1c5e69af52b8\") " pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:32:11.160144 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:11.160123    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 18:32:11.160218 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:11.160204    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls podName:2ea9768a-db15-4a87-b27c-1c5e69af52b8 nodeName:}" failed. No retries permitted until 2026-07-27 18:32:15.160182992 +0000 UTC m=+43.710476603 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls") pod "dns-default-wzhvk" (UID: "2ea9768a-db15-4a87-b27c-1c5e69af52b8") : secret "dns-default-metrics-tls" not found
Jul 27 18:32:12.121663 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:12.121293    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q9k9p" event={"ID":"be23ce78-126d-46a9-95d2-40d68c20f6f2","Type":"ContainerStarted","Data":"a4f1d1738ac39eacd0eafaea50c3ba7afb5711a2720bf3cb5f775f1d15d0e3c8"}
Jul 27 18:32:12.122665 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:12.122630    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xjh28" event={"ID":"243eab96-1170-4ea0-98fa-d9cb4a68adb2","Type":"ContainerStarted","Data":"2949e440f2176f697632ad8f509fb3c8de6452a7a7ed90ef817f141953059729"}
Jul 27 18:32:12.145629 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:12.145576    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-q9k9p" podStartSLOduration=7.083823782 podStartE2EDuration="40.14555922s" totalImagesPullingTime="33.061735438s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 18:31:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:32:12.144277126 +0000 UTC m=+40.694570755" watchObservedRunningTime="2026-07-27 18:32:12.14555922 +0000 UTC m=+40.695852854"
Jul 27 18:32:12.158764 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:12.158714    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xjh28" podStartSLOduration=1.0879986640000001 podStartE2EDuration="5.158694483s" totalImagesPullingTime="4.070695819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:32:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:32:12.158214131 +0000 UTC m=+40.708507777" watchObservedRunningTime="2026-07-27 18:32:12.158694483 +0000 UTC m=+40.708988116"
Jul 27 18:32:15.188460 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:15.188427    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls\") pod \"dns-default-wzhvk\" (UID: \"2ea9768a-db15-4a87-b27c-1c5e69af52b8\") " pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:32:15.188845 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:15.188594    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 18:32:15.188845 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:15.188673    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls podName:2ea9768a-db15-4a87-b27c-1c5e69af52b8 nodeName:}" failed. No retries permitted until 2026-07-27 18:32:23.188657242 +0000 UTC m=+51.738950852 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls") pod "dns-default-wzhvk" (UID: "2ea9768a-db15-4a87-b27c-1c5e69af52b8") : secret "dns-default-metrics-tls" not found
Jul 27 18:32:23.247135 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:23.247092    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls\") pod \"dns-default-wzhvk\" (UID: \"2ea9768a-db15-4a87-b27c-1c5e69af52b8\") " pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:32:23.247604 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:23.247248    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 18:32:23.247604 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:23.247326    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls podName:2ea9768a-db15-4a87-b27c-1c5e69af52b8 nodeName:}" failed. No retries permitted until 2026-07-27 18:32:39.247310121 +0000 UTC m=+67.797603733 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls") pod "dns-default-wzhvk" (UID: "2ea9768a-db15-4a87-b27c-1c5e69af52b8") : secret "dns-default-metrics-tls" not found
Jul 27 18:32:33.091517 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:33.091486    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5wkjf"
Jul 27 18:32:36.640011 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:36.639966    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs\") pod \"network-metrics-daemon-mhtmr\" (UID: \"88f67acb-91af-4850-bb4d-81ff499bc672\") " pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:32:36.640011 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:36.640008    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret\") pod \"global-pull-secret-syncer-rmf9d\" (UID: \"914d3bcb-4f6c-4153-aec8-e4547ff69068\") " pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:32:36.642735 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:36.642714    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 18:32:36.642836 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:36.642744    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 18:32:36.650566 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:36.650547    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 27 18:32:36.650632 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:36.650620    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs podName:88f67acb-91af-4850-bb4d-81ff499bc672 nodeName:}" failed. No retries permitted until 2026-07-27 18:33:40.650599977 +0000 UTC m=+129.200893606 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs") pod "network-metrics-daemon-mhtmr" (UID: "88f67acb-91af-4850-bb4d-81ff499bc672") : secret "metrics-daemon-secret" not found
Jul 27 18:32:36.653814 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:36.653788    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/914d3bcb-4f6c-4153-aec8-e4547ff69068-original-pull-secret\") pod \"global-pull-secret-syncer-rmf9d\" (UID: \"914d3bcb-4f6c-4153-aec8-e4547ff69068\") " pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:32:36.740956 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:36.740925    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrbmb\" (UniqueName: \"kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb\") pod \"network-check-target-zqgfw\" (UID: \"8e6d520b-feb0-4b85-921f-7fe3f1ab3549\") " pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:32:36.743827 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:36.743808    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 18:32:36.754023 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:36.754005    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 18:32:36.765482 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:36.765448    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrbmb\" (UniqueName: \"kubernetes.io/projected/8e6d520b-feb0-4b85-921f-7fe3f1ab3549-kube-api-access-mrbmb\") pod \"network-check-target-zqgfw\" (UID: \"8e6d520b-feb0-4b85-921f-7fe3f1ab3549\") " pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:32:36.843611 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:36.843576    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmf9d"
Jul 27 18:32:36.851839 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:36.851818    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nv59m\""
Jul 27 18:32:36.859851 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:36.859828    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:32:36.992848 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:36.992664    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rmf9d"]
Jul 27 18:32:36.997374 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:32:36.997343    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod914d3bcb_4f6c_4153_aec8_e4547ff69068.slice/crio-39c5ee66cfabba0aff9ad4584ea7b1767ac72754a32ba98f12f1a627c228641b WatchSource:0}: Error finding container 39c5ee66cfabba0aff9ad4584ea7b1767ac72754a32ba98f12f1a627c228641b: Status 404 returned error can't find the container with id 39c5ee66cfabba0aff9ad4584ea7b1767ac72754a32ba98f12f1a627c228641b
Jul 27 18:32:37.021013 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:37.020977    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zqgfw"]
Jul 27 18:32:37.023407 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:32:37.023380    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e6d520b_feb0_4b85_921f_7fe3f1ab3549.slice/crio-7ea48ca6fe26223d88da636bfc3fa37fe881d261372cab6230f5bcb2ed3d95f3 WatchSource:0}: Error finding container 7ea48ca6fe26223d88da636bfc3fa37fe881d261372cab6230f5bcb2ed3d95f3: Status 404 returned error can't find the container with id 7ea48ca6fe26223d88da636bfc3fa37fe881d261372cab6230f5bcb2ed3d95f3
Jul 27 18:32:37.179555 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:37.179456    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rmf9d" event={"ID":"914d3bcb-4f6c-4153-aec8-e4547ff69068","Type":"ContainerStarted","Data":"39c5ee66cfabba0aff9ad4584ea7b1767ac72754a32ba98f12f1a627c228641b"}
Jul 27 18:32:37.180528 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:37.180504    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zqgfw" event={"ID":"8e6d520b-feb0-4b85-921f-7fe3f1ab3549","Type":"ContainerStarted","Data":"7ea48ca6fe26223d88da636bfc3fa37fe881d261372cab6230f5bcb2ed3d95f3"}
Jul 27 18:32:38.389399 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.389041    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr"]
Jul 27 18:32:38.393013 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.392983    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-fwf4j"]
Jul 27 18:32:38.393167 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.393127    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr"
Jul 27 18:32:38.394953 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.394917    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-rbtsd"]
Jul 27 18:32:38.395186 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.395167    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-fwf4j"
Jul 27 18:32:38.396755 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.396736    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd"
Jul 27 18:32:38.401532 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.401502    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 27 18:32:38.403015 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.402841    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 27 18:32:38.403015 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.402954    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 27 18:32:38.403215 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.403060    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 27 18:32:38.405264 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.403968    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-5fz6k\""
Jul 27 18:32:38.405264 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.403992    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 27 18:32:38.405264 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.404206    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 27 18:32:38.405264 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.404216    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-ftrrg\""
Jul 27 18:32:38.405264 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.404759    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 27 18:32:38.405264 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.405003    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-lmlnt\""
Jul 27 18:32:38.410979 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.410954    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 27 18:32:38.414598 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.414417    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 27 18:32:38.414598 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.414462    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 27 18:32:38.414812 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.414794    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 27 18:32:38.416530 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.415162    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr"]
Jul 27 18:32:38.417051 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.417030    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 27 18:32:38.428271 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.428239    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 27 18:32:38.429008 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.428983    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 27 18:32:38.439885 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.439837    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-rbtsd"]
Jul 27 18:32:38.440224 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.439961    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-fwf4j"]
Jul 27 18:32:38.452146 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.452112    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rhrcr\" (UID: \"5d6f565b-25e3-4db4-89cc-28f0a0cbc890\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr"
Jul 27 18:32:38.452392 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.452373    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/476d7150-35a6-4b7d-beee-1fbac9faa00f-snapshots\") pod \"insights-operator-5885fdb759-fwf4j\" (UID: \"476d7150-35a6-4b7d-beee-1fbac9faa00f\") " pod="openshift-insights/insights-operator-5885fdb759-fwf4j"
Jul 27 18:32:38.452565 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.452548    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj9ml\" (UniqueName: \"kubernetes.io/projected/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-kube-api-access-gj9ml\") pod \"cluster-monitoring-operator-548b489cb8-rhrcr\" (UID: \"5d6f565b-25e3-4db4-89cc-28f0a0cbc890\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr"
Jul 27 18:32:38.452725 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.452708    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw22z\" (UniqueName: \"kubernetes.io/projected/2f2170b2-dbd3-4867-9eaf-291ba1ebe412-kube-api-access-mw22z\") pod \"console-operator-584dcb6b54-rbtsd\" (UID: \"2f2170b2-dbd3-4867-9eaf-291ba1ebe412\") " pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd"
Jul 27 18:32:38.452915 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.452889    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476d7150-35a6-4b7d-beee-1fbac9faa00f-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-fwf4j\" (UID: \"476d7150-35a6-4b7d-beee-1fbac9faa00f\") " pod="openshift-insights/insights-operator-5885fdb759-fwf4j"
Jul 27 18:32:38.453034 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.452947    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/476d7150-35a6-4b7d-beee-1fbac9faa00f-tmp\") pod \"insights-operator-5885fdb759-fwf4j\" (UID: \"476d7150-35a6-4b7d-beee-1fbac9faa00f\") " pod="openshift-insights/insights-operator-5885fdb759-fwf4j"
Jul 27 18:32:38.453034 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.452973    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbvrr\" (UniqueName: \"kubernetes.io/projected/476d7150-35a6-4b7d-beee-1fbac9faa00f-kube-api-access-pbvrr\") pod \"insights-operator-5885fdb759-fwf4j\" (UID: \"476d7150-35a6-4b7d-beee-1fbac9faa00f\") " pod="openshift-insights/insights-operator-5885fdb759-fwf4j"
Jul 27 18:32:38.453034 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.452999    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f2170b2-dbd3-4867-9eaf-291ba1ebe412-serving-cert\") pod \"console-operator-584dcb6b54-rbtsd\" (UID: \"2f2170b2-dbd3-4867-9eaf-291ba1ebe412\") " pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd"
Jul 27 18:32:38.453146 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.453046    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-rhrcr\" (UID: \"5d6f565b-25e3-4db4-89cc-28f0a0cbc890\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr"
Jul 27 18:32:38.453146 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.453066    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f2170b2-dbd3-4867-9eaf-291ba1ebe412-config\") pod \"console-operator-584dcb6b54-rbtsd\" (UID: \"2f2170b2-dbd3-4867-9eaf-291ba1ebe412\") " pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd"
Jul 27 18:32:38.453146 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.453081    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f2170b2-dbd3-4867-9eaf-291ba1ebe412-trusted-ca\") pod \"console-operator-584dcb6b54-rbtsd\" (UID: \"2f2170b2-dbd3-4867-9eaf-291ba1ebe412\") " pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd"
Jul 27 18:32:38.453146 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.453124    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/476d7150-35a6-4b7d-beee-1fbac9faa00f-serving-cert\") pod \"insights-operator-5885fdb759-fwf4j\" (UID: \"476d7150-35a6-4b7d-beee-1fbac9faa00f\") " pod="openshift-insights/insights-operator-5885fdb759-fwf4j"
Jul 27 18:32:38.453277 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.453177    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476d7150-35a6-4b7d-beee-1fbac9faa00f-service-ca-bundle\") pod \"insights-operator-5885fdb759-fwf4j\" (UID: \"476d7150-35a6-4b7d-beee-1fbac9faa00f\") " pod="openshift-insights/insights-operator-5885fdb759-fwf4j"
Jul 27 18:32:38.504058 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.504022    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-8jtmk"]
Jul 27 18:32:38.506165 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.506139    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-8jtmk"
Jul 27 18:32:38.509010 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.508978    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 27 18:32:38.509791 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.509771    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-v9r8j\""
Jul 27 18:32:38.514539 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.514391    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 27 18:32:38.527221 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.527189    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-phqs2"]
Jul 27 18:32:38.529313 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.529273    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-phqs2"
Jul 27 18:32:38.534972 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.534947    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-nz2r5\""
Jul 27 18:32:38.535776 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.535746    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-8jtmk"]
Jul 27 18:32:38.553498 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.553464    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/476d7150-35a6-4b7d-beee-1fbac9faa00f-tmp\") pod \"insights-operator-5885fdb759-fwf4j\" (UID: \"476d7150-35a6-4b7d-beee-1fbac9faa00f\") " pod="openshift-insights/insights-operator-5885fdb759-fwf4j"
Jul 27 18:32:38.553632 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.553508    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbvrr\" (UniqueName: \"kubernetes.io/projected/476d7150-35a6-4b7d-beee-1fbac9faa00f-kube-api-access-pbvrr\") pod \"insights-operator-5885fdb759-fwf4j\" (UID: \"476d7150-35a6-4b7d-beee-1fbac9faa00f\") " pod="openshift-insights/insights-operator-5885fdb759-fwf4j"
Jul 27 18:32:38.553632 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.553538    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f2170b2-dbd3-4867-9eaf-291ba1ebe412-serving-cert\") pod \"console-operator-584dcb6b54-rbtsd\" (UID: \"2f2170b2-dbd3-4867-9eaf-291ba1ebe412\") " pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd"
Jul 27 18:32:38.553632 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.553568    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsgbc\" (UniqueName: \"kubernetes.io/projected/a249f430-8f76-4490-a57c-340481be2e63-kube-api-access-bsgbc\") pod \"network-check-source-7d88cb55d8-phqs2\" (UID: \"a249f430-8f76-4490-a57c-340481be2e63\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-phqs2"
Jul 27 18:32:38.553632 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.553597    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-rhrcr\" (UID: \"5d6f565b-25e3-4db4-89cc-28f0a0cbc890\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr"
Jul 27 18:32:38.553632 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.553613    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f2170b2-dbd3-4867-9eaf-291ba1ebe412-config\") pod \"console-operator-584dcb6b54-rbtsd\" (UID: \"2f2170b2-dbd3-4867-9eaf-291ba1ebe412\") " pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd"
Jul 27 18:32:38.553632 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.553630    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f2170b2-dbd3-4867-9eaf-291ba1ebe412-trusted-ca\") pod \"console-operator-584dcb6b54-rbtsd\" (UID: \"2f2170b2-dbd3-4867-9eaf-291ba1ebe412\") " pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd"
Jul 27 18:32:38.554465 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.553651    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/476d7150-35a6-4b7d-beee-1fbac9faa00f-serving-cert\") pod \"insights-operator-5885fdb759-fwf4j\" (UID: \"476d7150-35a6-4b7d-beee-1fbac9faa00f\") " pod="openshift-insights/insights-operator-5885fdb759-fwf4j"
Jul 27 18:32:38.554465 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.553674    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0e1a6428-a17e-485f-96b7-6b1a92afb72d-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-8jtmk\" (UID: \"0e1a6428-a17e-485f-96b7-6b1a92afb72d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-8jtmk"
Jul 27 18:32:38.554465 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.553709    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476d7150-35a6-4b7d-beee-1fbac9faa00f-service-ca-bundle\") pod \"insights-operator-5885fdb759-fwf4j\" (UID: \"476d7150-35a6-4b7d-beee-1fbac9faa00f\") " pod="openshift-insights/insights-operator-5885fdb759-fwf4j"
Jul 27 18:32:38.554465 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.553749    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0e1a6428-a17e-485f-96b7-6b1a92afb72d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-8jtmk\" (UID: \"0e1a6428-a17e-485f-96b7-6b1a92afb72d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-8jtmk"
Jul 27 18:32:38.554465 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.553771    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rhrcr\" (UID: \"5d6f565b-25e3-4db4-89cc-28f0a0cbc890\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr"
Jul 27 18:32:38.554465 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.553796    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/476d7150-35a6-4b7d-beee-1fbac9faa00f-snapshots\") pod \"insights-operator-5885fdb759-fwf4j\" (UID: \"476d7150-35a6-4b7d-beee-1fbac9faa00f\") " pod="openshift-insights/insights-operator-5885fdb759-fwf4j"
Jul 27 18:32:38.554465 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.553982    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj9ml\" (UniqueName: \"kubernetes.io/projected/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-kube-api-access-gj9ml\") pod \"cluster-monitoring-operator-548b489cb8-rhrcr\" (UID: \"5d6f565b-25e3-4db4-89cc-28f0a0cbc890\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr"
Jul 27 18:32:38.554465 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:38.554003    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 18:32:38.554465 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.554017    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw22z\" (UniqueName: \"kubernetes.io/projected/2f2170b2-dbd3-4867-9eaf-291ba1ebe412-kube-api-access-mw22z\") pod \"console-operator-584dcb6b54-rbtsd\" (UID: \"2f2170b2-dbd3-4867-9eaf-291ba1ebe412\") " pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd"
Jul 27 18:32:38.554465 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:38.554097    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-cluster-monitoring-operator-tls podName:5d6f565b-25e3-4db4-89cc-28f0a0cbc890 nodeName:}" failed. No retries permitted until 2026-07-27 18:32:39.054080221 +0000 UTC m=+67.604373832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rhrcr" (UID: "5d6f565b-25e3-4db4-89cc-28f0a0cbc890") : secret "cluster-monitoring-operator-tls" not found
Jul 27 18:32:38.554465 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.554171    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476d7150-35a6-4b7d-beee-1fbac9faa00f-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-fwf4j\" (UID: \"476d7150-35a6-4b7d-beee-1fbac9faa00f\") " pod="openshift-insights/insights-operator-5885fdb759-fwf4j"
Jul 27 18:32:38.555074 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.554497    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/476d7150-35a6-4b7d-beee-1fbac9faa00f-tmp\") pod \"insights-operator-5885fdb759-fwf4j\" (UID: \"476d7150-35a6-4b7d-beee-1fbac9faa00f\") " pod="openshift-insights/insights-operator-5885fdb759-fwf4j"
Jul 27 18:32:38.555212 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.555150    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-rhrcr\" (UID: \"5d6f565b-25e3-4db4-89cc-28f0a0cbc890\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr"
Jul 27 18:32:38.555560 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.555543    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f2170b2-dbd3-4867-9eaf-291ba1ebe412-trusted-ca\") pod \"console-operator-584dcb6b54-rbtsd\" (UID: \"2f2170b2-dbd3-4867-9eaf-291ba1ebe412\") " pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd"
Jul 27 18:32:38.555796 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.555774    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-phqs2"]
Jul 27 18:32:38.556178 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.556155    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f2170b2-dbd3-4867-9eaf-291ba1ebe412-config\") pod \"console-operator-584dcb6b54-rbtsd\" (UID: \"2f2170b2-dbd3-4867-9eaf-291ba1ebe412\") " pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd"
Jul 27 18:32:38.556467 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.556418    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476d7150-35a6-4b7d-beee-1fbac9faa00f-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-fwf4j\" (UID: \"476d7150-35a6-4b7d-beee-1fbac9faa00f\") " pod="openshift-insights/insights-operator-5885fdb759-fwf4j"
Jul 27 18:32:38.557304 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.556944    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/476d7150-35a6-4b7d-beee-1fbac9faa00f-snapshots\") pod \"insights-operator-5885fdb759-fwf4j\" (UID: \"476d7150-35a6-4b7d-beee-1fbac9faa00f\") " pod="openshift-insights/insights-operator-5885fdb759-fwf4j"
Jul 27 18:32:38.557304 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.557266    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476d7150-35a6-4b7d-beee-1fbac9faa00f-service-ca-bundle\") pod \"insights-operator-5885fdb759-fwf4j\" (UID: \"476d7150-35a6-4b7d-beee-1fbac9faa00f\") " pod="openshift-insights/insights-operator-5885fdb759-fwf4j"
Jul 27 18:32:38.558240 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.558212    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f2170b2-dbd3-4867-9eaf-291ba1ebe412-serving-cert\") pod \"console-operator-584dcb6b54-rbtsd\" (UID: \"2f2170b2-dbd3-4867-9eaf-291ba1ebe412\") " pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd"
Jul 27 18:32:38.558696 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.558649    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/476d7150-35a6-4b7d-beee-1fbac9faa00f-serving-cert\") pod \"insights-operator-5885fdb759-fwf4j\" (UID: \"476d7150-35a6-4b7d-beee-1fbac9faa00f\") " pod="openshift-insights/insights-operator-5885fdb759-fwf4j"
Jul 27 18:32:38.576785 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.576730    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw22z\" (UniqueName: \"kubernetes.io/projected/2f2170b2-dbd3-4867-9eaf-291ba1ebe412-kube-api-access-mw22z\") pod \"console-operator-584dcb6b54-rbtsd\" (UID: \"2f2170b2-dbd3-4867-9eaf-291ba1ebe412\") " pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd"
Jul 27 18:32:38.577186 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.577163    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbvrr\" (UniqueName: \"kubernetes.io/projected/476d7150-35a6-4b7d-beee-1fbac9faa00f-kube-api-access-pbvrr\") pod \"insights-operator-5885fdb759-fwf4j\" (UID: \"476d7150-35a6-4b7d-beee-1fbac9faa00f\") " pod="openshift-insights/insights-operator-5885fdb759-fwf4j"
Jul 27 18:32:38.577801 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.577767    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj9ml\" (UniqueName: \"kubernetes.io/projected/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-kube-api-access-gj9ml\") pod \"cluster-monitoring-operator-548b489cb8-rhrcr\" (UID: \"5d6f565b-25e3-4db4-89cc-28f0a0cbc890\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr"
Jul 27 18:32:38.655982 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.655899    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsgbc\" (UniqueName: \"kubernetes.io/projected/a249f430-8f76-4490-a57c-340481be2e63-kube-api-access-bsgbc\") pod \"network-check-source-7d88cb55d8-phqs2\" (UID: \"a249f430-8f76-4490-a57c-340481be2e63\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-phqs2"
Jul 27 18:32:38.655982 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.655945    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0e1a6428-a17e-485f-96b7-6b1a92afb72d-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-8jtmk\" (UID: \"0e1a6428-a17e-485f-96b7-6b1a92afb72d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-8jtmk"
Jul 27 18:32:38.656187 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.655988    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0e1a6428-a17e-485f-96b7-6b1a92afb72d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-8jtmk\" (UID: \"0e1a6428-a17e-485f-96b7-6b1a92afb72d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-8jtmk"
Jul 27 18:32:38.656187 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:38.656096    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 18:32:38.656187 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:38.656175    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e1a6428-a17e-485f-96b7-6b1a92afb72d-networking-console-plugin-cert podName:0e1a6428-a17e-485f-96b7-6b1a92afb72d nodeName:}" failed. No retries permitted until 2026-07-27 18:32:39.156156243 +0000 UTC m=+67.706449855 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0e1a6428-a17e-485f-96b7-6b1a92afb72d-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-8jtmk" (UID: "0e1a6428-a17e-485f-96b7-6b1a92afb72d") : secret "networking-console-plugin-cert" not found
Jul 27 18:32:38.656701 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.656678    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0e1a6428-a17e-485f-96b7-6b1a92afb72d-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-8jtmk\" (UID: \"0e1a6428-a17e-485f-96b7-6b1a92afb72d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-8jtmk"
Jul 27 18:32:38.673168 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.673141    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsgbc\" (UniqueName: \"kubernetes.io/projected/a249f430-8f76-4490-a57c-340481be2e63-kube-api-access-bsgbc\") pod \"network-check-source-7d88cb55d8-phqs2\" (UID: \"a249f430-8f76-4490-a57c-340481be2e63\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-phqs2"
Jul 27 18:32:38.717442 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.717166    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-fwf4j"
Jul 27 18:32:38.724163 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.724132    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd"
Jul 27 18:32:38.843003 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.842720    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-phqs2"
Jul 27 18:32:38.885213 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.885123    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-fwf4j"]
Jul 27 18:32:38.889818 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:32:38.889782    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod476d7150_35a6_4b7d_beee_1fbac9faa00f.slice/crio-7c1e08228f93e0b9b46180cd4850f94c2711a45e5eb68193812f1dcf16c82e91 WatchSource:0}: Error finding container 7c1e08228f93e0b9b46180cd4850f94c2711a45e5eb68193812f1dcf16c82e91: Status 404 returned error can't find the container with id 7c1e08228f93e0b9b46180cd4850f94c2711a45e5eb68193812f1dcf16c82e91
Jul 27 18:32:38.905227 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:38.904182    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-rbtsd"]
Jul 27 18:32:38.915601 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:32:38.915547    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f2170b2_dbd3_4867_9eaf_291ba1ebe412.slice/crio-e0496c54a0c3a957d223f2e59709a8c9133a10ee852686098c68ed920a13ee57 WatchSource:0}: Error finding container e0496c54a0c3a957d223f2e59709a8c9133a10ee852686098c68ed920a13ee57: Status 404 returned error can't find the container with id e0496c54a0c3a957d223f2e59709a8c9133a10ee852686098c68ed920a13ee57
Jul 27 18:32:39.010168 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:39.010131    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-phqs2"]
Jul 27 18:32:39.014904 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:32:39.014842    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda249f430_8f76_4490_a57c_340481be2e63.slice/crio-4bb7bf6fc22a55316c10c8121fc4b75d11f7220f0113c2413fb61bdc063614ea WatchSource:0}: Error finding container 4bb7bf6fc22a55316c10c8121fc4b75d11f7220f0113c2413fb61bdc063614ea: Status 404 returned error can't find the container with id 4bb7bf6fc22a55316c10c8121fc4b75d11f7220f0113c2413fb61bdc063614ea
Jul 27 18:32:39.060543 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:39.060470    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rhrcr\" (UID: \"5d6f565b-25e3-4db4-89cc-28f0a0cbc890\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr"
Jul 27 18:32:39.060734 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:39.060630    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 18:32:39.060734 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:39.060717    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-cluster-monitoring-operator-tls podName:5d6f565b-25e3-4db4-89cc-28f0a0cbc890 nodeName:}" failed. No retries permitted until 2026-07-27 18:32:40.06069689 +0000 UTC m=+68.610990501 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rhrcr" (UID: "5d6f565b-25e3-4db4-89cc-28f0a0cbc890") : secret "cluster-monitoring-operator-tls" not found
Jul 27 18:32:39.161779 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:39.161645    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0e1a6428-a17e-485f-96b7-6b1a92afb72d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-8jtmk\" (UID: \"0e1a6428-a17e-485f-96b7-6b1a92afb72d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-8jtmk"
Jul 27 18:32:39.161978 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:39.161826    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 18:32:39.161978 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:39.161922    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e1a6428-a17e-485f-96b7-6b1a92afb72d-networking-console-plugin-cert podName:0e1a6428-a17e-485f-96b7-6b1a92afb72d nodeName:}" failed. No retries permitted until 2026-07-27 18:32:40.161901603 +0000 UTC m=+68.712195217 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0e1a6428-a17e-485f-96b7-6b1a92afb72d-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-8jtmk" (UID: "0e1a6428-a17e-485f-96b7-6b1a92afb72d") : secret "networking-console-plugin-cert" not found
Jul 27 18:32:39.188470 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:39.188431    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd" event={"ID":"2f2170b2-dbd3-4867-9eaf-291ba1ebe412","Type":"ContainerStarted","Data":"e0496c54a0c3a957d223f2e59709a8c9133a10ee852686098c68ed920a13ee57"}
Jul 27 18:32:39.189858 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:39.189831    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-phqs2" event={"ID":"a249f430-8f76-4490-a57c-340481be2e63","Type":"ContainerStarted","Data":"4bb7bf6fc22a55316c10c8121fc4b75d11f7220f0113c2413fb61bdc063614ea"}
Jul 27 18:32:39.190987 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:39.190968    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-fwf4j" event={"ID":"476d7150-35a6-4b7d-beee-1fbac9faa00f","Type":"ContainerStarted","Data":"7c1e08228f93e0b9b46180cd4850f94c2711a45e5eb68193812f1dcf16c82e91"}
Jul 27 18:32:39.262122 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:39.262073    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls\") pod \"dns-default-wzhvk\" (UID: \"2ea9768a-db15-4a87-b27c-1c5e69af52b8\") " pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:32:39.262299 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:39.262268    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 18:32:39.262366 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:39.262352    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls podName:2ea9768a-db15-4a87-b27c-1c5e69af52b8 nodeName:}" failed. No retries permitted until 2026-07-27 18:33:11.26233138 +0000 UTC m=+99.812624993 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls") pod "dns-default-wzhvk" (UID: "2ea9768a-db15-4a87-b27c-1c5e69af52b8") : secret "dns-default-metrics-tls" not found
Jul 27 18:32:40.069271 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:40.069116    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rhrcr\" (UID: \"5d6f565b-25e3-4db4-89cc-28f0a0cbc890\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr"
Jul 27 18:32:40.069754 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:40.069293    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 18:32:40.069754 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:40.069366    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-cluster-monitoring-operator-tls podName:5d6f565b-25e3-4db4-89cc-28f0a0cbc890 nodeName:}" failed. No retries permitted until 2026-07-27 18:32:42.069343175 +0000 UTC m=+70.619636794 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rhrcr" (UID: "5d6f565b-25e3-4db4-89cc-28f0a0cbc890") : secret "cluster-monitoring-operator-tls" not found
Jul 27 18:32:40.170704 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:40.170658    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0e1a6428-a17e-485f-96b7-6b1a92afb72d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-8jtmk\" (UID: \"0e1a6428-a17e-485f-96b7-6b1a92afb72d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-8jtmk"
Jul 27 18:32:40.182187 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:40.182109    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 18:32:40.182364 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:40.182216    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e1a6428-a17e-485f-96b7-6b1a92afb72d-networking-console-plugin-cert podName:0e1a6428-a17e-485f-96b7-6b1a92afb72d nodeName:}" failed. No retries permitted until 2026-07-27 18:32:42.182194834 +0000 UTC m=+70.732488449 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0e1a6428-a17e-485f-96b7-6b1a92afb72d-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-8jtmk" (UID: "0e1a6428-a17e-485f-96b7-6b1a92afb72d") : secret "networking-console-plugin-cert" not found
Jul 27 18:32:42.086962 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:42.086922    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rhrcr\" (UID: \"5d6f565b-25e3-4db4-89cc-28f0a0cbc890\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr"
Jul 27 18:32:42.087399 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:42.087070    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 18:32:42.087399 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:42.087176    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-cluster-monitoring-operator-tls podName:5d6f565b-25e3-4db4-89cc-28f0a0cbc890 nodeName:}" failed. No retries permitted until 2026-07-27 18:32:46.087153036 +0000 UTC m=+74.637446676 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rhrcr" (UID: "5d6f565b-25e3-4db4-89cc-28f0a0cbc890") : secret "cluster-monitoring-operator-tls" not found
Jul 27 18:32:42.188395 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:42.188349    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0e1a6428-a17e-485f-96b7-6b1a92afb72d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-8jtmk\" (UID: \"0e1a6428-a17e-485f-96b7-6b1a92afb72d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-8jtmk"
Jul 27 18:32:42.188582 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:42.188542    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 18:32:42.188641 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:42.188628    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e1a6428-a17e-485f-96b7-6b1a92afb72d-networking-console-plugin-cert podName:0e1a6428-a17e-485f-96b7-6b1a92afb72d nodeName:}" failed. No retries permitted until 2026-07-27 18:32:46.188604341 +0000 UTC m=+74.738897974 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0e1a6428-a17e-485f-96b7-6b1a92afb72d-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-8jtmk" (UID: "0e1a6428-a17e-485f-96b7-6b1a92afb72d") : secret "networking-console-plugin-cert" not found
Jul 27 18:32:44.461082 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:44.460612    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-2th6l"]
Jul 27 18:32:44.463549 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:44.463529    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-2th6l"
Jul 27 18:32:44.466935 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:44.466917    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 27 18:32:44.467016 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:44.466920    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 27 18:32:44.467339 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:44.467128    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 27 18:32:44.467764 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:44.467707    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 27 18:32:44.471322 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:44.468001    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-8qqt9\""
Jul 27 18:32:44.478396 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:44.478365    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-2th6l"]
Jul 27 18:32:44.507570 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:44.507541    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvldl\" (UniqueName: \"kubernetes.io/projected/eb075159-820a-49e1-b542-035d42f46347-kube-api-access-hvldl\") pod \"service-ca-658d7b7c78-2th6l\" (UID: \"eb075159-820a-49e1-b542-035d42f46347\") " pod="openshift-service-ca/service-ca-658d7b7c78-2th6l"
Jul 27 18:32:44.507744 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:44.507667    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/eb075159-820a-49e1-b542-035d42f46347-signing-cabundle\") pod \"service-ca-658d7b7c78-2th6l\" (UID: \"eb075159-820a-49e1-b542-035d42f46347\") " pod="openshift-service-ca/service-ca-658d7b7c78-2th6l"
Jul 27 18:32:44.507744 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:44.507731    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/eb075159-820a-49e1-b542-035d42f46347-signing-key\") pod \"service-ca-658d7b7c78-2th6l\" (UID: \"eb075159-820a-49e1-b542-035d42f46347\") " pod="openshift-service-ca/service-ca-658d7b7c78-2th6l"
Jul 27 18:32:44.608968 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:44.608928    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvldl\" (UniqueName: \"kubernetes.io/projected/eb075159-820a-49e1-b542-035d42f46347-kube-api-access-hvldl\") pod \"service-ca-658d7b7c78-2th6l\" (UID: \"eb075159-820a-49e1-b542-035d42f46347\") " pod="openshift-service-ca/service-ca-658d7b7c78-2th6l"
Jul 27 18:32:44.609240 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:44.609008    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/eb075159-820a-49e1-b542-035d42f46347-signing-cabundle\") pod \"service-ca-658d7b7c78-2th6l\" (UID: \"eb075159-820a-49e1-b542-035d42f46347\") " pod="openshift-service-ca/service-ca-658d7b7c78-2th6l"
Jul 27 18:32:44.609240 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:44.609039    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/eb075159-820a-49e1-b542-035d42f46347-signing-key\") pod \"service-ca-658d7b7c78-2th6l\" (UID: \"eb075159-820a-49e1-b542-035d42f46347\") " pod="openshift-service-ca/service-ca-658d7b7c78-2th6l"
Jul 27 18:32:44.610189 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:44.610160    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/eb075159-820a-49e1-b542-035d42f46347-signing-cabundle\") pod \"service-ca-658d7b7c78-2th6l\" (UID: \"eb075159-820a-49e1-b542-035d42f46347\") " pod="openshift-service-ca/service-ca-658d7b7c78-2th6l"
Jul 27 18:32:44.611546 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:44.611528    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/eb075159-820a-49e1-b542-035d42f46347-signing-key\") pod \"service-ca-658d7b7c78-2th6l\" (UID: \"eb075159-820a-49e1-b542-035d42f46347\") " pod="openshift-service-ca/service-ca-658d7b7c78-2th6l"
Jul 27 18:32:44.619705 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:44.619647    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvldl\" (UniqueName: \"kubernetes.io/projected/eb075159-820a-49e1-b542-035d42f46347-kube-api-access-hvldl\") pod \"service-ca-658d7b7c78-2th6l\" (UID: \"eb075159-820a-49e1-b542-035d42f46347\") " pod="openshift-service-ca/service-ca-658d7b7c78-2th6l"
Jul 27 18:32:44.814236 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:44.814183    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-2th6l"
Jul 27 18:32:44.960783 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:44.960747    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-2th6l"]
Jul 27 18:32:44.964278 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:32:44.964250    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb075159_820a_49e1_b542_035d42f46347.slice/crio-20e0fbbbe7ef2e9ada2fe977ce5708b655d27f7e617df9c7d712642f07ed2cc4 WatchSource:0}: Error finding container 20e0fbbbe7ef2e9ada2fe977ce5708b655d27f7e617df9c7d712642f07ed2cc4: Status 404 returned error can't find the container with id 20e0fbbbe7ef2e9ada2fe977ce5708b655d27f7e617df9c7d712642f07ed2cc4
Jul 27 18:32:45.211122 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:45.211080    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zqgfw" event={"ID":"8e6d520b-feb0-4b85-921f-7fe3f1ab3549","Type":"ContainerStarted","Data":"f23d12c054d74fb8c74dfd56049a8938bbc2d23fb95b90bcbc78051a3a54b39f"}
Jul 27 18:32:45.211317 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:45.211178    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:32:45.212759 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:45.212709    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-phqs2" event={"ID":"a249f430-8f76-4490-a57c-340481be2e63","Type":"ContainerStarted","Data":"f9e8359bc942c75650e4775d2f3addec4e93ce2f7d99326b3dda5709d2c0f4eb"}
Jul 27 18:32:45.214424 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:45.214396    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-fwf4j" event={"ID":"476d7150-35a6-4b7d-beee-1fbac9faa00f","Type":"ContainerStarted","Data":"f2893086d4162cde9e3fbae6319dbb05577312c30be3c38281987e2b3a8fab98"}
Jul 27 18:32:45.215830 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:45.215807    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rmf9d" event={"ID":"914d3bcb-4f6c-4153-aec8-e4547ff69068","Type":"ContainerStarted","Data":"a8d7cfd9cd485b525976224e5f7f96e6ff84fb332c4ff2bc74bad58d4da746c0"}
Jul 27 18:32:45.217448 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:45.217428    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/0.log"
Jul 27 18:32:45.217546 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:45.217461    2562 generic.go:356] "Generic (PLEG): container finished" podID="2f2170b2-dbd3-4867-9eaf-291ba1ebe412" containerID="3039ee5a0b814d4439b4fed2d2b7e4bee8988b69b8cd01d6034717a1253e2655" exitCode=255
Jul 27 18:32:45.217546 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:45.217522    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd" event={"ID":"2f2170b2-dbd3-4867-9eaf-291ba1ebe412","Type":"ContainerDied","Data":"3039ee5a0b814d4439b4fed2d2b7e4bee8988b69b8cd01d6034717a1253e2655"}
Jul 27 18:32:45.217713 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:45.217692    2562 scope.go:117] "RemoveContainer" containerID="3039ee5a0b814d4439b4fed2d2b7e4bee8988b69b8cd01d6034717a1253e2655"
Jul 27 18:32:45.218736 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:45.218713    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-2th6l" event={"ID":"eb075159-820a-49e1-b542-035d42f46347","Type":"ContainerStarted","Data":"20e0fbbbe7ef2e9ada2fe977ce5708b655d27f7e617df9c7d712642f07ed2cc4"}
Jul 27 18:32:45.241541 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:45.241487    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zqgfw" podStartSLOduration=65.882435205 podStartE2EDuration="1m13.241466384s" totalImagesPullingTime="7.359031179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:31:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:32:45.241063557 +0000 UTC m=+73.791357190" watchObservedRunningTime="2026-07-27 18:32:45.241466384 +0000 UTC m=+73.791760020"
Jul 27 18:32:45.272648 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:45.272590    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-fwf4j" podStartSLOduration=1.7806632420000001 podStartE2EDuration="7.272567692s" totalImagesPullingTime="5.49190445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:32:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:32:45.272519821 +0000 UTC m=+73.822813453" watchObservedRunningTime="2026-07-27 18:32:45.272567692 +0000 UTC m=+73.822861326"
Jul 27 18:32:45.319148 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:45.319095    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-phqs2" podStartSLOduration=1.951792835 podStartE2EDuration="7.319075675s" totalImagesPullingTime="5.36728284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:32:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:32:45.318237232 +0000 UTC m=+73.868530865" watchObservedRunningTime="2026-07-27 18:32:45.319075675 +0000 UTC m=+73.869369312"
Jul 27 18:32:46.121003 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:46.120962    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rhrcr\" (UID: \"5d6f565b-25e3-4db4-89cc-28f0a0cbc890\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr"
Jul 27 18:32:46.121428 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:46.121110    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 18:32:46.121428 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:46.121175    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-cluster-monitoring-operator-tls podName:5d6f565b-25e3-4db4-89cc-28f0a0cbc890 nodeName:}" failed. No retries permitted until 2026-07-27 18:32:54.121158903 +0000 UTC m=+82.671452516 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rhrcr" (UID: "5d6f565b-25e3-4db4-89cc-28f0a0cbc890") : secret "cluster-monitoring-operator-tls" not found
Jul 27 18:32:46.221989 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:46.221948    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0e1a6428-a17e-485f-96b7-6b1a92afb72d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-8jtmk\" (UID: \"0e1a6428-a17e-485f-96b7-6b1a92afb72d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-8jtmk"
Jul 27 18:32:46.222182 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:46.222075    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 18:32:46.222182 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:46.222140    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e1a6428-a17e-485f-96b7-6b1a92afb72d-networking-console-plugin-cert podName:0e1a6428-a17e-485f-96b7-6b1a92afb72d nodeName:}" failed. No retries permitted until 2026-07-27 18:32:54.222123569 +0000 UTC m=+82.772417179 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0e1a6428-a17e-485f-96b7-6b1a92afb72d-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-8jtmk" (UID: "0e1a6428-a17e-485f-96b7-6b1a92afb72d") : secret "networking-console-plugin-cert" not found
Jul 27 18:32:46.224144 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:46.224122    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/1.log"
Jul 27 18:32:46.224571 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:46.224549    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/0.log"
Jul 27 18:32:46.224672 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:46.224591    2562 generic.go:356] "Generic (PLEG): container finished" podID="2f2170b2-dbd3-4867-9eaf-291ba1ebe412" containerID="08509cb8854c3ec7dd970b8281459f4ad101192535c525033e3b11090f74c65d" exitCode=255
Jul 27 18:32:46.224721 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:46.224692    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd" event={"ID":"2f2170b2-dbd3-4867-9eaf-291ba1ebe412","Type":"ContainerDied","Data":"08509cb8854c3ec7dd970b8281459f4ad101192535c525033e3b11090f74c65d"}
Jul 27 18:32:46.224770 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:46.224748    2562 scope.go:117] "RemoveContainer" containerID="3039ee5a0b814d4439b4fed2d2b7e4bee8988b69b8cd01d6034717a1253e2655"
Jul 27 18:32:46.225046 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:46.225028    2562 scope.go:117] "RemoveContainer" containerID="08509cb8854c3ec7dd970b8281459f4ad101192535c525033e3b11090f74c65d"
Jul 27 18:32:46.226661 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:46.225491    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-rbtsd_openshift-console-operator(2f2170b2-dbd3-4867-9eaf-291ba1ebe412)\"" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd" podUID="2f2170b2-dbd3-4867-9eaf-291ba1ebe412"
Jul 27 18:32:46.246658 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:46.246588    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rmf9d" podStartSLOduration=65.860439496 podStartE2EDuration="1m13.246564373s" totalImagesPullingTime="7.386124877s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 18:31:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:32:45.350170348 +0000 UTC m=+73.900463984" watchObservedRunningTime="2026-07-27 18:32:46.246564373 +0000 UTC m=+74.796858039"
Jul 27 18:32:47.230416 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:47.230386    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/1.log"
Jul 27 18:32:47.230936 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:47.230830    2562 scope.go:117] "RemoveContainer" containerID="08509cb8854c3ec7dd970b8281459f4ad101192535c525033e3b11090f74c65d"
Jul 27 18:32:47.231062 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:47.231037    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-rbtsd_openshift-console-operator(2f2170b2-dbd3-4867-9eaf-291ba1ebe412)\"" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd" podUID="2f2170b2-dbd3-4867-9eaf-291ba1ebe412"
Jul 27 18:32:48.234764 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:48.234726    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-2th6l" event={"ID":"eb075159-820a-49e1-b542-035d42f46347","Type":"ContainerStarted","Data":"c4370d6206437f73f5d2cffc43e3aa06d81607f6e26b94b5b7289806ca88d6a0"}
Jul 27 18:32:48.253959 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:48.253906    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-2th6l" podStartSLOduration=1.8913363680000002 podStartE2EDuration="4.253890039s" totalImagesPullingTime="2.362553671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:32:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:32:48.252846313 +0000 UTC m=+76.803139946" watchObservedRunningTime="2026-07-27 18:32:48.253890039 +0000 UTC m=+76.804183662"
Jul 27 18:32:48.333203 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:48.333176    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8kqdg_05f049d3-8c6b-4155-8586-be26e91c5bb9/dns-node-resolver/0.log"
Jul 27 18:32:48.724521 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:48.724483    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd"
Jul 27 18:32:48.724729 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:48.724576    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd"
Jul 27 18:32:48.724985 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:48.724969    2562 scope.go:117] "RemoveContainer" containerID="08509cb8854c3ec7dd970b8281459f4ad101192535c525033e3b11090f74c65d"
Jul 27 18:32:48.725282 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:48.725244    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-rbtsd_openshift-console-operator(2f2170b2-dbd3-4867-9eaf-291ba1ebe412)\"" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd" podUID="2f2170b2-dbd3-4867-9eaf-291ba1ebe412"
Jul 27 18:32:49.237688 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:49.237661    2562 scope.go:117] "RemoveContainer" containerID="08509cb8854c3ec7dd970b8281459f4ad101192535c525033e3b11090f74c65d"
Jul 27 18:32:49.238074 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:49.237798    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-rbtsd_openshift-console-operator(2f2170b2-dbd3-4867-9eaf-291ba1ebe412)\"" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd" podUID="2f2170b2-dbd3-4867-9eaf-291ba1ebe412"
Jul 27 18:32:49.534758 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:49.534684    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jz5n4_2efd4785-cf26-4815-86af-86dd8dbd5108/node-ca/0.log"
Jul 27 18:32:50.534902 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:50.534859    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xjh28_243eab96-1170-4ea0-98fa-d9cb4a68adb2/serve-healthcheck-canary/0.log"
Jul 27 18:32:54.187750 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:54.187611    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rhrcr\" (UID: \"5d6f565b-25e3-4db4-89cc-28f0a0cbc890\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr"
Jul 27 18:32:54.188163 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:54.187773    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 18:32:54.188163 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:54.187840    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-cluster-monitoring-operator-tls podName:5d6f565b-25e3-4db4-89cc-28f0a0cbc890 nodeName:}" failed. No retries permitted until 2026-07-27 18:33:10.187824885 +0000 UTC m=+98.738118496 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rhrcr" (UID: "5d6f565b-25e3-4db4-89cc-28f0a0cbc890") : secret "cluster-monitoring-operator-tls" not found
Jul 27 18:32:54.288358 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:32:54.288323    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0e1a6428-a17e-485f-96b7-6b1a92afb72d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-8jtmk\" (UID: \"0e1a6428-a17e-485f-96b7-6b1a92afb72d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-8jtmk"
Jul 27 18:32:54.288574 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:54.288481    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 18:32:54.288574 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:32:54.288550    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e1a6428-a17e-485f-96b7-6b1a92afb72d-networking-console-plugin-cert podName:0e1a6428-a17e-485f-96b7-6b1a92afb72d nodeName:}" failed. No retries permitted until 2026-07-27 18:33:10.288533749 +0000 UTC m=+98.838827362 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0e1a6428-a17e-485f-96b7-6b1a92afb72d-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-8jtmk" (UID: "0e1a6428-a17e-485f-96b7-6b1a92afb72d") : secret "networking-console-plugin-cert" not found
Jul 27 18:33:04.931831 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:04.931796    2562 scope.go:117] "RemoveContainer" containerID="08509cb8854c3ec7dd970b8281459f4ad101192535c525033e3b11090f74c65d"
Jul 27 18:33:05.286947 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:05.286916    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 18:33:05.287265 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:05.287249    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/1.log"
Jul 27 18:33:05.287355 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:05.287287    2562 generic.go:356] "Generic (PLEG): container finished" podID="2f2170b2-dbd3-4867-9eaf-291ba1ebe412" containerID="dad8aef3d3a0e91af375e52b31265612f3667c9f95b8830388e057486f0a8400" exitCode=255
Jul 27 18:33:05.287402 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:05.287365    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd" event={"ID":"2f2170b2-dbd3-4867-9eaf-291ba1ebe412","Type":"ContainerDied","Data":"dad8aef3d3a0e91af375e52b31265612f3667c9f95b8830388e057486f0a8400"}
Jul 27 18:33:05.287441 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:05.287424    2562 scope.go:117] "RemoveContainer" containerID="08509cb8854c3ec7dd970b8281459f4ad101192535c525033e3b11090f74c65d"
Jul 27 18:33:05.287729 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:05.287711    2562 scope.go:117] "RemoveContainer" containerID="dad8aef3d3a0e91af375e52b31265612f3667c9f95b8830388e057486f0a8400"
Jul 27 18:33:05.287979 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:33:05.287957    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-rbtsd_openshift-console-operator(2f2170b2-dbd3-4867-9eaf-291ba1ebe412)\"" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd" podUID="2f2170b2-dbd3-4867-9eaf-291ba1ebe412"
Jul 27 18:33:06.292523 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:06.292496    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 18:33:08.725109 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:08.725068    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd"
Jul 27 18:33:08.725510 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:08.725142    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd"
Jul 27 18:33:08.725510 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:08.725467    2562 scope.go:117] "RemoveContainer" containerID="dad8aef3d3a0e91af375e52b31265612f3667c9f95b8830388e057486f0a8400"
Jul 27 18:33:08.725642 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:33:08.725620    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-rbtsd_openshift-console-operator(2f2170b2-dbd3-4867-9eaf-291ba1ebe412)\"" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd" podUID="2f2170b2-dbd3-4867-9eaf-291ba1ebe412"
Jul 27 18:33:10.208908 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.208848    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rhrcr\" (UID: \"5d6f565b-25e3-4db4-89cc-28f0a0cbc890\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr"
Jul 27 18:33:10.211411 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.211380    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d6f565b-25e3-4db4-89cc-28f0a0cbc890-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rhrcr\" (UID: \"5d6f565b-25e3-4db4-89cc-28f0a0cbc890\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr"
Jul 27 18:33:10.309591 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.309555    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0e1a6428-a17e-485f-96b7-6b1a92afb72d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-8jtmk\" (UID: \"0e1a6428-a17e-485f-96b7-6b1a92afb72d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-8jtmk"
Jul 27 18:33:10.312206 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.312179    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0e1a6428-a17e-485f-96b7-6b1a92afb72d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-8jtmk\" (UID: \"0e1a6428-a17e-485f-96b7-6b1a92afb72d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-8jtmk"
Jul 27 18:33:10.316965 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.316932    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-8jtmk"
Jul 27 18:33:10.452687 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.452655    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-8jtmk"]
Jul 27 18:33:10.455760 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:33:10.455735    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e1a6428_a17e_485f_96b7_6b1a92afb72d.slice/crio-c2b4f196c438d529a1895643cb0b9814091233c1e2b068a28d8d3497cc0fadd6 WatchSource:0}: Error finding container c2b4f196c438d529a1895643cb0b9814091233c1e2b068a28d8d3497cc0fadd6: Status 404 returned error can't find the container with id c2b4f196c438d529a1895643cb0b9814091233c1e2b068a28d8d3497cc0fadd6
Jul 27 18:33:10.507839 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.507802    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr"
Jul 27 18:33:10.643119 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.643077    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr"]
Jul 27 18:33:10.646142 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:33:10.646095    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d6f565b_25e3_4db4_89cc_28f0a0cbc890.slice/crio-37f46f8a47be5029da2b4d4df9e004a1316d6f5828c2befd175f7093528c116d WatchSource:0}: Error finding container 37f46f8a47be5029da2b4d4df9e004a1316d6f5828c2befd175f7093528c116d: Status 404 returned error can't find the container with id 37f46f8a47be5029da2b4d4df9e004a1316d6f5828c2befd175f7093528c116d
Jul 27 18:33:10.674368 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.674341    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mpdtb"]
Jul 27 18:33:10.704082 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.704045    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mpdtb"]
Jul 27 18:33:10.704241 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.704179    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mpdtb"
Jul 27 18:33:10.709667 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.709606    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 27 18:33:10.709936 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.709857    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 27 18:33:10.710083 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.710067    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-blrxc\""
Jul 27 18:33:10.791431 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.791400    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f9984ff4c-rhxth"]
Jul 27 18:33:10.813818 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.813783    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c71f0bfd-e960-46e5-b005-2978249db03a-crio-socket\") pod \"insights-runtime-extractor-mpdtb\" (UID: \"c71f0bfd-e960-46e5-b005-2978249db03a\") " pod="openshift-insights/insights-runtime-extractor-mpdtb"
Jul 27 18:33:10.814009 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.813826    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c71f0bfd-e960-46e5-b005-2978249db03a-data-volume\") pod \"insights-runtime-extractor-mpdtb\" (UID: \"c71f0bfd-e960-46e5-b005-2978249db03a\") " pod="openshift-insights/insights-runtime-extractor-mpdtb"
Jul 27 18:33:10.814009 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.813852    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksh2t\" (UniqueName: \"kubernetes.io/projected/c71f0bfd-e960-46e5-b005-2978249db03a-kube-api-access-ksh2t\") pod \"insights-runtime-extractor-mpdtb\" (UID: \"c71f0bfd-e960-46e5-b005-2978249db03a\") " pod="openshift-insights/insights-runtime-extractor-mpdtb"
Jul 27 18:33:10.814009 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.813970    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c71f0bfd-e960-46e5-b005-2978249db03a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mpdtb\" (UID: \"c71f0bfd-e960-46e5-b005-2978249db03a\") " pod="openshift-insights/insights-runtime-extractor-mpdtb"
Jul 27 18:33:10.814009 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.813997    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"]
Jul 27 18:33:10.814009 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.814005    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c71f0bfd-e960-46e5-b005-2978249db03a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mpdtb\" (UID: \"c71f0bfd-e960-46e5-b005-2978249db03a\") " pod="openshift-insights/insights-runtime-extractor-mpdtb"
Jul 27 18:33:10.814186 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.814172    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f9984ff4c-rhxth"
Jul 27 18:33:10.817112 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.817092    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 27 18:33:10.818984 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.818961    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 27 18:33:10.819120 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.818998    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 27 18:33:10.819120 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.819065    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 27 18:33:10.833988 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.833966    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f9984ff4c-rhxth"]
Jul 27 18:33:10.834128 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.834065    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"
Jul 27 18:33:10.837642 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.837366    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 27 18:33:10.837642 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.837629    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 27 18:33:10.837839 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.837751    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 27 18:33:10.838310 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.838292    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 27 18:33:10.838768 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.838749    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"]
Jul 27 18:33:10.915105 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.915071    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bc0f5544-e98f-4b88-9a17-650fe3f8e803-hub\") pod \"cluster-proxy-proxy-agent-54cbc88f69-nsgcp\" (UID: \"bc0f5544-e98f-4b88-9a17-650fe3f8e803\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"
Jul 27 18:33:10.915267 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.915121    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/727ed70a-3766-4437-8ad9-d87535a6c486-tmp\") pod \"klusterlet-addon-workmgr-f9984ff4c-rhxth\" (UID: \"727ed70a-3766-4437-8ad9-d87535a6c486\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f9984ff4c-rhxth"
Jul 27 18:33:10.915267 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.915138    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bc0f5544-e98f-4b88-9a17-650fe3f8e803-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-54cbc88f69-nsgcp\" (UID: \"bc0f5544-e98f-4b88-9a17-650fe3f8e803\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"
Jul 27 18:33:10.915267 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.915163    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bc0f5544-e98f-4b88-9a17-650fe3f8e803-ca\") pod \"cluster-proxy-proxy-agent-54cbc88f69-nsgcp\" (UID: \"bc0f5544-e98f-4b88-9a17-650fe3f8e803\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"
Jul 27 18:33:10.915267 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.915189    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c71f0bfd-e960-46e5-b005-2978249db03a-crio-socket\") pod \"insights-runtime-extractor-mpdtb\" (UID: \"c71f0bfd-e960-46e5-b005-2978249db03a\") " pod="openshift-insights/insights-runtime-extractor-mpdtb"
Jul 27 18:33:10.915267 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.915213    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bc0f5544-e98f-4b88-9a17-650fe3f8e803-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-54cbc88f69-nsgcp\" (UID: \"bc0f5544-e98f-4b88-9a17-650fe3f8e803\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"
Jul 27 18:33:10.915267 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.915230    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c55f\" (UniqueName: \"kubernetes.io/projected/bc0f5544-e98f-4b88-9a17-650fe3f8e803-kube-api-access-2c55f\") pod \"cluster-proxy-proxy-agent-54cbc88f69-nsgcp\" (UID: \"bc0f5544-e98f-4b88-9a17-650fe3f8e803\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"
Jul 27 18:33:10.915267 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.915252    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh6mq\" (UniqueName: \"kubernetes.io/projected/727ed70a-3766-4437-8ad9-d87535a6c486-kube-api-access-mh6mq\") pod \"klusterlet-addon-workmgr-f9984ff4c-rhxth\" (UID: \"727ed70a-3766-4437-8ad9-d87535a6c486\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f9984ff4c-rhxth"
Jul 27 18:33:10.915510 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.915289    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c71f0bfd-e960-46e5-b005-2978249db03a-data-volume\") pod \"insights-runtime-extractor-mpdtb\" (UID: \"c71f0bfd-e960-46e5-b005-2978249db03a\") " pod="openshift-insights/insights-runtime-extractor-mpdtb"
Jul 27 18:33:10.915510 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.915288    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c71f0bfd-e960-46e5-b005-2978249db03a-crio-socket\") pod \"insights-runtime-extractor-mpdtb\" (UID: \"c71f0bfd-e960-46e5-b005-2978249db03a\") " pod="openshift-insights/insights-runtime-extractor-mpdtb"
Jul 27 18:33:10.915510 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.915368    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksh2t\" (UniqueName: \"kubernetes.io/projected/c71f0bfd-e960-46e5-b005-2978249db03a-kube-api-access-ksh2t\") pod \"insights-runtime-extractor-mpdtb\" (UID: \"c71f0bfd-e960-46e5-b005-2978249db03a\") " pod="openshift-insights/insights-runtime-extractor-mpdtb"
Jul 27 18:33:10.915510 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.915409    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/727ed70a-3766-4437-8ad9-d87535a6c486-klusterlet-config\") pod \"klusterlet-addon-workmgr-f9984ff4c-rhxth\" (UID: \"727ed70a-3766-4437-8ad9-d87535a6c486\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f9984ff4c-rhxth"
Jul 27 18:33:10.915510 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.915444    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c71f0bfd-e960-46e5-b005-2978249db03a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mpdtb\" (UID: \"c71f0bfd-e960-46e5-b005-2978249db03a\") " pod="openshift-insights/insights-runtime-extractor-mpdtb"
Jul 27 18:33:10.915510 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.915487    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c71f0bfd-e960-46e5-b005-2978249db03a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mpdtb\" (UID: \"c71f0bfd-e960-46e5-b005-2978249db03a\") " pod="openshift-insights/insights-runtime-extractor-mpdtb"
Jul 27 18:33:10.915704 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.915526    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc0f5544-e98f-4b88-9a17-650fe3f8e803-additional-service-ca\") pod \"cluster-proxy-proxy-agent-54cbc88f69-nsgcp\" (UID: \"bc0f5544-e98f-4b88-9a17-650fe3f8e803\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"
Jul 27 18:33:10.915704 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.915619    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c71f0bfd-e960-46e5-b005-2978249db03a-data-volume\") pod \"insights-runtime-extractor-mpdtb\" (UID: \"c71f0bfd-e960-46e5-b005-2978249db03a\") " pod="openshift-insights/insights-runtime-extractor-mpdtb"
Jul 27 18:33:10.916026 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.916009    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c71f0bfd-e960-46e5-b005-2978249db03a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mpdtb\" (UID: \"c71f0bfd-e960-46e5-b005-2978249db03a\") " pod="openshift-insights/insights-runtime-extractor-mpdtb"
Jul 27 18:33:10.918055 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.918037    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c71f0bfd-e960-46e5-b005-2978249db03a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mpdtb\" (UID: \"c71f0bfd-e960-46e5-b005-2978249db03a\") " pod="openshift-insights/insights-runtime-extractor-mpdtb"
Jul 27 18:33:10.939572 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:10.939545    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksh2t\" (UniqueName: \"kubernetes.io/projected/c71f0bfd-e960-46e5-b005-2978249db03a-kube-api-access-ksh2t\") pod \"insights-runtime-extractor-mpdtb\" (UID: \"c71f0bfd-e960-46e5-b005-2978249db03a\") " pod="openshift-insights/insights-runtime-extractor-mpdtb"
Jul 27 18:33:11.014021 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.013924    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mpdtb"
Jul 27 18:33:11.016223 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.016203    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh6mq\" (UniqueName: \"kubernetes.io/projected/727ed70a-3766-4437-8ad9-d87535a6c486-kube-api-access-mh6mq\") pod \"klusterlet-addon-workmgr-f9984ff4c-rhxth\" (UID: \"727ed70a-3766-4437-8ad9-d87535a6c486\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f9984ff4c-rhxth"
Jul 27 18:33:11.016299 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.016235    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/727ed70a-3766-4437-8ad9-d87535a6c486-klusterlet-config\") pod \"klusterlet-addon-workmgr-f9984ff4c-rhxth\" (UID: \"727ed70a-3766-4437-8ad9-d87535a6c486\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f9984ff4c-rhxth"
Jul 27 18:33:11.016299 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.016277    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc0f5544-e98f-4b88-9a17-650fe3f8e803-additional-service-ca\") pod \"cluster-proxy-proxy-agent-54cbc88f69-nsgcp\" (UID: \"bc0f5544-e98f-4b88-9a17-650fe3f8e803\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"
Jul 27 18:33:11.016454 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.016433    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bc0f5544-e98f-4b88-9a17-650fe3f8e803-hub\") pod \"cluster-proxy-proxy-agent-54cbc88f69-nsgcp\" (UID: \"bc0f5544-e98f-4b88-9a17-650fe3f8e803\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"
Jul 27 18:33:11.016515 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.016494    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/727ed70a-3766-4437-8ad9-d87535a6c486-tmp\") pod \"klusterlet-addon-workmgr-f9984ff4c-rhxth\" (UID: \"727ed70a-3766-4437-8ad9-d87535a6c486\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f9984ff4c-rhxth"
Jul 27 18:33:11.016569 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.016525    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bc0f5544-e98f-4b88-9a17-650fe3f8e803-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-54cbc88f69-nsgcp\" (UID: \"bc0f5544-e98f-4b88-9a17-650fe3f8e803\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"
Jul 27 18:33:11.016569 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.016554    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bc0f5544-e98f-4b88-9a17-650fe3f8e803-ca\") pod \"cluster-proxy-proxy-agent-54cbc88f69-nsgcp\" (UID: \"bc0f5544-e98f-4b88-9a17-650fe3f8e803\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"
Jul 27 18:33:11.016666 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.016588    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bc0f5544-e98f-4b88-9a17-650fe3f8e803-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-54cbc88f69-nsgcp\" (UID: \"bc0f5544-e98f-4b88-9a17-650fe3f8e803\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"
Jul 27 18:33:11.016666 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.016616    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c55f\" (UniqueName: \"kubernetes.io/projected/bc0f5544-e98f-4b88-9a17-650fe3f8e803-kube-api-access-2c55f\") pod \"cluster-proxy-proxy-agent-54cbc88f69-nsgcp\" (UID: \"bc0f5544-e98f-4b88-9a17-650fe3f8e803\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"
Jul 27 18:33:11.016946 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.016921    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/727ed70a-3766-4437-8ad9-d87535a6c486-tmp\") pod \"klusterlet-addon-workmgr-f9984ff4c-rhxth\" (UID: \"727ed70a-3766-4437-8ad9-d87535a6c486\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f9984ff4c-rhxth"
Jul 27 18:33:11.017395 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.017366    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc0f5544-e98f-4b88-9a17-650fe3f8e803-additional-service-ca\") pod \"cluster-proxy-proxy-agent-54cbc88f69-nsgcp\" (UID: \"bc0f5544-e98f-4b88-9a17-650fe3f8e803\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"
Jul 27 18:33:11.019268 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.019237    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bc0f5544-e98f-4b88-9a17-650fe3f8e803-ca\") pod \"cluster-proxy-proxy-agent-54cbc88f69-nsgcp\" (UID: \"bc0f5544-e98f-4b88-9a17-650fe3f8e803\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"
Jul 27 18:33:11.019268 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.019265    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bc0f5544-e98f-4b88-9a17-650fe3f8e803-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-54cbc88f69-nsgcp\" (UID: \"bc0f5544-e98f-4b88-9a17-650fe3f8e803\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"
Jul 27 18:33:11.019432 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.019286    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bc0f5544-e98f-4b88-9a17-650fe3f8e803-hub\") pod \"cluster-proxy-proxy-agent-54cbc88f69-nsgcp\" (UID: \"bc0f5544-e98f-4b88-9a17-650fe3f8e803\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"
Jul 27 18:33:11.019571 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.019553    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/727ed70a-3766-4437-8ad9-d87535a6c486-klusterlet-config\") pod \"klusterlet-addon-workmgr-f9984ff4c-rhxth\" (UID: \"727ed70a-3766-4437-8ad9-d87535a6c486\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f9984ff4c-rhxth"
Jul 27 18:33:11.019839 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.019822    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bc0f5544-e98f-4b88-9a17-650fe3f8e803-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-54cbc88f69-nsgcp\" (UID: \"bc0f5544-e98f-4b88-9a17-650fe3f8e803\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"
Jul 27 18:33:11.031436 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.031406    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh6mq\" (UniqueName: \"kubernetes.io/projected/727ed70a-3766-4437-8ad9-d87535a6c486-kube-api-access-mh6mq\") pod \"klusterlet-addon-workmgr-f9984ff4c-rhxth\" (UID: \"727ed70a-3766-4437-8ad9-d87535a6c486\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f9984ff4c-rhxth"
Jul 27 18:33:11.032317 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.032300    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c55f\" (UniqueName: \"kubernetes.io/projected/bc0f5544-e98f-4b88-9a17-650fe3f8e803-kube-api-access-2c55f\") pod \"cluster-proxy-proxy-agent-54cbc88f69-nsgcp\" (UID: \"bc0f5544-e98f-4b88-9a17-650fe3f8e803\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"
Jul 27 18:33:11.123977 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.123945    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f9984ff4c-rhxth"
Jul 27 18:33:11.154328 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.154288    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"
Jul 27 18:33:11.201231 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.200892    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mpdtb"]
Jul 27 18:33:11.286367 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.286291    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f9984ff4c-rhxth"]
Jul 27 18:33:11.290321 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:33:11.290298    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod727ed70a_3766_4437_8ad9_d87535a6c486.slice/crio-deb58bc7d842f9e74087e7ebc85cc8eae0209f500779b48cc03547fd4bb01842 WatchSource:0}: Error finding container deb58bc7d842f9e74087e7ebc85cc8eae0209f500779b48cc03547fd4bb01842: Status 404 returned error can't find the container with id deb58bc7d842f9e74087e7ebc85cc8eae0209f500779b48cc03547fd4bb01842
Jul 27 18:33:11.305437 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.305404    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp"]
Jul 27 18:33:11.309315 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:33:11.309248    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc0f5544_e98f_4b88_9a17_650fe3f8e803.slice/crio-9fac657ad2febf7c2878d4c3f53e1ba12b364cea1e905bf636a16b69ccdf9470 WatchSource:0}: Error finding container 9fac657ad2febf7c2878d4c3f53e1ba12b364cea1e905bf636a16b69ccdf9470: Status 404 returned error can't find the container with id 9fac657ad2febf7c2878d4c3f53e1ba12b364cea1e905bf636a16b69ccdf9470
Jul 27 18:33:11.309809 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.309775    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f9984ff4c-rhxth" event={"ID":"727ed70a-3766-4437-8ad9-d87535a6c486","Type":"ContainerStarted","Data":"deb58bc7d842f9e74087e7ebc85cc8eae0209f500779b48cc03547fd4bb01842"}
Jul 27 18:33:11.311308 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.311264    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr" event={"ID":"5d6f565b-25e3-4db4-89cc-28f0a0cbc890","Type":"ContainerStarted","Data":"37f46f8a47be5029da2b4d4df9e004a1316d6f5828c2befd175f7093528c116d"}
Jul 27 18:33:11.313020 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.312995    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-8jtmk" event={"ID":"0e1a6428-a17e-485f-96b7-6b1a92afb72d","Type":"ContainerStarted","Data":"c2b4f196c438d529a1895643cb0b9814091233c1e2b068a28d8d3497cc0fadd6"}
Jul 27 18:33:11.315652 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.315626    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mpdtb" event={"ID":"c71f0bfd-e960-46e5-b005-2978249db03a","Type":"ContainerStarted","Data":"129ea7dc07348032236d0309092b01ba712d7de9e989d2060987d9b21bc59b6f"}
Jul 27 18:33:11.319236 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.319212    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls\") pod \"dns-default-wzhvk\" (UID: \"2ea9768a-db15-4a87-b27c-1c5e69af52b8\") " pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:33:11.322115 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.322094    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ea9768a-db15-4a87-b27c-1c5e69af52b8-metrics-tls\") pod \"dns-default-wzhvk\" (UID: \"2ea9768a-db15-4a87-b27c-1c5e69af52b8\") " pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:33:11.611251 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.611172    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m8x25\""
Jul 27 18:33:11.617985 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.617951    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:33:11.841227 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:11.841150    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wzhvk"]
Jul 27 18:33:11.849128 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:33:11.849088    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ea9768a_db15_4a87_b27c_1c5e69af52b8.slice/crio-64d6551a77b7bcb8fbb814d3b7c246a65e3b639c148f4ed52a4800b98707336b WatchSource:0}: Error finding container 64d6551a77b7bcb8fbb814d3b7c246a65e3b639c148f4ed52a4800b98707336b: Status 404 returned error can't find the container with id 64d6551a77b7bcb8fbb814d3b7c246a65e3b639c148f4ed52a4800b98707336b
Jul 27 18:33:12.323376 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:12.323304    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp" event={"ID":"bc0f5544-e98f-4b88-9a17-650fe3f8e803","Type":"ContainerStarted","Data":"9fac657ad2febf7c2878d4c3f53e1ba12b364cea1e905bf636a16b69ccdf9470"}
Jul 27 18:33:12.325300 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:12.325218    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wzhvk" event={"ID":"2ea9768a-db15-4a87-b27c-1c5e69af52b8","Type":"ContainerStarted","Data":"64d6551a77b7bcb8fbb814d3b7c246a65e3b639c148f4ed52a4800b98707336b"}
Jul 27 18:33:13.333294 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:13.333236    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr" event={"ID":"5d6f565b-25e3-4db4-89cc-28f0a0cbc890","Type":"ContainerStarted","Data":"4a392906fb442a77e2f8b3c7dfe3186cad1cf55b5ac9ede1674ec973de69c944"}
Jul 27 18:33:13.336403 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:13.336352    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-8jtmk" event={"ID":"0e1a6428-a17e-485f-96b7-6b1a92afb72d","Type":"ContainerStarted","Data":"cd288c1b8223494ea8a04a8d04f17de253364f384b2e86aa3b04f4615ef26977"}
Jul 27 18:33:13.361908 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:13.361824    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rhrcr" podStartSLOduration=32.885615277 podStartE2EDuration="35.361805918s" totalImagesPullingTime="2.476190641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:32:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:33:13.360980337 +0000 UTC m=+101.911273962" watchObservedRunningTime="2026-07-27 18:33:13.361805918 +0000 UTC m=+101.912099552"
Jul 27 18:33:13.419135 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:13.419020    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-8jtmk" podStartSLOduration=32.757191719 podStartE2EDuration="35.419001097s" totalImagesPullingTime="2.661809378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:32:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:33:13.418479384 +0000 UTC m=+101.968773032" watchObservedRunningTime="2026-07-27 18:33:13.419001097 +0000 UTC m=+101.969294727"
Jul 27 18:33:16.228983 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:16.228939    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zqgfw"
Jul 27 18:33:17.354921 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:17.354806    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mpdtb" event={"ID":"c71f0bfd-e960-46e5-b005-2978249db03a","Type":"ContainerStarted","Data":"8968c37c42dda954d7444315066e77a018cd0adc89af3b9eda355b5339c56ee4"}
Jul 27 18:33:17.357204 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:17.357161    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wzhvk" event={"ID":"2ea9768a-db15-4a87-b27c-1c5e69af52b8","Type":"ContainerStarted","Data":"8a10ad8c04fb2a8c420e64aa47662beb9f3581245b8f78196bf82d2682cc499e"}
Jul 27 18:33:17.357377 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:17.357226    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wzhvk" event={"ID":"2ea9768a-db15-4a87-b27c-1c5e69af52b8","Type":"ContainerStarted","Data":"170ec29104d9a7921497bb21bc0cbbeb995c693dba8bab24b3dcaf69cf75eefb"}
Jul 27 18:33:17.357486 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:17.357435    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:33:17.384853 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:17.384792    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wzhvk" podStartSLOduration=65.322795784 podStartE2EDuration="1m10.384768771s" totalImagesPullingTime="5.061972987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:32:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:33:17.38273955 +0000 UTC m=+105.933033184" watchObservedRunningTime="2026-07-27 18:33:17.384768771 +0000 UTC m=+105.935062406"
Jul 27 18:33:23.931885 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:23.931855    2562 scope.go:117] "RemoveContainer" containerID="dad8aef3d3a0e91af375e52b31265612f3667c9f95b8830388e057486f0a8400"
Jul 27 18:33:23.932265 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:33:23.932028    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-rbtsd_openshift-console-operator(2f2170b2-dbd3-4867-9eaf-291ba1ebe412)\"" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd" podUID="2f2170b2-dbd3-4867-9eaf-291ba1ebe412"
Jul 27 18:33:24.391428 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.391380    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mpdtb" event={"ID":"c71f0bfd-e960-46e5-b005-2978249db03a","Type":"ContainerStarted","Data":"dfb9b3c167840cd5a09bf362282d57620831aa44a0da7d99340920217a17ec80"}
Jul 27 18:33:24.395214 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.394582    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp" event={"ID":"bc0f5544-e98f-4b88-9a17-650fe3f8e803","Type":"ContainerStarted","Data":"776a40533ec983d2ff45b6081a850c86fcbe86ec336024a078c210fe312f8953"}
Jul 27 18:33:24.395214 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.394630    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp" event={"ID":"bc0f5544-e98f-4b88-9a17-650fe3f8e803","Type":"ContainerStarted","Data":"85277880f6cf651e7ed4fa3b6b37cb2616639a4c727f195e704df22f7d9ed46d"}
Jul 27 18:33:24.395214 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.394648    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp" event={"ID":"bc0f5544-e98f-4b88-9a17-650fe3f8e803","Type":"ContainerStarted","Data":"a1a1ecb2ecd9e3c31915586b949076d8dc6c204474da711fbc1110f02b887cc9"}
Jul 27 18:33:24.396710 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.396677    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f9984ff4c-rhxth" event={"ID":"727ed70a-3766-4437-8ad9-d87535a6c486","Type":"ContainerStarted","Data":"53a4a8c40c8377a22a6b19a65eba4913e74c4343e46ea583d2c9b5e22366ecde"}
Jul 27 18:33:24.397190 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.397172    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f9984ff4c-rhxth"
Jul 27 18:33:24.399105 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.399081    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f9984ff4c-rhxth"
Jul 27 18:33:24.419537 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.419484    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54cbc88f69-nsgcp" podStartSLOduration=2.025047455 podStartE2EDuration="14.419467978s" totalImagesPullingTime="12.394420523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:33:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:33:24.418680267 +0000 UTC m=+112.968973948" watchObservedRunningTime="2026-07-27 18:33:24.419467978 +0000 UTC m=+112.969761612"
Jul 27 18:33:24.441666 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.441612    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f9984ff4c-rhxth" podStartSLOduration=2.016559919 podStartE2EDuration="14.441590991s" totalImagesPullingTime="12.425031072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:33:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:33:24.439981971 +0000 UTC m=+112.990275605" watchObservedRunningTime="2026-07-27 18:33:24.441590991 +0000 UTC m=+112.991884623"
Jul 27 18:33:24.651100 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.650945    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-27krt"]
Jul 27 18:33:24.653538 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.653514    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.656357 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.656278    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 27 18:33:24.656730 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.656709    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-4q95x\""
Jul 27 18:33:24.656974 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.656924    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 27 18:33:24.657152 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.657137    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 27 18:33:24.657308 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.657294    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 27 18:33:24.841329 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.840955    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/285f4d97-01a4-43a9-a067-abcec72f14a2-node-exporter-wtmp\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.841329 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.841027    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/285f4d97-01a4-43a9-a067-abcec72f14a2-metrics-client-ca\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.841329 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.841046    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/285f4d97-01a4-43a9-a067-abcec72f14a2-root\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.841329 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.841073    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/285f4d97-01a4-43a9-a067-abcec72f14a2-node-exporter-textfile\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.841329 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.841087    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/285f4d97-01a4-43a9-a067-abcec72f14a2-node-exporter-tls\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.841329 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.841118    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/285f4d97-01a4-43a9-a067-abcec72f14a2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.841329 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.841161    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28zwr\" (UniqueName: \"kubernetes.io/projected/285f4d97-01a4-43a9-a067-abcec72f14a2-kube-api-access-28zwr\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.841329 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.841203    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/285f4d97-01a4-43a9-a067-abcec72f14a2-node-exporter-accelerators-collector-config\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.841329 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.841250    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/285f4d97-01a4-43a9-a067-abcec72f14a2-sys\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.941905 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.941793    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/285f4d97-01a4-43a9-a067-abcec72f14a2-root\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.941905 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.941849    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/285f4d97-01a4-43a9-a067-abcec72f14a2-node-exporter-textfile\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.941905 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.941901    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/285f4d97-01a4-43a9-a067-abcec72f14a2-node-exporter-tls\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.942471 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.941926    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/285f4d97-01a4-43a9-a067-abcec72f14a2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.942471 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.941964    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28zwr\" (UniqueName: \"kubernetes.io/projected/285f4d97-01a4-43a9-a067-abcec72f14a2-kube-api-access-28zwr\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.942471 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.942004    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/285f4d97-01a4-43a9-a067-abcec72f14a2-node-exporter-accelerators-collector-config\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.942471 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.942058    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/285f4d97-01a4-43a9-a067-abcec72f14a2-sys\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.942471 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.942104    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/285f4d97-01a4-43a9-a067-abcec72f14a2-node-exporter-wtmp\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.942471 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.942126    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/285f4d97-01a4-43a9-a067-abcec72f14a2-metrics-client-ca\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.942984 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.942961    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/285f4d97-01a4-43a9-a067-abcec72f14a2-metrics-client-ca\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.943059 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.943045    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/285f4d97-01a4-43a9-a067-abcec72f14a2-root\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.943290 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.943271    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/285f4d97-01a4-43a9-a067-abcec72f14a2-node-exporter-textfile\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.943400 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:33:24.943374    2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 27 18:33:24.943461 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:33:24.943447    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/285f4d97-01a4-43a9-a067-abcec72f14a2-node-exporter-tls podName:285f4d97-01a4-43a9-a067-abcec72f14a2 nodeName:}" failed. No retries permitted until 2026-07-27 18:33:25.443428847 +0000 UTC m=+113.993722463 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/285f4d97-01a4-43a9-a067-abcec72f14a2-node-exporter-tls") pod "node-exporter-27krt" (UID: "285f4d97-01a4-43a9-a067-abcec72f14a2") : secret "node-exporter-tls" not found
Jul 27 18:33:24.944509 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.944130    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/285f4d97-01a4-43a9-a067-abcec72f14a2-sys\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.944509 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.944254    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/285f4d97-01a4-43a9-a067-abcec72f14a2-node-exporter-wtmp\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.944687 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.944568    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/285f4d97-01a4-43a9-a067-abcec72f14a2-node-exporter-accelerators-collector-config\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.946463 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.946441    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/285f4d97-01a4-43a9-a067-abcec72f14a2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:24.967745 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:24.967709    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28zwr\" (UniqueName: \"kubernetes.io/projected/285f4d97-01a4-43a9-a067-abcec72f14a2-kube-api-access-28zwr\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:25.447907 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:25.447842    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/285f4d97-01a4-43a9-a067-abcec72f14a2-node-exporter-tls\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:25.450934 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:25.450906    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/285f4d97-01a4-43a9-a067-abcec72f14a2-node-exporter-tls\") pod \"node-exporter-27krt\" (UID: \"285f4d97-01a4-43a9-a067-abcec72f14a2\") " pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:25.567037 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:25.566960    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-27krt"
Jul 27 18:33:25.579291 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:33:25.579212    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod285f4d97_01a4_43a9_a067_abcec72f14a2.slice/crio-19dcdc45060c505c1092e092c859663e73a20faf37e841e278d1b0353535d2a9 WatchSource:0}: Error finding container 19dcdc45060c505c1092e092c859663e73a20faf37e841e278d1b0353535d2a9: Status 404 returned error can't find the container with id 19dcdc45060c505c1092e092c859663e73a20faf37e841e278d1b0353535d2a9
Jul 27 18:33:26.406174 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:26.406096    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-27krt" event={"ID":"285f4d97-01a4-43a9-a067-abcec72f14a2","Type":"ContainerStarted","Data":"19dcdc45060c505c1092e092c859663e73a20faf37e841e278d1b0353535d2a9"}
Jul 27 18:33:27.364955 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:27.364922    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wzhvk"
Jul 27 18:33:28.415275 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:28.415150    2562 generic.go:356] "Generic (PLEG): container finished" podID="285f4d97-01a4-43a9-a067-abcec72f14a2" containerID="4101ab9231cb675da861ed861845b7f4f453cb7e5953a969c5e577b6c830badf" exitCode=0
Jul 27 18:33:28.415275 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:28.415239    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-27krt" event={"ID":"285f4d97-01a4-43a9-a067-abcec72f14a2","Type":"ContainerDied","Data":"4101ab9231cb675da861ed861845b7f4f453cb7e5953a969c5e577b6c830badf"}
Jul 27 18:33:28.417351 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:28.417328    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mpdtb" event={"ID":"c71f0bfd-e960-46e5-b005-2978249db03a","Type":"ContainerStarted","Data":"4aaf49e8e44abcf280b2263208e36960ab1163ef89f3e354e77b0f40d4b6b069"}
Jul 27 18:33:28.536830 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:28.536785    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mpdtb" podStartSLOduration=2.433107381 podStartE2EDuration="18.53677005s" totalImagesPullingTime="16.103662669s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 18:33:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:33:28.53529522 +0000 UTC m=+117.085588870" watchObservedRunningTime="2026-07-27 18:33:28.53677005 +0000 UTC m=+117.087063682"
Jul 27 18:33:29.423208 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:29.423167    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-27krt" event={"ID":"285f4d97-01a4-43a9-a067-abcec72f14a2","Type":"ContainerStarted","Data":"628856d2459c6ae177dec74baccd54c3017f9b1ef8b831a14663cf1f3e552ff7"}
Jul 27 18:33:29.423619 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:29.423224    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-27krt" event={"ID":"285f4d97-01a4-43a9-a067-abcec72f14a2","Type":"ContainerStarted","Data":"66804d247e6fb57f615b1c70d92f4edeee393017526859927355c9db1cd6271f"}
Jul 27 18:33:29.448467 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:29.448408    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-27krt" podStartSLOduration=2.9830668510000002 podStartE2EDuration="5.448385447s" totalImagesPullingTime="2.390643674s" totalInitContainerRuntime="74.674922ms" isStatefulPod=true podCreationTimestamp="2026-07-27 18:33:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:33:29.446072986 +0000 UTC m=+117.996366631" watchObservedRunningTime="2026-07-27 18:33:29.448385447 +0000 UTC m=+117.998679086"
Jul 27 18:33:36.931409 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:36.931371    2562 scope.go:117] "RemoveContainer" containerID="dad8aef3d3a0e91af375e52b31265612f3667c9f95b8830388e057486f0a8400"
Jul 27 18:33:37.452751 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:37.452718    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 18:33:37.452944 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:37.452772    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd" event={"ID":"2f2170b2-dbd3-4867-9eaf-291ba1ebe412","Type":"ContainerStarted","Data":"ff0777e8f706de04ecb471e937ff78e2331c84fdafe70505dad60ba600bb8b4e"}
Jul 27 18:33:37.453095 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:37.453075    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd"
Jul 27 18:33:37.472267 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:37.472212    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd" podStartSLOduration=53.971266831 podStartE2EDuration="59.472195256s" totalImagesPullingTime="5.500928425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:32:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:33:37.47141451 +0000 UTC m=+126.021708186" watchObservedRunningTime="2026-07-27 18:33:37.472195256 +0000 UTC m=+126.022488889"
Jul 27 18:33:38.453153 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:38.453109    2562 patch_prober.go:28] interesting pod/console-operator-584dcb6b54-rbtsd container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.133.0.9:8443/readyz\": context deadline exceeded" start-of-body=
Jul 27 18:33:38.453608 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:38.453174    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd" podUID="2f2170b2-dbd3-4867-9eaf-291ba1ebe412" containerName="console-operator" probeResult="failure" output="Get \"https://10.133.0.9:8443/readyz\": context deadline exceeded"
Jul 27 18:33:38.489705 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:38.489672    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-rbtsd"
Jul 27 18:33:40.671889 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:40.671840    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs\") pod \"network-metrics-daemon-mhtmr\" (UID: \"88f67acb-91af-4850-bb4d-81ff499bc672\") " pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:33:40.674589 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:40.674562    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f67acb-91af-4850-bb4d-81ff499bc672-metrics-certs\") pod \"network-metrics-daemon-mhtmr\" (UID: \"88f67acb-91af-4850-bb4d-81ff499bc672\") " pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:33:40.945941 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:40.945841    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j4wf8\""
Jul 27 18:33:40.953518 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:40.953489    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mhtmr"
Jul 27 18:33:41.088701 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:41.088673    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mhtmr"]
Jul 27 18:33:41.090723 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:33:41.090687    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88f67acb_91af_4850_bb4d_81ff499bc672.slice/crio-fcee4cad39d5e721348eeca6846336cc31beb885219faa146a7258cfd300c760 WatchSource:0}: Error finding container fcee4cad39d5e721348eeca6846336cc31beb885219faa146a7258cfd300c760: Status 404 returned error can't find the container with id fcee4cad39d5e721348eeca6846336cc31beb885219faa146a7258cfd300c760
Jul 27 18:33:41.468071 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:41.468037    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mhtmr" event={"ID":"88f67acb-91af-4850-bb4d-81ff499bc672","Type":"ContainerStarted","Data":"fcee4cad39d5e721348eeca6846336cc31beb885219faa146a7258cfd300c760"}
Jul 27 18:33:42.472986 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:42.472953    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mhtmr" event={"ID":"88f67acb-91af-4850-bb4d-81ff499bc672","Type":"ContainerStarted","Data":"059851d10746a31b10b4d1cd281e70bc70575ea883fb572c8674ac22d7711966"}
Jul 27 18:33:42.473385 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:42.473008    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mhtmr" event={"ID":"88f67acb-91af-4850-bb4d-81ff499bc672","Type":"ContainerStarted","Data":"78e5c08c2799b213c7a3eea1890b055b23254940e8c699657c0bb612146c24b5"}
Jul 27 18:33:42.490426 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:33:42.490368    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mhtmr" podStartSLOduration=129.60109991 podStartE2EDuration="2m10.490348242s" totalImagesPullingTime="889.248332ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:31:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:33:42.490098007 +0000 UTC m=+131.040391641" watchObservedRunningTime="2026-07-27 18:33:42.490348242 +0000 UTC m=+131.040641876"
Jul 27 18:34:01.544263 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:34:01.544198    2562 generic.go:356] "Generic (PLEG): container finished" podID="476d7150-35a6-4b7d-beee-1fbac9faa00f" containerID="f2893086d4162cde9e3fbae6319dbb05577312c30be3c38281987e2b3a8fab98" exitCode=0
Jul 27 18:34:01.544686 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:34:01.544277    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-fwf4j" event={"ID":"476d7150-35a6-4b7d-beee-1fbac9faa00f","Type":"ContainerDied","Data":"f2893086d4162cde9e3fbae6319dbb05577312c30be3c38281987e2b3a8fab98"}
Jul 27 18:34:01.544686 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:34:01.544639    2562 scope.go:117] "RemoveContainer" containerID="f2893086d4162cde9e3fbae6319dbb05577312c30be3c38281987e2b3a8fab98"
Jul 27 18:34:02.549815 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:34:02.549783    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-fwf4j" event={"ID":"476d7150-35a6-4b7d-beee-1fbac9faa00f","Type":"ContainerStarted","Data":"95bf56bab60427724cec3d182581067f90b33cec0363981d9455d8426e699895"}
Jul 27 18:36:31.755477 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:31.755447    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 18:36:31.759257 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:31.759236    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 18:36:31.765721 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:31.765695    2562 kubelet.go:1648] "Image garbage collection succeeded"
Jul 27 18:36:35.891041 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:35.891001    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8jtcn"]
Jul 27 18:36:35.894201 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:35.894184    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8jtcn"
Jul 27 18:36:35.896663 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:35.896638    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 27 18:36:35.897157 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:35.897138    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-sw8hp\""
Jul 27 18:36:35.897239 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:35.897198    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 27 18:36:35.906132 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:35.906105    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8jtcn"]
Jul 27 18:36:36.003343 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:36.003306    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7453eef-b4bb-4849-8e3f-160d1fe97553-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8jtcn\" (UID: \"f7453eef-b4bb-4849-8e3f-160d1fe97553\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8jtcn"
Jul 27 18:36:36.003534 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:36.003358    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp8xj\" (UniqueName: \"kubernetes.io/projected/f7453eef-b4bb-4849-8e3f-160d1fe97553-kube-api-access-hp8xj\") pod \"cert-manager-cainjector-559cdd8c87-8jtcn\" (UID: \"f7453eef-b4bb-4849-8e3f-160d1fe97553\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8jtcn"
Jul 27 18:36:36.103839 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:36.103795    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7453eef-b4bb-4849-8e3f-160d1fe97553-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8jtcn\" (UID: \"f7453eef-b4bb-4849-8e3f-160d1fe97553\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8jtcn"
Jul 27 18:36:36.104067 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:36.103854    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp8xj\" (UniqueName: \"kubernetes.io/projected/f7453eef-b4bb-4849-8e3f-160d1fe97553-kube-api-access-hp8xj\") pod \"cert-manager-cainjector-559cdd8c87-8jtcn\" (UID: \"f7453eef-b4bb-4849-8e3f-160d1fe97553\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8jtcn"
Jul 27 18:36:36.113072 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:36.113038    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7453eef-b4bb-4849-8e3f-160d1fe97553-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8jtcn\" (UID: \"f7453eef-b4bb-4849-8e3f-160d1fe97553\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8jtcn"
Jul 27 18:36:36.113180 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:36.113141    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp8xj\" (UniqueName: \"kubernetes.io/projected/f7453eef-b4bb-4849-8e3f-160d1fe97553-kube-api-access-hp8xj\") pod \"cert-manager-cainjector-559cdd8c87-8jtcn\" (UID: \"f7453eef-b4bb-4849-8e3f-160d1fe97553\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8jtcn"
Jul 27 18:36:36.203217 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:36.203132    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8jtcn"
Jul 27 18:36:36.336839 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:36.336806    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8jtcn"]
Jul 27 18:36:36.339672 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:36:36.339651    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7453eef_b4bb_4849_8e3f_160d1fe97553.slice/crio-ff75317b04928bb0d5cb86ce6db3da16992fe4282598533ce1732dad916689c6 WatchSource:0}: Error finding container ff75317b04928bb0d5cb86ce6db3da16992fe4282598533ce1732dad916689c6: Status 404 returned error can't find the container with id ff75317b04928bb0d5cb86ce6db3da16992fe4282598533ce1732dad916689c6
Jul 27 18:36:36.341592 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:36.341569    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 18:36:37.072481 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:37.072443    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8jtcn" event={"ID":"f7453eef-b4bb-4849-8e3f-160d1fe97553","Type":"ContainerStarted","Data":"ff75317b04928bb0d5cb86ce6db3da16992fe4282598533ce1732dad916689c6"}
Jul 27 18:36:40.086120 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:40.086083    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8jtcn" event={"ID":"f7453eef-b4bb-4849-8e3f-160d1fe97553","Type":"ContainerStarted","Data":"e3a19d9be981368f3bae2453e7063fe453c994b8776b405c57c840b91795594f"}
Jul 27 18:36:40.110017 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:40.109964    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8jtcn" podStartSLOduration=2.053750731 podStartE2EDuration="5.109945265s" totalImagesPullingTime="3.056194534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:36:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:36:40.109090132 +0000 UTC m=+308.659383784" watchObservedRunningTime="2026-07-27 18:36:40.109945265 +0000 UTC m=+308.660238909"
Jul 27 18:36:59.633175 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.633139    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"]
Jul 27 18:36:59.636728 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.636697    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"
Jul 27 18:36:59.640613 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.640576    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 27 18:36:59.641157 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.641132    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 27 18:36:59.641271 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.641203    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-d46ft\""
Jul 27 18:36:59.641933 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.641912    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 27 18:36:59.642150 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.642128    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 27 18:36:59.668938 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.668905    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"]
Jul 27 18:36:59.699354 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.699311    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/760a81e6-82a4-4ae0-8407-e3536402cca0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk\" (UID: \"760a81e6-82a4-4ae0-8407-e3536402cca0\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"
Jul 27 18:36:59.699565 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.699379    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/760a81e6-82a4-4ae0-8407-e3536402cca0-webhook-cert\") pod \"opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk\" (UID: \"760a81e6-82a4-4ae0-8407-e3536402cca0\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"
Jul 27 18:36:59.699565 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.699404    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/760a81e6-82a4-4ae0-8407-e3536402cca0-tmp\") pod \"opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk\" (UID: \"760a81e6-82a4-4ae0-8407-e3536402cca0\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"
Jul 27 18:36:59.699565 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.699430    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/760a81e6-82a4-4ae0-8407-e3536402cca0-manifests\") pod \"opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk\" (UID: \"760a81e6-82a4-4ae0-8407-e3536402cca0\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"
Jul 27 18:36:59.699565 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.699467    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58xjk\" (UniqueName: \"kubernetes.io/projected/760a81e6-82a4-4ae0-8407-e3536402cca0-kube-api-access-58xjk\") pod \"opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk\" (UID: \"760a81e6-82a4-4ae0-8407-e3536402cca0\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"
Jul 27 18:36:59.800782 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.800745    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/760a81e6-82a4-4ae0-8407-e3536402cca0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk\" (UID: \"760a81e6-82a4-4ae0-8407-e3536402cca0\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"
Jul 27 18:36:59.800990 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.800789    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/760a81e6-82a4-4ae0-8407-e3536402cca0-webhook-cert\") pod \"opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk\" (UID: \"760a81e6-82a4-4ae0-8407-e3536402cca0\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"
Jul 27 18:36:59.800990 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.800810    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/760a81e6-82a4-4ae0-8407-e3536402cca0-tmp\") pod \"opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk\" (UID: \"760a81e6-82a4-4ae0-8407-e3536402cca0\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"
Jul 27 18:36:59.800990 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.800827    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/760a81e6-82a4-4ae0-8407-e3536402cca0-manifests\") pod \"opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk\" (UID: \"760a81e6-82a4-4ae0-8407-e3536402cca0\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"
Jul 27 18:36:59.800990 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.800861    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58xjk\" (UniqueName: \"kubernetes.io/projected/760a81e6-82a4-4ae0-8407-e3536402cca0-kube-api-access-58xjk\") pod \"opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk\" (UID: \"760a81e6-82a4-4ae0-8407-e3536402cca0\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"
Jul 27 18:36:59.801347 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.801323    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/760a81e6-82a4-4ae0-8407-e3536402cca0-manifests\") pod \"opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk\" (UID: \"760a81e6-82a4-4ae0-8407-e3536402cca0\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"
Jul 27 18:36:59.803386 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.803356    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/760a81e6-82a4-4ae0-8407-e3536402cca0-tmp\") pod \"opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk\" (UID: \"760a81e6-82a4-4ae0-8407-e3536402cca0\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"
Jul 27 18:36:59.803539 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.803507    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/760a81e6-82a4-4ae0-8407-e3536402cca0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk\" (UID: \"760a81e6-82a4-4ae0-8407-e3536402cca0\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"
Jul 27 18:36:59.803599 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.803566    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/760a81e6-82a4-4ae0-8407-e3536402cca0-webhook-cert\") pod \"opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk\" (UID: \"760a81e6-82a4-4ae0-8407-e3536402cca0\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"
Jul 27 18:36:59.813567 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.813540    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58xjk\" (UniqueName: \"kubernetes.io/projected/760a81e6-82a4-4ae0-8407-e3536402cca0-kube-api-access-58xjk\") pod \"opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk\" (UID: \"760a81e6-82a4-4ae0-8407-e3536402cca0\") " pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"
Jul 27 18:36:59.947675 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:36:59.947578    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"
Jul 27 18:37:00.090702 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:00.090670    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"]
Jul 27 18:37:00.093385 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:37:00.093358    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod760a81e6_82a4_4ae0_8407_e3536402cca0.slice/crio-5769a9088aa55c94081e37bb5ac1cf304ea982bfdc32417416b09cb3d8d131bf WatchSource:0}: Error finding container 5769a9088aa55c94081e37bb5ac1cf304ea982bfdc32417416b09cb3d8d131bf: Status 404 returned error can't find the container with id 5769a9088aa55c94081e37bb5ac1cf304ea982bfdc32417416b09cb3d8d131bf
Jul 27 18:37:00.156026 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:00.155988    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk" event={"ID":"760a81e6-82a4-4ae0-8407-e3536402cca0","Type":"ContainerStarted","Data":"5769a9088aa55c94081e37bb5ac1cf304ea982bfdc32417416b09cb3d8d131bf"}
Jul 27 18:37:04.175007 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:04.174969    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk" event={"ID":"760a81e6-82a4-4ae0-8407-e3536402cca0","Type":"ContainerStarted","Data":"33c16eb9b89fa7e0e06335aa5ee7a3382fee67f9d8366de664ea3ae783625655"}
Jul 27 18:37:05.180950 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:05.180906    2562 generic.go:356] "Generic (PLEG): container finished" podID="760a81e6-82a4-4ae0-8407-e3536402cca0" containerID="33c16eb9b89fa7e0e06335aa5ee7a3382fee67f9d8366de664ea3ae783625655" exitCode=0
Jul 27 18:37:05.181345 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:05.180961    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk" event={"ID":"760a81e6-82a4-4ae0-8407-e3536402cca0","Type":"ContainerDied","Data":"33c16eb9b89fa7e0e06335aa5ee7a3382fee67f9d8366de664ea3ae783625655"}
Jul 27 18:37:06.186533 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:06.186495    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk" event={"ID":"760a81e6-82a4-4ae0-8407-e3536402cca0","Type":"ContainerStarted","Data":"a65c6d0e82b57280d332a2ce7c06be1f3fdbd361fa1f76f37f69efc159a8c1a9"}
Jul 27 18:37:06.186945 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:06.186601    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"
Jul 27 18:37:06.225667 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:06.225610    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk" podStartSLOduration=2.477499264 podStartE2EDuration="7.225592035s" totalImagesPullingTime="3.68997702s" totalInitContainerRuntime="1.058115751s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:36:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:37:06.224019645 +0000 UTC m=+334.774313280" watchObservedRunningTime="2026-07-27 18:37:06.225592035 +0000 UTC m=+334.775885659"
Jul 27 18:37:17.192912 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:17.192861    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk"
Jul 27 18:37:19.749169 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.749135    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-55894cd494-n5g7d"]
Jul 27 18:37:19.757232 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.757207    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-55894cd494-n5g7d"
Jul 27 18:37:19.760321 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.760299    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 27 18:37:19.760459 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.760360    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 27 18:37:19.760459 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.760382    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 27 18:37:19.760587 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.760493    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ckxx\" (UniqueName: \"kubernetes.io/projected/6384df57-3269-47c8-8187-e65b70d8b57a-kube-api-access-5ckxx\") pod \"kube-auth-proxy-55894cd494-n5g7d\" (UID: \"6384df57-3269-47c8-8187-e65b70d8b57a\") " pod="openshift-ingress/kube-auth-proxy-55894cd494-n5g7d"
Jul 27 18:37:19.760587 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.760568    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6384df57-3269-47c8-8187-e65b70d8b57a-tls-certs\") pod \"kube-auth-proxy-55894cd494-n5g7d\" (UID: \"6384df57-3269-47c8-8187-e65b70d8b57a\") " pod="openshift-ingress/kube-auth-proxy-55894cd494-n5g7d"
Jul 27 18:37:19.760658 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.760587    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6384df57-3269-47c8-8187-e65b70d8b57a-tmp\") pod \"kube-auth-proxy-55894cd494-n5g7d\" (UID: \"6384df57-3269-47c8-8187-e65b70d8b57a\") " pod="openshift-ingress/kube-auth-proxy-55894cd494-n5g7d"
Jul 27 18:37:19.761103 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.761081    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 27 18:37:19.761103 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.761091    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-gng6z\""
Jul 27 18:37:19.785963 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.780314    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-55894cd494-n5g7d"]
Jul 27 18:37:19.861001 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.860967    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6384df57-3269-47c8-8187-e65b70d8b57a-tls-certs\") pod \"kube-auth-proxy-55894cd494-n5g7d\" (UID: \"6384df57-3269-47c8-8187-e65b70d8b57a\") " pod="openshift-ingress/kube-auth-proxy-55894cd494-n5g7d"
Jul 27 18:37:19.861001 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.860998    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6384df57-3269-47c8-8187-e65b70d8b57a-tmp\") pod \"kube-auth-proxy-55894cd494-n5g7d\" (UID: \"6384df57-3269-47c8-8187-e65b70d8b57a\") " pod="openshift-ingress/kube-auth-proxy-55894cd494-n5g7d"
Jul 27 18:37:19.861237 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.861025    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ckxx\" (UniqueName: \"kubernetes.io/projected/6384df57-3269-47c8-8187-e65b70d8b57a-kube-api-access-5ckxx\") pod \"kube-auth-proxy-55894cd494-n5g7d\" (UID: \"6384df57-3269-47c8-8187-e65b70d8b57a\") " pod="openshift-ingress/kube-auth-proxy-55894cd494-n5g7d"
Jul 27 18:37:19.863304 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.863277    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6384df57-3269-47c8-8187-e65b70d8b57a-tmp\") pod \"kube-auth-proxy-55894cd494-n5g7d\" (UID: \"6384df57-3269-47c8-8187-e65b70d8b57a\") " pod="openshift-ingress/kube-auth-proxy-55894cd494-n5g7d"
Jul 27 18:37:19.863615 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.863592    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6384df57-3269-47c8-8187-e65b70d8b57a-tls-certs\") pod \"kube-auth-proxy-55894cd494-n5g7d\" (UID: \"6384df57-3269-47c8-8187-e65b70d8b57a\") " pod="openshift-ingress/kube-auth-proxy-55894cd494-n5g7d"
Jul 27 18:37:19.877559 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.877530    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ckxx\" (UniqueName: \"kubernetes.io/projected/6384df57-3269-47c8-8187-e65b70d8b57a-kube-api-access-5ckxx\") pod \"kube-auth-proxy-55894cd494-n5g7d\" (UID: \"6384df57-3269-47c8-8187-e65b70d8b57a\") " pod="openshift-ingress/kube-auth-proxy-55894cd494-n5g7d"
Jul 27 18:37:19.953570 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.953531    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"]
Jul 27 18:37:19.957078 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.957054    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"
Jul 27 18:37:19.961436 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.961412    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d0926f63-f6cc-41e9-a804-ca6ccd217df1-cert\") pod \"lws-controller-manager-5c45656c54-qhv7b\" (UID: \"d0926f63-f6cc-41e9-a804-ca6ccd217df1\") " pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"
Jul 27 18:37:19.961595 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.961455    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srdg5\" (UniqueName: \"kubernetes.io/projected/d0926f63-f6cc-41e9-a804-ca6ccd217df1-kube-api-access-srdg5\") pod \"lws-controller-manager-5c45656c54-qhv7b\" (UID: \"d0926f63-f6cc-41e9-a804-ca6ccd217df1\") " pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"
Jul 27 18:37:19.961595 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.961485    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d0926f63-f6cc-41e9-a804-ca6ccd217df1-metrics-cert\") pod \"lws-controller-manager-5c45656c54-qhv7b\" (UID: \"d0926f63-f6cc-41e9-a804-ca6ccd217df1\") " pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"
Jul 27 18:37:19.961595 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.961556    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d0926f63-f6cc-41e9-a804-ca6ccd217df1-manager-config\") pod \"lws-controller-manager-5c45656c54-qhv7b\" (UID: \"d0926f63-f6cc-41e9-a804-ca6ccd217df1\") " pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"
Jul 27 18:37:19.968923 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.968897    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 27 18:37:19.969031 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.969008    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 27 18:37:19.969098 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.969080    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-5swx8\""
Jul 27 18:37:19.971461 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:37:19.971443    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-132-186.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-132-186.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Jul 27 18:37:19.971799 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.971787    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 27 18:37:19.971936 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.971920    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 27 18:37:19.996573 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:19.996535    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"]
Jul 27 18:37:20.061905 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:20.061809    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srdg5\" (UniqueName: \"kubernetes.io/projected/d0926f63-f6cc-41e9-a804-ca6ccd217df1-kube-api-access-srdg5\") pod \"lws-controller-manager-5c45656c54-qhv7b\" (UID: \"d0926f63-f6cc-41e9-a804-ca6ccd217df1\") " pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"
Jul 27 18:37:20.061905 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:20.061853    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d0926f63-f6cc-41e9-a804-ca6ccd217df1-metrics-cert\") pod \"lws-controller-manager-5c45656c54-qhv7b\" (UID: \"d0926f63-f6cc-41e9-a804-ca6ccd217df1\") " pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"
Jul 27 18:37:20.061905 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:20.061899    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d0926f63-f6cc-41e9-a804-ca6ccd217df1-manager-config\") pod \"lws-controller-manager-5c45656c54-qhv7b\" (UID: \"d0926f63-f6cc-41e9-a804-ca6ccd217df1\") " pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"
Jul 27 18:37:20.062130 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:20.061932    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d0926f63-f6cc-41e9-a804-ca6ccd217df1-cert\") pod \"lws-controller-manager-5c45656c54-qhv7b\" (UID: \"d0926f63-f6cc-41e9-a804-ca6ccd217df1\") " pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"
Jul 27 18:37:20.062622 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:20.062604    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d0926f63-f6cc-41e9-a804-ca6ccd217df1-manager-config\") pod \"lws-controller-manager-5c45656c54-qhv7b\" (UID: \"d0926f63-f6cc-41e9-a804-ca6ccd217df1\") " pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"
Jul 27 18:37:20.064538 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:20.064511    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d0926f63-f6cc-41e9-a804-ca6ccd217df1-cert\") pod \"lws-controller-manager-5c45656c54-qhv7b\" (UID: \"d0926f63-f6cc-41e9-a804-ca6ccd217df1\") " pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"
Jul 27 18:37:20.064642 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:20.064560    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d0926f63-f6cc-41e9-a804-ca6ccd217df1-metrics-cert\") pod \"lws-controller-manager-5c45656c54-qhv7b\" (UID: \"d0926f63-f6cc-41e9-a804-ca6ccd217df1\") " pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"
Jul 27 18:37:20.066448 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:20.066428    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-55894cd494-n5g7d"
Jul 27 18:37:20.236766 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:20.236727    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-55894cd494-n5g7d"]
Jul 27 18:37:20.242223 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:37:20.242190    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6384df57_3269_47c8_8187_e65b70d8b57a.slice/crio-1fc25105d853e11463428c1677ac19c659e36feea3af35600d56d28e9588412e WatchSource:0}: Error finding container 1fc25105d853e11463428c1677ac19c659e36feea3af35600d56d28e9588412e: Status 404 returned error can't find the container with id 1fc25105d853e11463428c1677ac19c659e36feea3af35600d56d28e9588412e
Jul 27 18:37:21.077984 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:37:21.077944    2562 projected.go:291] Couldn't get configMap openshift-lws-operator/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Jul 27 18:37:21.078442 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:37:21.077994    2562 projected.go:196] Error preparing data for projected volume kube-api-access-srdg5 for pod openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b: failed to sync configmap cache: timed out waiting for the condition
Jul 27 18:37:21.078442 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:37:21.078089    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d0926f63-f6cc-41e9-a804-ca6ccd217df1-kube-api-access-srdg5 podName:d0926f63-f6cc-41e9-a804-ca6ccd217df1 nodeName:}" failed. No retries permitted until 2026-07-27 18:37:21.578066996 +0000 UTC m=+350.128360627 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-srdg5" (UniqueName: "kubernetes.io/projected/d0926f63-f6cc-41e9-a804-ca6ccd217df1-kube-api-access-srdg5") pod "lws-controller-manager-5c45656c54-qhv7b" (UID: "d0926f63-f6cc-41e9-a804-ca6ccd217df1") : failed to sync configmap cache: timed out waiting for the condition
Jul 27 18:37:21.239456 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:21.239426    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-55894cd494-n5g7d" event={"ID":"6384df57-3269-47c8-8187-e65b70d8b57a","Type":"ContainerStarted","Data":"1fc25105d853e11463428c1677ac19c659e36feea3af35600d56d28e9588412e"}
Jul 27 18:37:21.506697 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:21.506661    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 27 18:37:21.675003 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:21.674963    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srdg5\" (UniqueName: \"kubernetes.io/projected/d0926f63-f6cc-41e9-a804-ca6ccd217df1-kube-api-access-srdg5\") pod \"lws-controller-manager-5c45656c54-qhv7b\" (UID: \"d0926f63-f6cc-41e9-a804-ca6ccd217df1\") " pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"
Jul 27 18:37:21.677604 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:21.677573    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srdg5\" (UniqueName: \"kubernetes.io/projected/d0926f63-f6cc-41e9-a804-ca6ccd217df1-kube-api-access-srdg5\") pod \"lws-controller-manager-5c45656c54-qhv7b\" (UID: \"d0926f63-f6cc-41e9-a804-ca6ccd217df1\") " pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"
Jul 27 18:37:21.766148 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:21.766055    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"
Jul 27 18:37:21.947286 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:21.947258    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"]
Jul 27 18:37:21.951655 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:37:21.951592    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0926f63_f6cc_41e9_a804_ca6ccd217df1.slice/crio-b72d0dd7543d39cb11d532444ef460c9c70847493e05a2dda1c5d649af269af7 WatchSource:0}: Error finding container b72d0dd7543d39cb11d532444ef460c9c70847493e05a2dda1c5d649af269af7: Status 404 returned error can't find the container with id b72d0dd7543d39cb11d532444ef460c9c70847493e05a2dda1c5d649af269af7
Jul 27 18:37:22.245204 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:22.245163    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b" event={"ID":"d0926f63-f6cc-41e9-a804-ca6ccd217df1","Type":"ContainerStarted","Data":"b72d0dd7543d39cb11d532444ef460c9c70847493e05a2dda1c5d649af269af7"}
Jul 27 18:37:24.258580 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:24.258536    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-55894cd494-n5g7d" event={"ID":"6384df57-3269-47c8-8187-e65b70d8b57a","Type":"ContainerStarted","Data":"2dae915b385c8d85bb3b47d4bdd58e9ef1b229986d7714ecfe1e4e34faca1637"}
Jul 27 18:37:24.280179 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:24.280127    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-55894cd494-n5g7d" podStartSLOduration=2.353754772 podStartE2EDuration="5.280110441s" totalImagesPullingTime="2.926355669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:37:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:37:24.278817195 +0000 UTC m=+352.829110828" watchObservedRunningTime="2026-07-27 18:37:24.280110441 +0000 UTC m=+352.830404073"
Jul 27 18:37:25.264437 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:25.264390    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b" event={"ID":"d0926f63-f6cc-41e9-a804-ca6ccd217df1","Type":"ContainerStarted","Data":"7e6497ff20718bd9e49cb02aea96f6870771ce8e96f7c66e523f5e2a87e3abbb"}
Jul 27 18:37:25.264821 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:25.264675    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"
Jul 27 18:37:25.281041 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:25.280993    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b" podStartSLOduration=3.32379522 podStartE2EDuration="6.280978161s" totalImagesPullingTime="2.957182941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:37:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:37:25.280947163 +0000 UTC m=+353.831240791" watchObservedRunningTime="2026-07-27 18:37:25.280978161 +0000 UTC m=+353.831271793"
Jul 27 18:37:36.270445 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:37:36.270409    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"
Jul 27 18:39:12.889998 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:12.889958    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-c4ml2"]
Jul 27 18:39:12.892129 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:12.892109    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ml2"
Jul 27 18:39:12.896554 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:12.896527    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 27 18:39:12.896554 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:12.896552    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 27 18:39:12.896748 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:12.896566    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 27 18:39:12.896748 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:12.896551    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 27 18:39:12.896748 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:12.896613    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-snhgl\""
Jul 27 18:39:12.921777 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:12.921738    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-c4ml2"]
Jul 27 18:39:12.954057 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:12.954015    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrlpm\" (UniqueName: \"kubernetes.io/projected/6aed4676-7892-4dd9-8e18-a15ab50cbaa2-kube-api-access-vrlpm\") pod \"kuadrant-console-plugin-66b5774985-c4ml2\" (UID: \"6aed4676-7892-4dd9-8e18-a15ab50cbaa2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ml2"
Jul 27 18:39:12.954267 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:12.954096    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6aed4676-7892-4dd9-8e18-a15ab50cbaa2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-c4ml2\" (UID: \"6aed4676-7892-4dd9-8e18-a15ab50cbaa2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ml2"
Jul 27 18:39:12.954267 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:12.954144    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6aed4676-7892-4dd9-8e18-a15ab50cbaa2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-c4ml2\" (UID: \"6aed4676-7892-4dd9-8e18-a15ab50cbaa2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ml2"
Jul 27 18:39:13.054991 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:13.054956    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrlpm\" (UniqueName: \"kubernetes.io/projected/6aed4676-7892-4dd9-8e18-a15ab50cbaa2-kube-api-access-vrlpm\") pod \"kuadrant-console-plugin-66b5774985-c4ml2\" (UID: \"6aed4676-7892-4dd9-8e18-a15ab50cbaa2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ml2"
Jul 27 18:39:13.055168 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:13.055028    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6aed4676-7892-4dd9-8e18-a15ab50cbaa2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-c4ml2\" (UID: \"6aed4676-7892-4dd9-8e18-a15ab50cbaa2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ml2"
Jul 27 18:39:13.055168 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:13.055068    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6aed4676-7892-4dd9-8e18-a15ab50cbaa2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-c4ml2\" (UID: \"6aed4676-7892-4dd9-8e18-a15ab50cbaa2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ml2"
Jul 27 18:39:13.055240 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:39:13.055170    2562 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 27 18:39:13.055274 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:39:13.055239    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6aed4676-7892-4dd9-8e18-a15ab50cbaa2-plugin-serving-cert podName:6aed4676-7892-4dd9-8e18-a15ab50cbaa2 nodeName:}" failed. No retries permitted until 2026-07-27 18:39:13.555218665 +0000 UTC m=+462.105512283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/6aed4676-7892-4dd9-8e18-a15ab50cbaa2-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-c4ml2" (UID: "6aed4676-7892-4dd9-8e18-a15ab50cbaa2") : secret "plugin-serving-cert" not found
Jul 27 18:39:13.055753 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:13.055734    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6aed4676-7892-4dd9-8e18-a15ab50cbaa2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-c4ml2\" (UID: \"6aed4676-7892-4dd9-8e18-a15ab50cbaa2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ml2"
Jul 27 18:39:13.065281 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:13.065253    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrlpm\" (UniqueName: \"kubernetes.io/projected/6aed4676-7892-4dd9-8e18-a15ab50cbaa2-kube-api-access-vrlpm\") pod \"kuadrant-console-plugin-66b5774985-c4ml2\" (UID: \"6aed4676-7892-4dd9-8e18-a15ab50cbaa2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ml2"
Jul 27 18:39:13.557841 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:13.557802    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6aed4676-7892-4dd9-8e18-a15ab50cbaa2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-c4ml2\" (UID: \"6aed4676-7892-4dd9-8e18-a15ab50cbaa2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ml2"
Jul 27 18:39:13.560448 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:13.560415    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6aed4676-7892-4dd9-8e18-a15ab50cbaa2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-c4ml2\" (UID: \"6aed4676-7892-4dd9-8e18-a15ab50cbaa2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ml2"
Jul 27 18:39:13.802265 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:13.802214    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ml2"
Jul 27 18:39:13.944719 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:13.944684    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-c4ml2"]
Jul 27 18:39:13.948285 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:39:13.948253    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6aed4676_7892_4dd9_8e18_a15ab50cbaa2.slice/crio-1cbf18065c871770506a92ac020a23c14b07e394be81e5d835a1baa8f62abdae WatchSource:0}: Error finding container 1cbf18065c871770506a92ac020a23c14b07e394be81e5d835a1baa8f62abdae: Status 404 returned error can't find the container with id 1cbf18065c871770506a92ac020a23c14b07e394be81e5d835a1baa8f62abdae
Jul 27 18:39:14.690234 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:14.690192    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ml2" event={"ID":"6aed4676-7892-4dd9-8e18-a15ab50cbaa2","Type":"ContainerStarted","Data":"1cbf18065c871770506a92ac020a23c14b07e394be81e5d835a1baa8f62abdae"}
Jul 27 18:39:36.796290 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:36.796252    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ml2" event={"ID":"6aed4676-7892-4dd9-8e18-a15ab50cbaa2","Type":"ContainerStarted","Data":"35d69ab3ea9b06eab8c06a0abbcbbbfd962ee1361eeb20fa5b0ebb6d3630f752"}
Jul 27 18:39:59.330437 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:59.330361    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ml2" podStartSLOduration=24.58279372 podStartE2EDuration="47.330340496s" totalImagesPullingTime="22.747546776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:39:36.816511171 +0000 UTC m=+485.366804823" watchObservedRunningTime="2026-07-27 18:39:59.330340496 +0000 UTC m=+507.880634133"
Jul 27 18:39:59.331567 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:59.331548    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-j2nl2"]
Jul 27 18:39:59.390500 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:59.390461    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-j2nl2"]
Jul 27 18:39:59.390653 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:59.390582    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-j2nl2"
Jul 27 18:39:59.394038 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:59.394014    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lxnsr\""
Jul 27 18:39:59.463328 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:59.463293    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvgqc\" (UniqueName: \"kubernetes.io/projected/3af39b78-efba-44e8-8512-1a9d2febd8d2-kube-api-access-wvgqc\") pod \"authorino-58cd955675-j2nl2\" (UID: \"3af39b78-efba-44e8-8512-1a9d2febd8d2\") " pod="kuadrant-system/authorino-58cd955675-j2nl2"
Jul 27 18:39:59.564651 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:59.564608    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvgqc\" (UniqueName: \"kubernetes.io/projected/3af39b78-efba-44e8-8512-1a9d2febd8d2-kube-api-access-wvgqc\") pod \"authorino-58cd955675-j2nl2\" (UID: \"3af39b78-efba-44e8-8512-1a9d2febd8d2\") " pod="kuadrant-system/authorino-58cd955675-j2nl2"
Jul 27 18:39:59.574581 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:59.574555    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvgqc\" (UniqueName: \"kubernetes.io/projected/3af39b78-efba-44e8-8512-1a9d2febd8d2-kube-api-access-wvgqc\") pod \"authorino-58cd955675-j2nl2\" (UID: \"3af39b78-efba-44e8-8512-1a9d2febd8d2\") " pod="kuadrant-system/authorino-58cd955675-j2nl2"
Jul 27 18:39:59.699862 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:59.699786    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-j2nl2"
Jul 27 18:39:59.870039 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:59.870007    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-j2nl2"]
Jul 27 18:39:59.873252 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:39:59.873218    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3af39b78_efba_44e8_8512_1a9d2febd8d2.slice/crio-e503bb2fc841806a9bc7b6e2015bcde31d312daf30afa1a36a652f164256a11d WatchSource:0}: Error finding container e503bb2fc841806a9bc7b6e2015bcde31d312daf30afa1a36a652f164256a11d: Status 404 returned error can't find the container with id e503bb2fc841806a9bc7b6e2015bcde31d312daf30afa1a36a652f164256a11d
Jul 27 18:39:59.894943 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:39:59.894911    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-j2nl2" event={"ID":"3af39b78-efba-44e8-8512-1a9d2febd8d2","Type":"ContainerStarted","Data":"e503bb2fc841806a9bc7b6e2015bcde31d312daf30afa1a36a652f164256a11d"}
Jul 27 18:40:04.921504 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:04.921466    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-j2nl2" event={"ID":"3af39b78-efba-44e8-8512-1a9d2febd8d2","Type":"ContainerStarted","Data":"1a300ce45bf283be5b28e4fd5ee3ec22d3bee3510009ade648b5d99d60347a97"}
Jul 27 18:40:04.938247 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:04.938187    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-j2nl2" podStartSLOduration=1.431821048 podStartE2EDuration="5.93817347s" totalImagesPullingTime="4.506352422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:39:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:40:04.937701949 +0000 UTC m=+513.487995607" watchObservedRunningTime="2026-07-27 18:40:04.93817347 +0000 UTC m=+513.488467097"
Jul 27 18:40:05.153012 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:05.152979    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-j2nl2"]
Jul 27 18:40:06.929453 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:06.929385    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-j2nl2" podUID="3af39b78-efba-44e8-8512-1a9d2febd8d2" containerName="authorino" containerID="cri-o://1a300ce45bf283be5b28e4fd5ee3ec22d3bee3510009ade648b5d99d60347a97" gracePeriod=30
Jul 27 18:40:07.175443 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:07.175416    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-j2nl2"
Jul 27 18:40:07.338055 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:07.338004    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvgqc\" (UniqueName: \"kubernetes.io/projected/3af39b78-efba-44e8-8512-1a9d2febd8d2-kube-api-access-wvgqc\") pod \"3af39b78-efba-44e8-8512-1a9d2febd8d2\" (UID: \"3af39b78-efba-44e8-8512-1a9d2febd8d2\") "
Jul 27 18:40:07.340395 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:07.340364    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3af39b78-efba-44e8-8512-1a9d2febd8d2-kube-api-access-wvgqc" (OuterVolumeSpecName: "kube-api-access-wvgqc") pod "3af39b78-efba-44e8-8512-1a9d2febd8d2" (UID: "3af39b78-efba-44e8-8512-1a9d2febd8d2"). InnerVolumeSpecName "kube-api-access-wvgqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:40:07.438694 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:07.438642    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvgqc\" (UniqueName: \"kubernetes.io/projected/3af39b78-efba-44e8-8512-1a9d2febd8d2-kube-api-access-wvgqc\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 18:40:07.935453 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:07.935423    2562 generic.go:356] "Generic (PLEG): container finished" podID="3af39b78-efba-44e8-8512-1a9d2febd8d2" containerID="1a300ce45bf283be5b28e4fd5ee3ec22d3bee3510009ade648b5d99d60347a97" exitCode=0
Jul 27 18:40:07.935951 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:07.935495    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-j2nl2"
Jul 27 18:40:07.936183 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:07.936157    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-j2nl2" event={"ID":"3af39b78-efba-44e8-8512-1a9d2febd8d2","Type":"ContainerDied","Data":"1a300ce45bf283be5b28e4fd5ee3ec22d3bee3510009ade648b5d99d60347a97"}
Jul 27 18:40:07.936264 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:07.936196    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-j2nl2" event={"ID":"3af39b78-efba-44e8-8512-1a9d2febd8d2","Type":"ContainerDied","Data":"e503bb2fc841806a9bc7b6e2015bcde31d312daf30afa1a36a652f164256a11d"}
Jul 27 18:40:07.936264 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:07.936221    2562 scope.go:117] "RemoveContainer" containerID="1a300ce45bf283be5b28e4fd5ee3ec22d3bee3510009ade648b5d99d60347a97"
Jul 27 18:40:07.945204 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:07.945182    2562 scope.go:117] "RemoveContainer" containerID="1a300ce45bf283be5b28e4fd5ee3ec22d3bee3510009ade648b5d99d60347a97"
Jul 27 18:40:07.945586 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:40:07.945560    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a300ce45bf283be5b28e4fd5ee3ec22d3bee3510009ade648b5d99d60347a97\": container with ID starting with 1a300ce45bf283be5b28e4fd5ee3ec22d3bee3510009ade648b5d99d60347a97 not found: ID does not exist" containerID="1a300ce45bf283be5b28e4fd5ee3ec22d3bee3510009ade648b5d99d60347a97"
Jul 27 18:40:07.945692 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:07.945592    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a300ce45bf283be5b28e4fd5ee3ec22d3bee3510009ade648b5d99d60347a97"} err="failed to get container status \"1a300ce45bf283be5b28e4fd5ee3ec22d3bee3510009ade648b5d99d60347a97\": rpc error: code = NotFound desc = could not find container \"1a300ce45bf283be5b28e4fd5ee3ec22d3bee3510009ade648b5d99d60347a97\": container with ID starting with 1a300ce45bf283be5b28e4fd5ee3ec22d3bee3510009ade648b5d99d60347a97 not found: ID does not exist"
Jul 27 18:40:07.959929 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:07.959895    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-j2nl2"]
Jul 27 18:40:07.966174 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:07.966144    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-j2nl2"]
Jul 27 18:40:09.936257 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:09.936224    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3af39b78-efba-44e8-8512-1a9d2febd8d2" path="/var/lib/kubelet/pods/3af39b78-efba-44e8-8512-1a9d2febd8d2/volumes"
Jul 27 18:40:33.804468 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:33.804430    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-zdwx7"]
Jul 27 18:40:33.804993 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:33.804742    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3af39b78-efba-44e8-8512-1a9d2febd8d2" containerName="authorino"
Jul 27 18:40:33.804993 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:33.804753    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3af39b78-efba-44e8-8512-1a9d2febd8d2" containerName="authorino"
Jul 27 18:40:33.804993 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:33.804803    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3af39b78-efba-44e8-8512-1a9d2febd8d2" containerName="authorino"
Jul 27 18:40:33.812129 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:33.812109    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-zdwx7"
Jul 27 18:40:33.819736 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:33.819711    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lxnsr\""
Jul 27 18:40:33.830082 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:33.830051    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vxq4\" (UniqueName: \"kubernetes.io/projected/db5e16a1-1848-4cff-ae13-ae6b212268a9-kube-api-access-8vxq4\") pod \"authorino-5d4b67c446-zdwx7\" (UID: \"db5e16a1-1848-4cff-ae13-ae6b212268a9\") " pod="kuadrant-system/authorino-5d4b67c446-zdwx7"
Jul 27 18:40:33.849087 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:33.849054    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-zdwx7"]
Jul 27 18:40:33.930733 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:33.930692    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vxq4\" (UniqueName: \"kubernetes.io/projected/db5e16a1-1848-4cff-ae13-ae6b212268a9-kube-api-access-8vxq4\") pod \"authorino-5d4b67c446-zdwx7\" (UID: \"db5e16a1-1848-4cff-ae13-ae6b212268a9\") " pod="kuadrant-system/authorino-5d4b67c446-zdwx7"
Jul 27 18:40:33.942928 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:33.942902    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vxq4\" (UniqueName: \"kubernetes.io/projected/db5e16a1-1848-4cff-ae13-ae6b212268a9-kube-api-access-8vxq4\") pod \"authorino-5d4b67c446-zdwx7\" (UID: \"db5e16a1-1848-4cff-ae13-ae6b212268a9\") " pod="kuadrant-system/authorino-5d4b67c446-zdwx7"
Jul 27 18:40:34.090082 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:34.089992    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-zdwx7"]
Jul 27 18:40:34.090242 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:34.090230    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-zdwx7"
Jul 27 18:40:34.203626 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:34.203593    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-55b969d5c8-c44t8"]
Jul 27 18:40:34.207883 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:34.207841    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55b969d5c8-c44t8"
Jul 27 18:40:34.231709 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:34.231676    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55b969d5c8-c44t8"]
Jul 27 18:40:34.232442 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:34.232417    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnh4l\" (UniqueName: \"kubernetes.io/projected/20577b57-55f0-4db6-a8f7-27f38133c70c-kube-api-access-bnh4l\") pod \"authorino-55b969d5c8-c44t8\" (UID: \"20577b57-55f0-4db6-a8f7-27f38133c70c\") " pod="kuadrant-system/authorino-55b969d5c8-c44t8"
Jul 27 18:40:34.255401 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:34.255365    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-zdwx7"]
Jul 27 18:40:34.261490 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:40:34.261454    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb5e16a1_1848_4cff_ae13_ae6b212268a9.slice/crio-819997108c076c3f2f8eff0ad67ea8da789b285e3fa7cf35cf7094448596c639 WatchSource:0}: Error finding container 819997108c076c3f2f8eff0ad67ea8da789b285e3fa7cf35cf7094448596c639: Status 404 returned error can't find the container with id 819997108c076c3f2f8eff0ad67ea8da789b285e3fa7cf35cf7094448596c639
Jul 27 18:40:34.333758 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:34.333721    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnh4l\" (UniqueName: \"kubernetes.io/projected/20577b57-55f0-4db6-a8f7-27f38133c70c-kube-api-access-bnh4l\") pod \"authorino-55b969d5c8-c44t8\" (UID: \"20577b57-55f0-4db6-a8f7-27f38133c70c\") " pod="kuadrant-system/authorino-55b969d5c8-c44t8"
Jul 27 18:40:34.346207 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:34.346135    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnh4l\" (UniqueName: \"kubernetes.io/projected/20577b57-55f0-4db6-a8f7-27f38133c70c-kube-api-access-bnh4l\") pod \"authorino-55b969d5c8-c44t8\" (UID: \"20577b57-55f0-4db6-a8f7-27f38133c70c\") " pod="kuadrant-system/authorino-55b969d5c8-c44t8"
Jul 27 18:40:34.477954 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:34.477897    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-55b969d5c8-c44t8"]
Jul 27 18:40:34.478176 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:34.478163    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55b969d5c8-c44t8"
Jul 27 18:40:34.544516 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:34.544484    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7d74fb5494-l2s2b"]
Jul 27 18:40:34.548114 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:34.548090    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d74fb5494-l2s2b"
Jul 27 18:40:34.550928 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:40:34.550898    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"authorino-7d74fb5494-l2s2b\" is forbidden: User \"system:node:ip-10-0-132-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-186.ec2.internal' and this object" podUID="dcb42398-4182-4c32-844b-d69dc7d97e81" pod="kuadrant-system/authorino-7d74fb5494-l2s2b"
Jul 27 18:40:34.551088 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:40:34.551058    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"authorino-server-cert\" is forbidden: User \"system:node:ip-10-0-132-186.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-186.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" type="*v1.Secret"
Jul 27 18:40:34.579069 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:34.579018    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d74fb5494-l2s2b"]
Jul 27 18:40:34.635900 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:34.635852    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dcb42398-4182-4c32-844b-d69dc7d97e81-tls-cert\") pod \"authorino-7d74fb5494-l2s2b\" (UID: \"dcb42398-4182-4c32-844b-d69dc7d97e81\") " pod="kuadrant-system/authorino-7d74fb5494-l2s2b"
Jul 27 18:40:34.636028 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:34.635979    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxj9r\" (UniqueName: \"kubernetes.io/projected/dcb42398-4182-4c32-844b-d69dc7d97e81-kube-api-access-nxj9r\") pod \"authorino-7d74fb5494-l2s2b\" (UID: \"dcb42398-4182-4c32-844b-d69dc7d97e81\") " pod="kuadrant-system/authorino-7d74fb5494-l2s2b"
Jul 27 18:40:34.642419 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:34.642394    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-55b969d5c8-c44t8"]
Jul 27 18:40:34.643848 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:40:34.643821    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20577b57_55f0_4db6_a8f7_27f38133c70c.slice/crio-9d655d5de68606c8433f6aa6721e91d3d87a632ce72136c277cbc752d2844c24 WatchSource:0}: Error finding container 9d655d5de68606c8433f6aa6721e91d3d87a632ce72136c277cbc752d2844c24: Status 404 returned error can't find the container with id 9d655d5de68606c8433f6aa6721e91d3d87a632ce72136c277cbc752d2844c24
Jul 27 18:40:34.736642 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:34.736547    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dcb42398-4182-4c32-844b-d69dc7d97e81-tls-cert\") pod \"authorino-7d74fb5494-l2s2b\" (UID: \"dcb42398-4182-4c32-844b-d69dc7d97e81\") " pod="kuadrant-system/authorino-7d74fb5494-l2s2b"
Jul 27 18:40:34.736855 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:34.736678    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxj9r\" (UniqueName: \"kubernetes.io/projected/dcb42398-4182-4c32-844b-d69dc7d97e81-kube-api-access-nxj9r\") pod \"authorino-7d74fb5494-l2s2b\" (UID: \"dcb42398-4182-4c32-844b-d69dc7d97e81\") " pod="kuadrant-system/authorino-7d74fb5494-l2s2b"
Jul 27 18:40:34.767589 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:34.767555    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxj9r\" (UniqueName: \"kubernetes.io/projected/dcb42398-4182-4c32-844b-d69dc7d97e81-kube-api-access-nxj9r\") pod \"authorino-7d74fb5494-l2s2b\" (UID: \"dcb42398-4182-4c32-844b-d69dc7d97e81\") " pod="kuadrant-system/authorino-7d74fb5494-l2s2b"
Jul 27 18:40:35.049603 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:35.049567    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55b969d5c8-c44t8" event={"ID":"20577b57-55f0-4db6-a8f7-27f38133c70c","Type":"ContainerStarted","Data":"9d655d5de68606c8433f6aa6721e91d3d87a632ce72136c277cbc752d2844c24"}
Jul 27 18:40:35.050915 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:35.050889    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-zdwx7" event={"ID":"db5e16a1-1848-4cff-ae13-ae6b212268a9","Type":"ContainerStarted","Data":"3a7c7e4aad3da995dabe6c04fd63b1a1b16fe76b39600b2945ccd8e130207cd5"}
Jul 27 18:40:35.051021 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:35.050928    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-zdwx7" event={"ID":"db5e16a1-1848-4cff-ae13-ae6b212268a9","Type":"ContainerStarted","Data":"819997108c076c3f2f8eff0ad67ea8da789b285e3fa7cf35cf7094448596c639"}
Jul 27 18:40:35.051021 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:35.050973    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-zdwx7" podUID="db5e16a1-1848-4cff-ae13-ae6b212268a9" containerName="authorino" containerID="cri-o://3a7c7e4aad3da995dabe6c04fd63b1a1b16fe76b39600b2945ccd8e130207cd5" gracePeriod=30
Jul 27 18:40:35.149799 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:35.149743    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-zdwx7" podStartSLOduration=1.806991032 podStartE2EDuration="2.149725697s" totalImagesPullingTime="342.734665ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:40:35.149128052 +0000 UTC m=+543.699421685" watchObservedRunningTime="2026-07-27 18:40:35.149725697 +0000 UTC m=+543.700019371"
Jul 27 18:40:35.306456 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:35.306393    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-zdwx7"
Jul 27 18:40:35.341961 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:35.341918    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vxq4\" (UniqueName: \"kubernetes.io/projected/db5e16a1-1848-4cff-ae13-ae6b212268a9-kube-api-access-8vxq4\") pod \"db5e16a1-1848-4cff-ae13-ae6b212268a9\" (UID: \"db5e16a1-1848-4cff-ae13-ae6b212268a9\") "
Jul 27 18:40:35.344402 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:35.344358    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db5e16a1-1848-4cff-ae13-ae6b212268a9-kube-api-access-8vxq4" (OuterVolumeSpecName: "kube-api-access-8vxq4") pod "db5e16a1-1848-4cff-ae13-ae6b212268a9" (UID: "db5e16a1-1848-4cff-ae13-ae6b212268a9"). InnerVolumeSpecName "kube-api-access-8vxq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:40:35.443393 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:35.443355    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vxq4\" (UniqueName: \"kubernetes.io/projected/db5e16a1-1848-4cff-ae13-ae6b212268a9-kube-api-access-8vxq4\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 18:40:35.533566 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:35.533528    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 27 18:40:35.540089 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:35.540064    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dcb42398-4182-4c32-844b-d69dc7d97e81-tls-cert\") pod \"authorino-7d74fb5494-l2s2b\" (UID: \"dcb42398-4182-4c32-844b-d69dc7d97e81\") " pod="kuadrant-system/authorino-7d74fb5494-l2s2b"
Jul 27 18:40:35.757245 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:35.757205    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d74fb5494-l2s2b"
Jul 27 18:40:35.901921 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:35.901894    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d74fb5494-l2s2b"]
Jul 27 18:40:35.904362 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:40:35.904334    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddcb42398_4182_4c32_844b_d69dc7d97e81.slice/crio-f5bcc123b35d5f245a1f0405116664560ddfa521f48c09f79da142bb845eb419 WatchSource:0}: Error finding container f5bcc123b35d5f245a1f0405116664560ddfa521f48c09f79da142bb845eb419: Status 404 returned error can't find the container with id f5bcc123b35d5f245a1f0405116664560ddfa521f48c09f79da142bb845eb419
Jul 27 18:40:36.056486 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:36.056393    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55b969d5c8-c44t8" event={"ID":"20577b57-55f0-4db6-a8f7-27f38133c70c","Type":"ContainerStarted","Data":"4f92c63739e5cd8bd5e112c274568480a86180ffccbd6c3a437c6118f4928fb5"}
Jul 27 18:40:36.056486 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:36.056447    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-55b969d5c8-c44t8" podUID="20577b57-55f0-4db6-a8f7-27f38133c70c" containerName="authorino" containerID="cri-o://4f92c63739e5cd8bd5e112c274568480a86180ffccbd6c3a437c6118f4928fb5" gracePeriod=30
Jul 27 18:40:36.057812 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:36.057787    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d74fb5494-l2s2b" event={"ID":"dcb42398-4182-4c32-844b-d69dc7d97e81","Type":"ContainerStarted","Data":"f5bcc123b35d5f245a1f0405116664560ddfa521f48c09f79da142bb845eb419"}
Jul 27 18:40:36.059034 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:36.059006    2562 generic.go:356] "Generic (PLEG): container finished" podID="db5e16a1-1848-4cff-ae13-ae6b212268a9" containerID="3a7c7e4aad3da995dabe6c04fd63b1a1b16fe76b39600b2945ccd8e130207cd5" exitCode=0
Jul 27 18:40:36.059133 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:36.059045    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-zdwx7"
Jul 27 18:40:36.059133 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:36.059078    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-zdwx7" event={"ID":"db5e16a1-1848-4cff-ae13-ae6b212268a9","Type":"ContainerDied","Data":"3a7c7e4aad3da995dabe6c04fd63b1a1b16fe76b39600b2945ccd8e130207cd5"}
Jul 27 18:40:36.059133 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:36.059113    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-zdwx7" event={"ID":"db5e16a1-1848-4cff-ae13-ae6b212268a9","Type":"ContainerDied","Data":"819997108c076c3f2f8eff0ad67ea8da789b285e3fa7cf35cf7094448596c639"}
Jul 27 18:40:36.059133 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:36.059132    2562 scope.go:117] "RemoveContainer" containerID="3a7c7e4aad3da995dabe6c04fd63b1a1b16fe76b39600b2945ccd8e130207cd5"
Jul 27 18:40:36.067484 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:36.067464    2562 scope.go:117] "RemoveContainer" containerID="3a7c7e4aad3da995dabe6c04fd63b1a1b16fe76b39600b2945ccd8e130207cd5"
Jul 27 18:40:36.067742 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:40:36.067724    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a7c7e4aad3da995dabe6c04fd63b1a1b16fe76b39600b2945ccd8e130207cd5\": container with ID starting with 3a7c7e4aad3da995dabe6c04fd63b1a1b16fe76b39600b2945ccd8e130207cd5 not found: ID does not exist" containerID="3a7c7e4aad3da995dabe6c04fd63b1a1b16fe76b39600b2945ccd8e130207cd5"
Jul 27 18:40:36.067800 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:36.067755    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a7c7e4aad3da995dabe6c04fd63b1a1b16fe76b39600b2945ccd8e130207cd5"} err="failed to get container status \"3a7c7e4aad3da995dabe6c04fd63b1a1b16fe76b39600b2945ccd8e130207cd5\": rpc error: code = NotFound desc = could not find container \"3a7c7e4aad3da995dabe6c04fd63b1a1b16fe76b39600b2945ccd8e130207cd5\": container with ID starting with 3a7c7e4aad3da995dabe6c04fd63b1a1b16fe76b39600b2945ccd8e130207cd5 not found: ID does not exist"
Jul 27 18:40:36.136892 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:36.136828    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-55b969d5c8-c44t8" podStartSLOduration=1.6739556439999999 podStartE2EDuration="2.136810262s" totalImagesPullingTime="462.854618ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:40:36.109774549 +0000 UTC m=+544.660068179" watchObservedRunningTime="2026-07-27 18:40:36.136810262 +0000 UTC m=+544.687103892"
Jul 27 18:40:36.137550 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:36.137532    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-zdwx7"]
Jul 27 18:40:36.142677 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:36.142645    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-zdwx7"]
Jul 27 18:40:36.301606 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:36.301578    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55b969d5c8-c44t8"
Jul 27 18:40:36.350830 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:36.350760    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnh4l\" (UniqueName: \"kubernetes.io/projected/20577b57-55f0-4db6-a8f7-27f38133c70c-kube-api-access-bnh4l\") pod \"20577b57-55f0-4db6-a8f7-27f38133c70c\" (UID: \"20577b57-55f0-4db6-a8f7-27f38133c70c\") "
Jul 27 18:40:36.352952 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:36.352931    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20577b57-55f0-4db6-a8f7-27f38133c70c-kube-api-access-bnh4l" (OuterVolumeSpecName: "kube-api-access-bnh4l") pod "20577b57-55f0-4db6-a8f7-27f38133c70c" (UID: "20577b57-55f0-4db6-a8f7-27f38133c70c"). InnerVolumeSpecName "kube-api-access-bnh4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:40:36.452337 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:36.452297    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnh4l\" (UniqueName: \"kubernetes.io/projected/20577b57-55f0-4db6-a8f7-27f38133c70c-kube-api-access-bnh4l\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 18:40:37.064329 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:37.064290    2562 generic.go:356] "Generic (PLEG): container finished" podID="20577b57-55f0-4db6-a8f7-27f38133c70c" containerID="4f92c63739e5cd8bd5e112c274568480a86180ffccbd6c3a437c6118f4928fb5" exitCode=0
Jul 27 18:40:37.064672 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:37.064346    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55b969d5c8-c44t8"
Jul 27 18:40:37.064672 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:37.064353    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55b969d5c8-c44t8" event={"ID":"20577b57-55f0-4db6-a8f7-27f38133c70c","Type":"ContainerDied","Data":"4f92c63739e5cd8bd5e112c274568480a86180ffccbd6c3a437c6118f4928fb5"}
Jul 27 18:40:37.064672 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:37.064391    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55b969d5c8-c44t8" event={"ID":"20577b57-55f0-4db6-a8f7-27f38133c70c","Type":"ContainerDied","Data":"9d655d5de68606c8433f6aa6721e91d3d87a632ce72136c277cbc752d2844c24"}
Jul 27 18:40:37.064672 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:37.064412    2562 scope.go:117] "RemoveContainer" containerID="4f92c63739e5cd8bd5e112c274568480a86180ffccbd6c3a437c6118f4928fb5"
Jul 27 18:40:37.074044 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:37.074025    2562 scope.go:117] "RemoveContainer" containerID="4f92c63739e5cd8bd5e112c274568480a86180ffccbd6c3a437c6118f4928fb5"
Jul 27 18:40:37.074368 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:40:37.074342    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f92c63739e5cd8bd5e112c274568480a86180ffccbd6c3a437c6118f4928fb5\": container with ID starting with 4f92c63739e5cd8bd5e112c274568480a86180ffccbd6c3a437c6118f4928fb5 not found: ID does not exist" containerID="4f92c63739e5cd8bd5e112c274568480a86180ffccbd6c3a437c6118f4928fb5"
Jul 27 18:40:37.074419 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:37.074382    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f92c63739e5cd8bd5e112c274568480a86180ffccbd6c3a437c6118f4928fb5"} err="failed to get container status \"4f92c63739e5cd8bd5e112c274568480a86180ffccbd6c3a437c6118f4928fb5\": rpc error: code = NotFound desc = could not find container \"4f92c63739e5cd8bd5e112c274568480a86180ffccbd6c3a437c6118f4928fb5\": container with ID starting with 4f92c63739e5cd8bd5e112c274568480a86180ffccbd6c3a437c6118f4928fb5 not found: ID does not exist"
Jul 27 18:40:37.102765 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:37.102735    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-55b969d5c8-c44t8"]
Jul 27 18:40:37.114446 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:37.114414    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-55b969d5c8-c44t8"]
Jul 27 18:40:37.936829 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:37.936792    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20577b57-55f0-4db6-a8f7-27f38133c70c" path="/var/lib/kubelet/pods/20577b57-55f0-4db6-a8f7-27f38133c70c/volumes"
Jul 27 18:40:37.937255 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:37.937234    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db5e16a1-1848-4cff-ae13-ae6b212268a9" path="/var/lib/kubelet/pods/db5e16a1-1848-4cff-ae13-ae6b212268a9/volumes"
Jul 27 18:40:38.070382 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:38.070347    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d74fb5494-l2s2b" event={"ID":"dcb42398-4182-4c32-844b-d69dc7d97e81","Type":"ContainerStarted","Data":"b019be941d07296d62ef162edd567897fadc3b8ef14b42edaad02c317508a637"}
Jul 27 18:40:38.153644 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:40:38.153585    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7d74fb5494-l2s2b" podStartSLOduration=3.052231288 podStartE2EDuration="4.153566654s" totalImagesPullingTime="1.101335366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:40:38.152652537 +0000 UTC m=+546.702946197" watchObservedRunningTime="2026-07-27 18:40:38.153566654 +0000 UTC m=+546.703860325"
Jul 27 18:41:31.780985 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:41:31.780957    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 18:41:31.782167 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:41:31.782147    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 18:42:01.016138 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.016058    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"]
Jul 27 18:42:01.018423 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.016357    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db5e16a1-1848-4cff-ae13-ae6b212268a9" containerName="authorino"
Jul 27 18:42:01.018423 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.016370    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="db5e16a1-1848-4cff-ae13-ae6b212268a9" containerName="authorino"
Jul 27 18:42:01.018423 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.016382    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20577b57-55f0-4db6-a8f7-27f38133c70c" containerName="authorino"
Jul 27 18:42:01.018423 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.016388    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="20577b57-55f0-4db6-a8f7-27f38133c70c" containerName="authorino"
Jul 27 18:42:01.018423 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.016434    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="20577b57-55f0-4db6-a8f7-27f38133c70c" containerName="authorino"
Jul 27 18:42:01.018423 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.016442    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="db5e16a1-1848-4cff-ae13-ae6b212268a9" containerName="authorino"
Jul 27 18:42:01.019314 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.019298    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:01.022264 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.022242    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 27 18:42:01.022415 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.022242    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kt8p7\""
Jul 27 18:42:01.022415 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.022242    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 27 18:42:01.022415 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.022254    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 27 18:42:01.034927 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.034894    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"]
Jul 27 18:42:01.133219 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.133181    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddpnl\" (UniqueName: \"kubernetes.io/projected/5fc6b031-e7ad-41bf-952e-a532a490e8c6-kube-api-access-ddpnl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf\" (UID: \"5fc6b031-e7ad-41bf-952e-a532a490e8c6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:01.133420 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.133233    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fc6b031-e7ad-41bf-952e-a532a490e8c6-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf\" (UID: \"5fc6b031-e7ad-41bf-952e-a532a490e8c6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:01.133420 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.133296    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fc6b031-e7ad-41bf-952e-a532a490e8c6-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf\" (UID: \"5fc6b031-e7ad-41bf-952e-a532a490e8c6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:01.133420 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.133353    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fc6b031-e7ad-41bf-952e-a532a490e8c6-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf\" (UID: \"5fc6b031-e7ad-41bf-952e-a532a490e8c6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:01.133420 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.133380    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fc6b031-e7ad-41bf-952e-a532a490e8c6-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf\" (UID: \"5fc6b031-e7ad-41bf-952e-a532a490e8c6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:01.133420 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.133413    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fc6b031-e7ad-41bf-952e-a532a490e8c6-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf\" (UID: \"5fc6b031-e7ad-41bf-952e-a532a490e8c6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:01.234278 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.234223    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fc6b031-e7ad-41bf-952e-a532a490e8c6-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf\" (UID: \"5fc6b031-e7ad-41bf-952e-a532a490e8c6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:01.234278 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.234281    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fc6b031-e7ad-41bf-952e-a532a490e8c6-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf\" (UID: \"5fc6b031-e7ad-41bf-952e-a532a490e8c6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:01.234278 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.234296    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fc6b031-e7ad-41bf-952e-a532a490e8c6-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf\" (UID: \"5fc6b031-e7ad-41bf-952e-a532a490e8c6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:01.234566 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.234340    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddpnl\" (UniqueName: \"kubernetes.io/projected/5fc6b031-e7ad-41bf-952e-a532a490e8c6-kube-api-access-ddpnl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf\" (UID: \"5fc6b031-e7ad-41bf-952e-a532a490e8c6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:01.234566 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.234364    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fc6b031-e7ad-41bf-952e-a532a490e8c6-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf\" (UID: \"5fc6b031-e7ad-41bf-952e-a532a490e8c6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:01.234566 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.234383    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fc6b031-e7ad-41bf-952e-a532a490e8c6-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf\" (UID: \"5fc6b031-e7ad-41bf-952e-a532a490e8c6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:01.234737 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.234711    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fc6b031-e7ad-41bf-952e-a532a490e8c6-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf\" (UID: \"5fc6b031-e7ad-41bf-952e-a532a490e8c6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:01.234737 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.234727    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fc6b031-e7ad-41bf-952e-a532a490e8c6-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf\" (UID: \"5fc6b031-e7ad-41bf-952e-a532a490e8c6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:01.234842 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.234778    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fc6b031-e7ad-41bf-952e-a532a490e8c6-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf\" (UID: \"5fc6b031-e7ad-41bf-952e-a532a490e8c6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:01.236913 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.236884    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fc6b031-e7ad-41bf-952e-a532a490e8c6-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf\" (UID: \"5fc6b031-e7ad-41bf-952e-a532a490e8c6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:01.237153 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.237136    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fc6b031-e7ad-41bf-952e-a532a490e8c6-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf\" (UID: \"5fc6b031-e7ad-41bf-952e-a532a490e8c6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:01.242823 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.242800    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddpnl\" (UniqueName: \"kubernetes.io/projected/5fc6b031-e7ad-41bf-952e-a532a490e8c6-kube-api-access-ddpnl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf\" (UID: \"5fc6b031-e7ad-41bf-952e-a532a490e8c6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:01.330818 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.330726    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:01.474442 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.474406    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"]
Jul 27 18:42:01.477691 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:42:01.477659    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fc6b031_e7ad_41bf_952e_a532a490e8c6.slice/crio-37e1a0983ea575fee5c0842678059a1708635945f2f0a471ec32fdebc7dbba18 WatchSource:0}: Error finding container 37e1a0983ea575fee5c0842678059a1708635945f2f0a471ec32fdebc7dbba18: Status 404 returned error can't find the container with id 37e1a0983ea575fee5c0842678059a1708635945f2f0a471ec32fdebc7dbba18
Jul 27 18:42:01.479012 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:01.478995    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 18:42:02.410140 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:02.410105    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf" event={"ID":"5fc6b031-e7ad-41bf-952e-a532a490e8c6","Type":"ContainerStarted","Data":"37e1a0983ea575fee5c0842678059a1708635945f2f0a471ec32fdebc7dbba18"}
Jul 27 18:42:04.421587 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:04.421499    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf" event={"ID":"5fc6b031-e7ad-41bf-952e-a532a490e8c6","Type":"ContainerStarted","Data":"499de542d6fd7102ecf902072278b97187084c39fe08fdb3eac5b45839bd9abe"}
Jul 27 18:42:04.443597 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:04.443546    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf" podStartSLOduration=1.791373429 podStartE2EDuration="4.443528895s" totalImagesPullingTime="2.652155466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:42:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:42:04.441546768 +0000 UTC m=+632.991840418" watchObservedRunningTime="2026-07-27 18:42:04.443528895 +0000 UTC m=+632.993822528"
Jul 27 18:42:11.331818 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:11.331779    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:11.332298 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:11.332110    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:11.344501 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:11.344471    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:11.461223 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:11.461191    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf"
Jul 27 18:42:23.788596 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:23.788556    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"]
Jul 27 18:42:23.795899 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:23.795851    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:23.798802 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:23.798774    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 27 18:42:23.805185 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:23.805163    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"]
Jul 27 18:42:23.937763 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:23.937729    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ebccf45-daca-4e84-a865-354a4ff658d9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v82k8\" (UID: \"0ebccf45-daca-4e84-a865-354a4ff658d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:23.937763 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:23.937766    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8fr6\" (UniqueName: \"kubernetes.io/projected/0ebccf45-daca-4e84-a865-354a4ff658d9-kube-api-access-n8fr6\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v82k8\" (UID: \"0ebccf45-daca-4e84-a865-354a4ff658d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:23.938028 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:23.937796    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ebccf45-daca-4e84-a865-354a4ff658d9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v82k8\" (UID: \"0ebccf45-daca-4e84-a865-354a4ff658d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:23.938028 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:23.937854    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ebccf45-daca-4e84-a865-354a4ff658d9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v82k8\" (UID: \"0ebccf45-daca-4e84-a865-354a4ff658d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:23.938028 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:23.937921    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ebccf45-daca-4e84-a865-354a4ff658d9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v82k8\" (UID: \"0ebccf45-daca-4e84-a865-354a4ff658d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:23.938028 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:23.937940    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ebccf45-daca-4e84-a865-354a4ff658d9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v82k8\" (UID: \"0ebccf45-daca-4e84-a865-354a4ff658d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:24.039160 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:24.039066    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ebccf45-daca-4e84-a865-354a4ff658d9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v82k8\" (UID: \"0ebccf45-daca-4e84-a865-354a4ff658d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:24.039160 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:24.039106    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ebccf45-daca-4e84-a865-354a4ff658d9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v82k8\" (UID: \"0ebccf45-daca-4e84-a865-354a4ff658d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:24.039387 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:24.039244    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ebccf45-daca-4e84-a865-354a4ff658d9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v82k8\" (UID: \"0ebccf45-daca-4e84-a865-354a4ff658d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:24.039387 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:24.039284    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ebccf45-daca-4e84-a865-354a4ff658d9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v82k8\" (UID: \"0ebccf45-daca-4e84-a865-354a4ff658d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:24.039387 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:24.039310    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ebccf45-daca-4e84-a865-354a4ff658d9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v82k8\" (UID: \"0ebccf45-daca-4e84-a865-354a4ff658d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:24.039387 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:24.039334    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8fr6\" (UniqueName: \"kubernetes.io/projected/0ebccf45-daca-4e84-a865-354a4ff658d9-kube-api-access-n8fr6\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v82k8\" (UID: \"0ebccf45-daca-4e84-a865-354a4ff658d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:24.039588 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:24.039469    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ebccf45-daca-4e84-a865-354a4ff658d9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v82k8\" (UID: \"0ebccf45-daca-4e84-a865-354a4ff658d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:24.039588 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:24.039536    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ebccf45-daca-4e84-a865-354a4ff658d9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v82k8\" (UID: \"0ebccf45-daca-4e84-a865-354a4ff658d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:24.039686 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:24.039669    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ebccf45-daca-4e84-a865-354a4ff658d9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v82k8\" (UID: \"0ebccf45-daca-4e84-a865-354a4ff658d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:24.041618 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:24.041586    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ebccf45-daca-4e84-a865-354a4ff658d9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v82k8\" (UID: \"0ebccf45-daca-4e84-a865-354a4ff658d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:24.041999 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:24.041975    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ebccf45-daca-4e84-a865-354a4ff658d9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v82k8\" (UID: \"0ebccf45-daca-4e84-a865-354a4ff658d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:24.050093 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:24.050073    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8fr6\" (UniqueName: \"kubernetes.io/projected/0ebccf45-daca-4e84-a865-354a4ff658d9-kube-api-access-n8fr6\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v82k8\" (UID: \"0ebccf45-daca-4e84-a865-354a4ff658d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:24.106969 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:24.106928    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:24.246374 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:24.246344    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"]
Jul 27 18:42:24.248358 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:42:24.248328    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ebccf45_daca_4e84_a865_354a4ff658d9.slice/crio-e989bd3973a4445583eb7b941fe2cbf0e4ddf382c13d3822e57c2cbd284bf88b WatchSource:0}: Error finding container e989bd3973a4445583eb7b941fe2cbf0e4ddf382c13d3822e57c2cbd284bf88b: Status 404 returned error can't find the container with id e989bd3973a4445583eb7b941fe2cbf0e4ddf382c13d3822e57c2cbd284bf88b
Jul 27 18:42:24.502879 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:24.502833    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8" event={"ID":"0ebccf45-daca-4e84-a865-354a4ff658d9","Type":"ContainerStarted","Data":"ee680e372cf747ce4ea8531215abf7ba9d63475125a00f593a3a3aee87b020c6"}
Jul 27 18:42:24.503040 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:24.502908    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8" event={"ID":"0ebccf45-daca-4e84-a865-354a4ff658d9","Type":"ContainerStarted","Data":"e989bd3973a4445583eb7b941fe2cbf0e4ddf382c13d3822e57c2cbd284bf88b"}
Jul 27 18:42:24.521019 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:24.520898    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8" podStartSLOduration=1.339899942 podStartE2EDuration="1.520857461s" totalImagesPullingTime="180.957519ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:42:24.520768465 +0000 UTC m=+653.071062122" watchObservedRunningTime="2026-07-27 18:42:24.520857461 +0000 UTC m=+653.071151095"
Jul 27 18:42:34.107286 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:34.107244    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:34.107781 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:34.107307    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:34.119645 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:34.119622    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:34.555555 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:34.555527    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v82k8"
Jul 27 18:42:46.201969 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.201933    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"]
Jul 27 18:42:46.205426 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.205409    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:46.214367 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.214343    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 27 18:42:46.222395 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.222371    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"]
Jul 27 18:42:46.323106 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.323064    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4a97a97-176b-4e28-9388-ed7c33b6e250-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v4j68\" (UID: \"c4a97a97-176b-4e28-9388-ed7c33b6e250\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:46.323298 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.323126    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4a97a97-176b-4e28-9388-ed7c33b6e250-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v4j68\" (UID: \"c4a97a97-176b-4e28-9388-ed7c33b6e250\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:46.323298 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.323206    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkkj6\" (UniqueName: \"kubernetes.io/projected/c4a97a97-176b-4e28-9388-ed7c33b6e250-kube-api-access-vkkj6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v4j68\" (UID: \"c4a97a97-176b-4e28-9388-ed7c33b6e250\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:46.323298 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.323251    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4a97a97-176b-4e28-9388-ed7c33b6e250-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v4j68\" (UID: \"c4a97a97-176b-4e28-9388-ed7c33b6e250\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:46.323421 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.323304    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4a97a97-176b-4e28-9388-ed7c33b6e250-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v4j68\" (UID: \"c4a97a97-176b-4e28-9388-ed7c33b6e250\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:46.323421 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.323342    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4a97a97-176b-4e28-9388-ed7c33b6e250-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v4j68\" (UID: \"c4a97a97-176b-4e28-9388-ed7c33b6e250\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:46.423862 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.423820    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4a97a97-176b-4e28-9388-ed7c33b6e250-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v4j68\" (UID: \"c4a97a97-176b-4e28-9388-ed7c33b6e250\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:46.424062 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.423896    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkkj6\" (UniqueName: \"kubernetes.io/projected/c4a97a97-176b-4e28-9388-ed7c33b6e250-kube-api-access-vkkj6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v4j68\" (UID: \"c4a97a97-176b-4e28-9388-ed7c33b6e250\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:46.424062 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.423918    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4a97a97-176b-4e28-9388-ed7c33b6e250-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v4j68\" (UID: \"c4a97a97-176b-4e28-9388-ed7c33b6e250\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:46.424062 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.423954    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4a97a97-176b-4e28-9388-ed7c33b6e250-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v4j68\" (UID: \"c4a97a97-176b-4e28-9388-ed7c33b6e250\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:46.424062 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.423981    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4a97a97-176b-4e28-9388-ed7c33b6e250-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v4j68\" (UID: \"c4a97a97-176b-4e28-9388-ed7c33b6e250\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:46.424062 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.424012    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4a97a97-176b-4e28-9388-ed7c33b6e250-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v4j68\" (UID: \"c4a97a97-176b-4e28-9388-ed7c33b6e250\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:46.424346 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.424323    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4a97a97-176b-4e28-9388-ed7c33b6e250-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v4j68\" (UID: \"c4a97a97-176b-4e28-9388-ed7c33b6e250\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:46.424413 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.424361    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4a97a97-176b-4e28-9388-ed7c33b6e250-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v4j68\" (UID: \"c4a97a97-176b-4e28-9388-ed7c33b6e250\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:46.424454 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.424423    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4a97a97-176b-4e28-9388-ed7c33b6e250-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v4j68\" (UID: \"c4a97a97-176b-4e28-9388-ed7c33b6e250\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:46.426391 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.426371    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4a97a97-176b-4e28-9388-ed7c33b6e250-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v4j68\" (UID: \"c4a97a97-176b-4e28-9388-ed7c33b6e250\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:46.426611 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.426593    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4a97a97-176b-4e28-9388-ed7c33b6e250-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v4j68\" (UID: \"c4a97a97-176b-4e28-9388-ed7c33b6e250\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:46.433025 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.433001    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkkj6\" (UniqueName: \"kubernetes.io/projected/c4a97a97-176b-4e28-9388-ed7c33b6e250-kube-api-access-vkkj6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v4j68\" (UID: \"c4a97a97-176b-4e28-9388-ed7c33b6e250\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:46.515375 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.515334    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:46.654889 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:46.654822    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"]
Jul 27 18:42:46.657020 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:42:46.656990    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4a97a97_176b_4e28_9388_ed7c33b6e250.slice/crio-7b801bac12b0d499515a64dee2eba8449830309aa3e1efedceeb39d7a8546fd6 WatchSource:0}: Error finding container 7b801bac12b0d499515a64dee2eba8449830309aa3e1efedceeb39d7a8546fd6: Status 404 returned error can't find the container with id 7b801bac12b0d499515a64dee2eba8449830309aa3e1efedceeb39d7a8546fd6
Jul 27 18:42:47.600950 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:47.600914    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68" event={"ID":"c4a97a97-176b-4e28-9388-ed7c33b6e250","Type":"ContainerStarted","Data":"45fce81bfab9a479b82c421a869c569fc0722d0c10a276a08e7d47ffd00071dc"}
Jul 27 18:42:47.601351 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:47.600966    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68" event={"ID":"c4a97a97-176b-4e28-9388-ed7c33b6e250","Type":"ContainerStarted","Data":"7b801bac12b0d499515a64dee2eba8449830309aa3e1efedceeb39d7a8546fd6"}
Jul 27 18:42:47.626793 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:47.626743    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68" podStartSLOduration=1.415346953 podStartE2EDuration="1.626723021s" totalImagesPullingTime="211.376068ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:42:47.623437898 +0000 UTC m=+676.173731532" watchObservedRunningTime="2026-07-27 18:42:47.626723021 +0000 UTC m=+676.177016656"
Jul 27 18:42:54.980553 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:54.980515    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"]
Jul 27 18:42:54.983613 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:54.983595    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:42:54.985781 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:54.985760    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 27 18:42:54.995623 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:54.995599    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"]
Jul 27 18:42:55.096336 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:55.096299    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fefdc4f8-7128-41fa-b0c4-c5d919415561-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb\" (UID: \"fefdc4f8-7128-41fa-b0c4-c5d919415561\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:42:55.096493 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:55.096353    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fefdc4f8-7128-41fa-b0c4-c5d919415561-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb\" (UID: \"fefdc4f8-7128-41fa-b0c4-c5d919415561\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:42:55.096493 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:55.096421    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fefdc4f8-7128-41fa-b0c4-c5d919415561-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb\" (UID: \"fefdc4f8-7128-41fa-b0c4-c5d919415561\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:42:55.096493 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:55.096469    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fefdc4f8-7128-41fa-b0c4-c5d919415561-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb\" (UID: \"fefdc4f8-7128-41fa-b0c4-c5d919415561\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:42:55.096642 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:55.096498    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fefdc4f8-7128-41fa-b0c4-c5d919415561-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb\" (UID: \"fefdc4f8-7128-41fa-b0c4-c5d919415561\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:42:55.096642 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:55.096541    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv7nc\" (UniqueName: \"kubernetes.io/projected/fefdc4f8-7128-41fa-b0c4-c5d919415561-kube-api-access-gv7nc\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb\" (UID: \"fefdc4f8-7128-41fa-b0c4-c5d919415561\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:42:55.197765 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:55.197719    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fefdc4f8-7128-41fa-b0c4-c5d919415561-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb\" (UID: \"fefdc4f8-7128-41fa-b0c4-c5d919415561\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:42:55.197998 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:55.197782    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fefdc4f8-7128-41fa-b0c4-c5d919415561-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb\" (UID: \"fefdc4f8-7128-41fa-b0c4-c5d919415561\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:42:55.197998 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:55.197815    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fefdc4f8-7128-41fa-b0c4-c5d919415561-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb\" (UID: \"fefdc4f8-7128-41fa-b0c4-c5d919415561\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:42:55.197998 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:55.197847    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fefdc4f8-7128-41fa-b0c4-c5d919415561-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb\" (UID: \"fefdc4f8-7128-41fa-b0c4-c5d919415561\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:42:55.197998 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:55.197905    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fefdc4f8-7128-41fa-b0c4-c5d919415561-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb\" (UID: \"fefdc4f8-7128-41fa-b0c4-c5d919415561\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:42:55.197998 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:55.197950    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv7nc\" (UniqueName: \"kubernetes.io/projected/fefdc4f8-7128-41fa-b0c4-c5d919415561-kube-api-access-gv7nc\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb\" (UID: \"fefdc4f8-7128-41fa-b0c4-c5d919415561\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:42:55.198268 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:55.198197    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fefdc4f8-7128-41fa-b0c4-c5d919415561-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb\" (UID: \"fefdc4f8-7128-41fa-b0c4-c5d919415561\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:42:55.198268 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:55.198250    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fefdc4f8-7128-41fa-b0c4-c5d919415561-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb\" (UID: \"fefdc4f8-7128-41fa-b0c4-c5d919415561\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:42:55.198347 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:55.198315    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fefdc4f8-7128-41fa-b0c4-c5d919415561-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb\" (UID: \"fefdc4f8-7128-41fa-b0c4-c5d919415561\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:42:55.200395 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:55.200363    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fefdc4f8-7128-41fa-b0c4-c5d919415561-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb\" (UID: \"fefdc4f8-7128-41fa-b0c4-c5d919415561\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:42:55.200536 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:55.200493    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fefdc4f8-7128-41fa-b0c4-c5d919415561-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb\" (UID: \"fefdc4f8-7128-41fa-b0c4-c5d919415561\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:42:55.206522 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:55.206494    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv7nc\" (UniqueName: \"kubernetes.io/projected/fefdc4f8-7128-41fa-b0c4-c5d919415561-kube-api-access-gv7nc\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb\" (UID: \"fefdc4f8-7128-41fa-b0c4-c5d919415561\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:42:55.293564 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:55.293457    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:42:55.442214 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:55.442177    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"]
Jul 27 18:42:55.445113 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:42:55.445085    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfefdc4f8_7128_41fa_b0c4_c5d919415561.slice/crio-ff19ac5158c4d1c4d907b3c62f565a31e65131a7cb7c84ba09445c9406e614c7 WatchSource:0}: Error finding container ff19ac5158c4d1c4d907b3c62f565a31e65131a7cb7c84ba09445c9406e614c7: Status 404 returned error can't find the container with id ff19ac5158c4d1c4d907b3c62f565a31e65131a7cb7c84ba09445c9406e614c7
Jul 27 18:42:55.637054 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:55.636964    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb" event={"ID":"fefdc4f8-7128-41fa-b0c4-c5d919415561","Type":"ContainerStarted","Data":"ff19ac5158c4d1c4d907b3c62f565a31e65131a7cb7c84ba09445c9406e614c7"}
Jul 27 18:42:56.516241 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:56.516201    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:56.516639 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:56.516294    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:56.528935 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:56.528911    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:56.643016 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:56.642970    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb" event={"ID":"fefdc4f8-7128-41fa-b0c4-c5d919415561","Type":"ContainerStarted","Data":"80393db4573b3c27909e7e6f7419024e8f3aabb00d7c4c99049db7ad3826c13f"}
Jul 27 18:42:56.654751 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:56.654714    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v4j68"
Jul 27 18:42:56.667069 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:42:56.667012    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb" podStartSLOduration=2.355557409 podStartE2EDuration="2.666995466s" totalImagesPullingTime="311.438057ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:42:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:42:56.664722876 +0000 UTC m=+685.215016511" watchObservedRunningTime="2026-07-27 18:42:56.666995466 +0000 UTC m=+685.217289098"
Jul 27 18:43:05.293607 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:05.293550    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:43:05.293607 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:05.293622    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:43:05.306107 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:05.306078    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:43:05.696800 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:05.696719    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb"
Jul 27 18:43:56.436186 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:56.436111    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d74fb5494-l2s2b"]
Jul 27 18:43:56.436753 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:56.436301    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7d74fb5494-l2s2b" podUID="dcb42398-4182-4c32-844b-d69dc7d97e81" containerName="authorino" containerID="cri-o://b019be941d07296d62ef162edd567897fadc3b8ef14b42edaad02c317508a637" gracePeriod=30
Jul 27 18:43:56.686609 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:56.686547    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d74fb5494-l2s2b"
Jul 27 18:43:56.751161 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:56.751120    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dcb42398-4182-4c32-844b-d69dc7d97e81-tls-cert\") pod \"dcb42398-4182-4c32-844b-d69dc7d97e81\" (UID: \"dcb42398-4182-4c32-844b-d69dc7d97e81\") "
Jul 27 18:43:56.751357 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:56.751206    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxj9r\" (UniqueName: \"kubernetes.io/projected/dcb42398-4182-4c32-844b-d69dc7d97e81-kube-api-access-nxj9r\") pod \"dcb42398-4182-4c32-844b-d69dc7d97e81\" (UID: \"dcb42398-4182-4c32-844b-d69dc7d97e81\") "
Jul 27 18:43:56.753357 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:56.753314    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcb42398-4182-4c32-844b-d69dc7d97e81-kube-api-access-nxj9r" (OuterVolumeSpecName: "kube-api-access-nxj9r") pod "dcb42398-4182-4c32-844b-d69dc7d97e81" (UID: "dcb42398-4182-4c32-844b-d69dc7d97e81"). InnerVolumeSpecName "kube-api-access-nxj9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:43:56.763167 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:56.763125    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcb42398-4182-4c32-844b-d69dc7d97e81-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "dcb42398-4182-4c32-844b-d69dc7d97e81" (UID: "dcb42398-4182-4c32-844b-d69dc7d97e81"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:43:56.852627 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:56.852587    2562 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dcb42398-4182-4c32-844b-d69dc7d97e81-tls-cert\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 18:43:56.852627 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:56.852621    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxj9r\" (UniqueName: \"kubernetes.io/projected/dcb42398-4182-4c32-844b-d69dc7d97e81-kube-api-access-nxj9r\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 18:43:56.922060 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:56.922024    2562 generic.go:356] "Generic (PLEG): container finished" podID="dcb42398-4182-4c32-844b-d69dc7d97e81" containerID="b019be941d07296d62ef162edd567897fadc3b8ef14b42edaad02c317508a637" exitCode=0
Jul 27 18:43:56.922247 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:56.922066    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d74fb5494-l2s2b" event={"ID":"dcb42398-4182-4c32-844b-d69dc7d97e81","Type":"ContainerDied","Data":"b019be941d07296d62ef162edd567897fadc3b8ef14b42edaad02c317508a637"}
Jul 27 18:43:56.922247 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:56.922101    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d74fb5494-l2s2b" event={"ID":"dcb42398-4182-4c32-844b-d69dc7d97e81","Type":"ContainerDied","Data":"f5bcc123b35d5f245a1f0405116664560ddfa521f48c09f79da142bb845eb419"}
Jul 27 18:43:56.922247 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:56.922122    2562 scope.go:117] "RemoveContainer" containerID="b019be941d07296d62ef162edd567897fadc3b8ef14b42edaad02c317508a637"
Jul 27 18:43:56.922247 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:56.922071    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d74fb5494-l2s2b"
Jul 27 18:43:56.931208 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:56.931191    2562 scope.go:117] "RemoveContainer" containerID="b019be941d07296d62ef162edd567897fadc3b8ef14b42edaad02c317508a637"
Jul 27 18:43:56.931497 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:43:56.931473    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b019be941d07296d62ef162edd567897fadc3b8ef14b42edaad02c317508a637\": container with ID starting with b019be941d07296d62ef162edd567897fadc3b8ef14b42edaad02c317508a637 not found: ID does not exist" containerID="b019be941d07296d62ef162edd567897fadc3b8ef14b42edaad02c317508a637"
Jul 27 18:43:56.931593 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:56.931503    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b019be941d07296d62ef162edd567897fadc3b8ef14b42edaad02c317508a637"} err="failed to get container status \"b019be941d07296d62ef162edd567897fadc3b8ef14b42edaad02c317508a637\": rpc error: code = NotFound desc = could not find container \"b019be941d07296d62ef162edd567897fadc3b8ef14b42edaad02c317508a637\": container with ID starting with b019be941d07296d62ef162edd567897fadc3b8ef14b42edaad02c317508a637 not found: ID does not exist"
Jul 27 18:43:56.951005 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:56.950935    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d74fb5494-l2s2b"]
Jul 27 18:43:56.961769 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:56.961742    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7d74fb5494-l2s2b"]
Jul 27 18:43:57.935915 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:43:57.935814    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcb42398-4182-4c32-844b-d69dc7d97e81" path="/var/lib/kubelet/pods/dcb42398-4182-4c32-844b-d69dc7d97e81/volumes"
Jul 27 18:46:31.805609 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:46:31.805536    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 18:46:31.805609 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:46:31.805549    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 18:51:31.830338 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:51:31.830311    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 18:51:31.832475 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:51:31.832451    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 18:54:54.993202 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:54.993171    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"]
Jul 27 18:54:54.993816 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:54.993463    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcb42398-4182-4c32-844b-d69dc7d97e81" containerName="authorino"
Jul 27 18:54:54.993816 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:54.993474    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcb42398-4182-4c32-844b-d69dc7d97e81" containerName="authorino"
Jul 27 18:54:54.993816 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:54.993533    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcb42398-4182-4c32-844b-d69dc7d97e81" containerName="authorino"
Jul 27 18:54:54.996350 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:54.996331    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:54:55.001564 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.001533    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-ad128bd3-kserve-self-signed-certs\""
Jul 27 18:54:55.026169 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.026138    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"]
Jul 27 18:54:55.056456 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.056424    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-tls-certs\") pod \"e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") " pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:54:55.056644 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.056489    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-home\") pod \"e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") " pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:54:55.056644 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.056531    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-tmp-dir\") pod \"e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") " pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:54:55.056644 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.056574    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-model-cache\") pod \"e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") " pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:54:55.056644 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.056598    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6jkt\" (UniqueName: \"kubernetes.io/projected/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-kube-api-access-c6jkt\") pod \"e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") " pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:54:55.056644 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.056627    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-dshm\") pod \"e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") " pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:54:55.157488 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.157448    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-home\") pod \"e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") " pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:54:55.157681 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.157502    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-tmp-dir\") pod \"e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") " pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:54:55.157681 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.157539    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-model-cache\") pod \"e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") " pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:54:55.157681 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.157557    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6jkt\" (UniqueName: \"kubernetes.io/projected/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-kube-api-access-c6jkt\") pod \"e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") " pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:54:55.157681 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.157578    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-dshm\") pod \"e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") " pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:54:55.157681 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.157599    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-tls-certs\") pod \"e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") " pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:54:55.157985 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.157965    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-home\") pod \"e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") " pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:54:55.158030 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.157988    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-model-cache\") pod \"e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") " pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:54:55.158079 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.158061    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-tmp-dir\") pod \"e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") " pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:54:55.160040 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.160014    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-dshm\") pod \"e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") " pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:54:55.160245 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.160226    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-tls-certs\") pod \"e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") " pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:54:55.165567 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.165547    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6jkt\" (UniqueName: \"kubernetes.io/projected/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-kube-api-access-c6jkt\") pod \"e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") " pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:54:55.309389 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.309306    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:54:55.447473 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.447449    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"]
Jul 27 18:54:55.450046 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:54:55.450016    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod56c332cb_e4da_4c24_8286_4f1bf29ec9a8.slice/crio-123f8abfa8d961990e0022149a7aa462f0e1171718831d21491ba1991aee5b93 WatchSource:0}: Error finding container 123f8abfa8d961990e0022149a7aa462f0e1171718831d21491ba1991aee5b93: Status 404 returned error can't find the container with id 123f8abfa8d961990e0022149a7aa462f0e1171718831d21491ba1991aee5b93
Jul 27 18:54:55.671901 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.671793    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht" event={"ID":"56c332cb-e4da-4c24-8286-4f1bf29ec9a8","Type":"ContainerStarted","Data":"aed592a616f76ae7ad54bf0d686be9f72fcc6f7a591d4dc861f69670d115d5e2"}
Jul 27 18:54:55.671901 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.671838    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht" event={"ID":"56c332cb-e4da-4c24-8286-4f1bf29ec9a8","Type":"ContainerStarted","Data":"123f8abfa8d961990e0022149a7aa462f0e1171718831d21491ba1991aee5b93"}
Jul 27 18:54:55.690426 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:54:55.690368    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht" podStartSLOduration=1.690352826 podStartE2EDuration="1.690352826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:54:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:54:55.688785189 +0000 UTC m=+1404.239078814" watchObservedRunningTime="2026-07-27 18:54:55.690352826 +0000 UTC m=+1404.240646459"
Jul 27 18:55:05.309973 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:05.309940    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:55:05.310367 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:05.309997    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:55:05.322299 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:05.322270    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:55:05.725845 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:05.725818    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:55:31.691255 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.691171    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"]
Jul 27 18:55:31.694860 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.694840    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:31.697033 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.697012    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-ad128bd3-kserve-self-signed-certs\""
Jul 27 18:55:31.704508 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.704481    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"]
Jul 27 18:55:31.788261 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.788224    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-dshm\") pod \"e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") " pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:31.788261 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.788268    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-home\") pod \"e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") " pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:31.788516 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.788286    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz6nv\" (UniqueName: \"kubernetes.io/projected/01fd4b38-23ae-44fc-9e32-835d56f2eeea-kube-api-access-mz6nv\") pod \"e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") " pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:31.788516 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.788410    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01fd4b38-23ae-44fc-9e32-835d56f2eeea-tls-certs\") pod \"e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") " pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:31.788516 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.788493    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-tmp-dir\") pod \"e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") " pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:31.788653 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.788548    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-model-cache\") pod \"e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") " pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:31.889608 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.889558    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-tmp-dir\") pod \"e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") " pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:31.889814 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.889623    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-model-cache\") pod \"e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") " pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:31.889814 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.889652    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-dshm\") pod \"e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") " pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:31.889814 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.889684    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-home\") pod \"e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") " pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:31.889814 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.889707    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz6nv\" (UniqueName: \"kubernetes.io/projected/01fd4b38-23ae-44fc-9e32-835d56f2eeea-kube-api-access-mz6nv\") pod \"e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") " pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:31.889814 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.889776    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01fd4b38-23ae-44fc-9e32-835d56f2eeea-tls-certs\") pod \"e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") " pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:31.890091 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.890072    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-tmp-dir\") pod \"e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") " pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:31.890133 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.890115    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-home\") pod \"e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") " pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:31.890174 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.890147    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-model-cache\") pod \"e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") " pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:31.892077 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.892057    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-ad128bd3-kserve-self-signed-certs\""
Jul 27 18:55:31.892311 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.892293    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-dshm\") pod \"e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") " pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:31.897922 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.897899    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz6nv\" (UniqueName: \"kubernetes.io/projected/01fd4b38-23ae-44fc-9e32-835d56f2eeea-kube-api-access-mz6nv\") pod \"e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") " pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:31.902827 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:31.902805    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01fd4b38-23ae-44fc-9e32-835d56f2eeea-tls-certs\") pod \"e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") " pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:32.005688 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:32.005635    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:32.147458 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:32.147434    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"]
Jul 27 18:55:32.149319 ip-10-0-132-186 kubenswrapper[2562]: W0727 18:55:32.149293    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01fd4b38_23ae_44fc_9e32_835d56f2eeea.slice/crio-d6441f5234e67caf88e09d6c4486e400ba1f74dcdd2a5ca2ab19d61ca5a5ecab WatchSource:0}: Error finding container d6441f5234e67caf88e09d6c4486e400ba1f74dcdd2a5ca2ab19d61ca5a5ecab: Status 404 returned error can't find the container with id d6441f5234e67caf88e09d6c4486e400ba1f74dcdd2a5ca2ab19d61ca5a5ecab
Jul 27 18:55:32.830476 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:32.830440    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q" event={"ID":"01fd4b38-23ae-44fc-9e32-835d56f2eeea","Type":"ContainerStarted","Data":"f0f63b7ce51c760f725b82046ae2b30e22bc8fcbf2e3e6b2cdf6adab963c5669"}
Jul 27 18:55:32.830899 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:32.830485    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q" event={"ID":"01fd4b38-23ae-44fc-9e32-835d56f2eeea","Type":"ContainerStarted","Data":"d6441f5234e67caf88e09d6c4486e400ba1f74dcdd2a5ca2ab19d61ca5a5ecab"}
Jul 27 18:55:32.846621 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:32.846566    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q" podStartSLOduration=1.8465514889999999 podStartE2EDuration="1.846551489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:55:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:55:32.844786084 +0000 UTC m=+1441.395079713" watchObservedRunningTime="2026-07-27 18:55:32.846551489 +0000 UTC m=+1441.396845121"
Jul 27 18:55:42.005937 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:42.005850    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:42.005937 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:42.005947    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:42.018956 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:42.018928    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:55:42.882615 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:55:42.882577    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:56:11.247035 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:11.246998    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"]
Jul 27 18:56:12.356050 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:12.356012    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"]
Jul 27 18:56:26.266555 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:26.266485    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht" podUID="56c332cb-e4da-4c24-8286-4f1bf29ec9a8" containerName="main" containerID="cri-o://aed592a616f76ae7ad54bf0d686be9f72fcc6f7a591d4dc861f69670d115d5e2" gracePeriod=45
Jul 27 18:56:26.513671 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:26.513645    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:56:26.576682 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:26.576582    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6jkt\" (UniqueName: \"kubernetes.io/projected/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-kube-api-access-c6jkt\") pod \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") "
Jul 27 18:56:26.576682 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:26.576655    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-home\") pod \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") "
Jul 27 18:56:26.576682 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:26.576677    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-tmp-dir\") pod \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") "
Jul 27 18:56:26.577034 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:26.576715    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-model-cache\") pod \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") "
Jul 27 18:56:26.577034 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:26.576748    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-tls-certs\") pod \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") "
Jul 27 18:56:26.577034 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:26.576779    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-dshm\") pod \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\" (UID: \"56c332cb-e4da-4c24-8286-4f1bf29ec9a8\") "
Jul 27 18:56:26.577034 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:26.577025    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-home" (OuterVolumeSpecName: "home") pod "56c332cb-e4da-4c24-8286-4f1bf29ec9a8" (UID: "56c332cb-e4da-4c24-8286-4f1bf29ec9a8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:56:26.577298 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:26.577024    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "56c332cb-e4da-4c24-8286-4f1bf29ec9a8" (UID: "56c332cb-e4da-4c24-8286-4f1bf29ec9a8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:56:26.577298 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:26.577047    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-model-cache" (OuterVolumeSpecName: "model-cache") pod "56c332cb-e4da-4c24-8286-4f1bf29ec9a8" (UID: "56c332cb-e4da-4c24-8286-4f1bf29ec9a8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:56:26.577298 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:26.577105    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-home\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 18:56:26.577298 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:26.577122    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-tmp-dir\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 18:56:26.577298 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:26.577137    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-model-cache\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 18:56:26.579114 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:26.579082    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "56c332cb-e4da-4c24-8286-4f1bf29ec9a8" (UID: "56c332cb-e4da-4c24-8286-4f1bf29ec9a8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:56:26.579211 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:26.579126    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-dshm" (OuterVolumeSpecName: "dshm") pod "56c332cb-e4da-4c24-8286-4f1bf29ec9a8" (UID: "56c332cb-e4da-4c24-8286-4f1bf29ec9a8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:56:26.579251 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:26.579213    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-kube-api-access-c6jkt" (OuterVolumeSpecName: "kube-api-access-c6jkt") pod "56c332cb-e4da-4c24-8286-4f1bf29ec9a8" (UID: "56c332cb-e4da-4c24-8286-4f1bf29ec9a8"). InnerVolumeSpecName "kube-api-access-c6jkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:56:26.677928 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:26.677860    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6jkt\" (UniqueName: \"kubernetes.io/projected/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-kube-api-access-c6jkt\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 18:56:26.677928 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:26.677923    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-tls-certs\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 18:56:26.677928 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:26.677934    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56c332cb-e4da-4c24-8286-4f1bf29ec9a8-dshm\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 18:56:27.056706 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.056672    2562 generic.go:356] "Generic (PLEG): container finished" podID="56c332cb-e4da-4c24-8286-4f1bf29ec9a8" containerID="aed592a616f76ae7ad54bf0d686be9f72fcc6f7a591d4dc861f69670d115d5e2" exitCode=0
Jul 27 18:56:27.056897 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.056715    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"
Jul 27 18:56:27.056897 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.056723    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht" event={"ID":"56c332cb-e4da-4c24-8286-4f1bf29ec9a8","Type":"ContainerDied","Data":"aed592a616f76ae7ad54bf0d686be9f72fcc6f7a591d4dc861f69670d115d5e2"}
Jul 27 18:56:27.056897 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.056784    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht" event={"ID":"56c332cb-e4da-4c24-8286-4f1bf29ec9a8","Type":"ContainerDied","Data":"123f8abfa8d961990e0022149a7aa462f0e1171718831d21491ba1991aee5b93"}
Jul 27 18:56:27.056897 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.056806    2562 scope.go:117] "RemoveContainer" containerID="aed592a616f76ae7ad54bf0d686be9f72fcc6f7a591d4dc861f69670d115d5e2"
Jul 27 18:56:27.066186 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.066164    2562 scope.go:117] "RemoveContainer" containerID="aed592a616f76ae7ad54bf0d686be9f72fcc6f7a591d4dc861f69670d115d5e2"
Jul 27 18:56:27.066452 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:56:27.066434    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aed592a616f76ae7ad54bf0d686be9f72fcc6f7a591d4dc861f69670d115d5e2\": container with ID starting with aed592a616f76ae7ad54bf0d686be9f72fcc6f7a591d4dc861f69670d115d5e2 not found: ID does not exist" containerID="aed592a616f76ae7ad54bf0d686be9f72fcc6f7a591d4dc861f69670d115d5e2"
Jul 27 18:56:27.066490 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.066461    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aed592a616f76ae7ad54bf0d686be9f72fcc6f7a591d4dc861f69670d115d5e2"} err="failed to get container status \"aed592a616f76ae7ad54bf0d686be9f72fcc6f7a591d4dc861f69670d115d5e2\": rpc error: code = NotFound desc = could not find container \"aed592a616f76ae7ad54bf0d686be9f72fcc6f7a591d4dc861f69670d115d5e2\": container with ID starting with aed592a616f76ae7ad54bf0d686be9f72fcc6f7a591d4dc861f69670d115d5e2 not found: ID does not exist"
Jul 27 18:56:27.081786 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.081756    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"]
Jul 27 18:56:27.083115 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.083095    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-ad128bd3-kserve-7b8674955f-pzbht"]
Jul 27 18:56:27.375303 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.375220    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q" podUID="01fd4b38-23ae-44fc-9e32-835d56f2eeea" containerName="main" containerID="cri-o://f0f63b7ce51c760f725b82046ae2b30e22bc8fcbf2e3e6b2cdf6adab963c5669" gracePeriod=45
Jul 27 18:56:27.632755 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.632691    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:56:27.685487 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.685446    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-tmp-dir\") pod \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") "
Jul 27 18:56:27.685743 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.685498    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-dshm\") pod \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") "
Jul 27 18:56:27.685743 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.685571    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01fd4b38-23ae-44fc-9e32-835d56f2eeea-tls-certs\") pod \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") "
Jul 27 18:56:27.685743 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.685611    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-home\") pod \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") "
Jul 27 18:56:27.685743 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.685671    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mz6nv\" (UniqueName: \"kubernetes.io/projected/01fd4b38-23ae-44fc-9e32-835d56f2eeea-kube-api-access-mz6nv\") pod \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") "
Jul 27 18:56:27.685743 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.685698    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-model-cache\") pod \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\" (UID: \"01fd4b38-23ae-44fc-9e32-835d56f2eeea\") "
Jul 27 18:56:27.686070 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.685758    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "01fd4b38-23ae-44fc-9e32-835d56f2eeea" (UID: "01fd4b38-23ae-44fc-9e32-835d56f2eeea"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:56:27.686070 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.686016    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-tmp-dir\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 18:56:27.686070 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.686036    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-model-cache" (OuterVolumeSpecName: "model-cache") pod "01fd4b38-23ae-44fc-9e32-835d56f2eeea" (UID: "01fd4b38-23ae-44fc-9e32-835d56f2eeea"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:56:27.686225 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.686187    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-home" (OuterVolumeSpecName: "home") pod "01fd4b38-23ae-44fc-9e32-835d56f2eeea" (UID: "01fd4b38-23ae-44fc-9e32-835d56f2eeea"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:56:27.687920 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.687891    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01fd4b38-23ae-44fc-9e32-835d56f2eeea-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "01fd4b38-23ae-44fc-9e32-835d56f2eeea" (UID: "01fd4b38-23ae-44fc-9e32-835d56f2eeea"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:56:27.688031 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.687953    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-dshm" (OuterVolumeSpecName: "dshm") pod "01fd4b38-23ae-44fc-9e32-835d56f2eeea" (UID: "01fd4b38-23ae-44fc-9e32-835d56f2eeea"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:56:27.688031 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.687952    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01fd4b38-23ae-44fc-9e32-835d56f2eeea-kube-api-access-mz6nv" (OuterVolumeSpecName: "kube-api-access-mz6nv") pod "01fd4b38-23ae-44fc-9e32-835d56f2eeea" (UID: "01fd4b38-23ae-44fc-9e32-835d56f2eeea"). InnerVolumeSpecName "kube-api-access-mz6nv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:56:27.786685 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.786647    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-dshm\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 18:56:27.786685 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.786679    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01fd4b38-23ae-44fc-9e32-835d56f2eeea-tls-certs\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 18:56:27.786685 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.786691    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-home\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 18:56:27.786952 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.786700    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mz6nv\" (UniqueName: \"kubernetes.io/projected/01fd4b38-23ae-44fc-9e32-835d56f2eeea-kube-api-access-mz6nv\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 18:56:27.786952 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.786711    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01fd4b38-23ae-44fc-9e32-835d56f2eeea-model-cache\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 18:56:27.936932 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:27.936831    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56c332cb-e4da-4c24-8286-4f1bf29ec9a8" path="/var/lib/kubelet/pods/56c332cb-e4da-4c24-8286-4f1bf29ec9a8/volumes"
Jul 27 18:56:28.064199 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:28.064166    2562 generic.go:356] "Generic (PLEG): container finished" podID="01fd4b38-23ae-44fc-9e32-835d56f2eeea" containerID="f0f63b7ce51c760f725b82046ae2b30e22bc8fcbf2e3e6b2cdf6adab963c5669" exitCode=0
Jul 27 18:56:28.064376 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:28.064222    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"
Jul 27 18:56:28.064376 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:28.064225    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q" event={"ID":"01fd4b38-23ae-44fc-9e32-835d56f2eeea","Type":"ContainerDied","Data":"f0f63b7ce51c760f725b82046ae2b30e22bc8fcbf2e3e6b2cdf6adab963c5669"}
Jul 27 18:56:28.064376 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:28.064328    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q" event={"ID":"01fd4b38-23ae-44fc-9e32-835d56f2eeea","Type":"ContainerDied","Data":"d6441f5234e67caf88e09d6c4486e400ba1f74dcdd2a5ca2ab19d61ca5a5ecab"}
Jul 27 18:56:28.064376 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:28.064345    2562 scope.go:117] "RemoveContainer" containerID="f0f63b7ce51c760f725b82046ae2b30e22bc8fcbf2e3e6b2cdf6adab963c5669"
Jul 27 18:56:28.073241 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:28.073219    2562 scope.go:117] "RemoveContainer" containerID="f0f63b7ce51c760f725b82046ae2b30e22bc8fcbf2e3e6b2cdf6adab963c5669"
Jul 27 18:56:28.073576 ip-10-0-132-186 kubenswrapper[2562]: E0727 18:56:28.073556    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0f63b7ce51c760f725b82046ae2b30e22bc8fcbf2e3e6b2cdf6adab963c5669\": container with ID starting with f0f63b7ce51c760f725b82046ae2b30e22bc8fcbf2e3e6b2cdf6adab963c5669 not found: ID does not exist" containerID="f0f63b7ce51c760f725b82046ae2b30e22bc8fcbf2e3e6b2cdf6adab963c5669"
Jul 27 18:56:28.073631 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:28.073585    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0f63b7ce51c760f725b82046ae2b30e22bc8fcbf2e3e6b2cdf6adab963c5669"} err="failed to get container status \"f0f63b7ce51c760f725b82046ae2b30e22bc8fcbf2e3e6b2cdf6adab963c5669\": rpc error: code = NotFound desc = could not find container \"f0f63b7ce51c760f725b82046ae2b30e22bc8fcbf2e3e6b2cdf6adab963c5669\": container with ID starting with f0f63b7ce51c760f725b82046ae2b30e22bc8fcbf2e3e6b2cdf6adab963c5669 not found: ID does not exist"
Jul 27 18:56:28.080861 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:28.080830    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"]
Jul 27 18:56:28.084687 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:28.084658    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-ad128bd3-kserve-997c64d55-z7q6q"]
Jul 27 18:56:29.937064 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:29.937027    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01fd4b38-23ae-44fc-9e32-835d56f2eeea" path="/var/lib/kubelet/pods/01fd4b38-23ae-44fc-9e32-835d56f2eeea/volumes"
Jul 27 18:56:31.854448 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:31.854420    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 18:56:31.856933 ip-10-0-132-186 kubenswrapper[2562]: I0727 18:56:31.856908    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 19:01:08.619200 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:08.619159    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e045185e-j46cv"]
Jul 27 19:01:08.619663 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:08.619478    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01fd4b38-23ae-44fc-9e32-835d56f2eeea" containerName="main"
Jul 27 19:01:08.619663 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:08.619490    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="01fd4b38-23ae-44fc-9e32-835d56f2eeea" containerName="main"
Jul 27 19:01:08.619663 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:08.619508    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56c332cb-e4da-4c24-8286-4f1bf29ec9a8" containerName="main"
Jul 27 19:01:08.619663 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:08.619513    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="56c332cb-e4da-4c24-8286-4f1bf29ec9a8" containerName="main"
Jul 27 19:01:08.619663 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:08.619579    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="01fd4b38-23ae-44fc-9e32-835d56f2eeea" containerName="main"
Jul 27 19:01:08.619663 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:08.619589    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="56c332cb-e4da-4c24-8286-4f1bf29ec9a8" containerName="main"
Jul 27 19:01:08.622488 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:08.622469    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e045185e-j46cv"
Jul 27 19:01:08.624699 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:08.624678    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 19:01:08.624825 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:08.624805    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gwtlm\""
Jul 27 19:01:08.629402 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:08.629376    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e045185e-j46cv"]
Jul 27 19:01:08.677710 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:08.677674    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a99002a6-f3a9-4718-a3ec-f9ad434ec2dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e045185e-j46cv\" (UID: \"a99002a6-f3a9-4718-a3ec-f9ad434ec2dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e045185e-j46cv"
Jul 27 19:01:08.778325 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:08.778285    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a99002a6-f3a9-4718-a3ec-f9ad434ec2dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e045185e-j46cv\" (UID: \"a99002a6-f3a9-4718-a3ec-f9ad434ec2dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e045185e-j46cv"
Jul 27 19:01:08.778951 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:08.778930    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a99002a6-f3a9-4718-a3ec-f9ad434ec2dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e045185e-j46cv\" (UID: \"a99002a6-f3a9-4718-a3ec-f9ad434ec2dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e045185e-j46cv"
Jul 27 19:01:08.933981 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:08.933891    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e045185e-j46cv"
Jul 27 19:01:09.074497 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:09.074467    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e045185e-j46cv"]
Jul 27 19:01:09.077149 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:01:09.077114    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda99002a6_f3a9_4718_a3ec_f9ad434ec2dd.slice/crio-e07763974b54d30f5e2648c55779d9c3751cc28fbef7de0ce6f66297d6fa5b5a WatchSource:0}: Error finding container e07763974b54d30f5e2648c55779d9c3751cc28fbef7de0ce6f66297d6fa5b5a: Status 404 returned error can't find the container with id e07763974b54d30f5e2648c55779d9c3751cc28fbef7de0ce6f66297d6fa5b5a
Jul 27 19:01:09.079116 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:09.079099    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 19:01:09.277386 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:09.277341    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e045185e-j46cv" event={"ID":"a99002a6-f3a9-4718-a3ec-f9ad434ec2dd","Type":"ContainerStarted","Data":"e07763974b54d30f5e2648c55779d9c3751cc28fbef7de0ce6f66297d6fa5b5a"}
Jul 27 19:01:14.301720 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:14.301675    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e045185e-j46cv" event={"ID":"a99002a6-f3a9-4718-a3ec-f9ad434ec2dd","Type":"ContainerStarted","Data":"0e979a04f979e88d212005086309be25f963f16007a8255622eab6104a10814b"}
Jul 27 19:01:14.316428 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:14.316377    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e045185e-j46cv" podStartSLOduration=1.441525984 podStartE2EDuration="6.316359433s" totalImagesPullingTime="4.874833449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:01:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:01:14.315641201 +0000 UTC m=+1782.865934835" watchObservedRunningTime="2026-07-27 19:01:14.316359433 +0000 UTC m=+1782.866653066"
Jul 27 19:01:15.307539 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:15.307499    2562 generic.go:356] "Generic (PLEG): container finished" podID="a99002a6-f3a9-4718-a3ec-f9ad434ec2dd" containerID="0e979a04f979e88d212005086309be25f963f16007a8255622eab6104a10814b" exitCode=0
Jul 27 19:01:15.307970 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:15.307580    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e045185e-j46cv" event={"ID":"a99002a6-f3a9-4718-a3ec-f9ad434ec2dd","Type":"ContainerDied","Data":"0e979a04f979e88d212005086309be25f963f16007a8255622eab6104a10814b"}
Jul 27 19:01:16.449452 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:16.449426    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e045185e-j46cv"
Jul 27 19:01:16.545779 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:16.545741    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a99002a6-f3a9-4718-a3ec-f9ad434ec2dd-maas-api-service-ca\") pod \"a99002a6-f3a9-4718-a3ec-f9ad434ec2dd\" (UID: \"a99002a6-f3a9-4718-a3ec-f9ad434ec2dd\") "
Jul 27 19:01:16.546150 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:16.546127    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a99002a6-f3a9-4718-a3ec-f9ad434ec2dd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a99002a6-f3a9-4718-a3ec-f9ad434ec2dd" (UID: "a99002a6-f3a9-4718-a3ec-f9ad434ec2dd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:01:16.646516 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:16.646433    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a99002a6-f3a9-4718-a3ec-f9ad434ec2dd-maas-api-service-ca\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:01:17.318190 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:17.318159    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e045185e-j46cv"
Jul 27 19:01:17.318367 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:17.318169    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e045185e-j46cv" event={"ID":"a99002a6-f3a9-4718-a3ec-f9ad434ec2dd","Type":"ContainerDied","Data":"e07763974b54d30f5e2648c55779d9c3751cc28fbef7de0ce6f66297d6fa5b5a"}
Jul 27 19:01:17.318367 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:17.318281    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e07763974b54d30f5e2648c55779d9c3751cc28fbef7de0ce6f66297d6fa5b5a"
Jul 27 19:01:23.693209 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:23.693123    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e045185e-j46cv"]
Jul 27 19:01:23.695959 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:23.695927    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e045185e-j46cv"]
Jul 27 19:01:23.936076 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:23.936040    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a99002a6-f3a9-4718-a3ec-f9ad434ec2dd" path="/var/lib/kubelet/pods/a99002a6-f3a9-4718-a3ec-f9ad434ec2dd/volumes"
Jul 27 19:01:31.888360 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:31.888325    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 19:01:31.890477 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:31.890447    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 19:01:52.571472 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:52.571437    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf4c6eb2-fd9jx"]
Jul 27 19:01:52.571932 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:52.571751    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a99002a6-f3a9-4718-a3ec-f9ad434ec2dd" containerName="revoke-keys"
Jul 27 19:01:52.571932 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:52.571763    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a99002a6-f3a9-4718-a3ec-f9ad434ec2dd" containerName="revoke-keys"
Jul 27 19:01:52.571932 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:52.571841    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a99002a6-f3a9-4718-a3ec-f9ad434ec2dd" containerName="revoke-keys"
Jul 27 19:01:52.574801 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:52.574783    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf4c6eb2-fd9jx"
Jul 27 19:01:52.577014 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:52.576992    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 19:01:52.577014 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:52.577013    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gwtlm\""
Jul 27 19:01:52.583344 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:52.583314    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf4c6eb2-fd9jx"]
Jul 27 19:01:52.761179 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:52.761138    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99bb6059-85aa-42a7-a845-9f50951db1db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cf4c6eb2-fd9jx\" (UID: \"99bb6059-85aa-42a7-a845-9f50951db1db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf4c6eb2-fd9jx"
Jul 27 19:01:52.862819 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:52.862731    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99bb6059-85aa-42a7-a845-9f50951db1db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cf4c6eb2-fd9jx\" (UID: \"99bb6059-85aa-42a7-a845-9f50951db1db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf4c6eb2-fd9jx"
Jul 27 19:01:52.863395 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:52.863372    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99bb6059-85aa-42a7-a845-9f50951db1db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cf4c6eb2-fd9jx\" (UID: \"99bb6059-85aa-42a7-a845-9f50951db1db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf4c6eb2-fd9jx"
Jul 27 19:01:52.886503 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:52.886467    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf4c6eb2-fd9jx"
Jul 27 19:01:53.028637 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:53.028609    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf4c6eb2-fd9jx"]
Jul 27 19:01:53.031713 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:01:53.031687    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99bb6059_85aa_42a7_a845_9f50951db1db.slice/crio-8ae2401eb925819ba87afaedd3f936476f58ee05e4bae8916d90e0ded0284615 WatchSource:0}: Error finding container 8ae2401eb925819ba87afaedd3f936476f58ee05e4bae8916d90e0ded0284615: Status 404 returned error can't find the container with id 8ae2401eb925819ba87afaedd3f936476f58ee05e4bae8916d90e0ded0284615
Jul 27 19:01:53.474796 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:53.474747    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf4c6eb2-fd9jx" event={"ID":"99bb6059-85aa-42a7-a845-9f50951db1db","Type":"ContainerStarted","Data":"f2772a8c3c599654a6a6d6221720a8bf0e3b03c22581168dcd04775409ea9f03"}
Jul 27 19:01:53.475015 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:53.474815    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf4c6eb2-fd9jx" event={"ID":"99bb6059-85aa-42a7-a845-9f50951db1db","Type":"ContainerStarted","Data":"8ae2401eb925819ba87afaedd3f936476f58ee05e4bae8916d90e0ded0284615"}
Jul 27 19:01:53.509698 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:53.509643    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf4c6eb2-fd9jx" podStartSLOduration=1.509625145 podStartE2EDuration="1.509625145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:01:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:01:53.508218141 +0000 UTC m=+1822.058511774" watchObservedRunningTime="2026-07-27 19:01:53.509625145 +0000 UTC m=+1822.059918777"
Jul 27 19:01:54.481276 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:54.481237    2562 generic.go:356] "Generic (PLEG): container finished" podID="99bb6059-85aa-42a7-a845-9f50951db1db" containerID="f2772a8c3c599654a6a6d6221720a8bf0e3b03c22581168dcd04775409ea9f03" exitCode=0
Jul 27 19:01:54.481276 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:54.481278    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf4c6eb2-fd9jx" event={"ID":"99bb6059-85aa-42a7-a845-9f50951db1db","Type":"ContainerDied","Data":"f2772a8c3c599654a6a6d6221720a8bf0e3b03c22581168dcd04775409ea9f03"}
Jul 27 19:01:55.620125 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:55.620098    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf4c6eb2-fd9jx"
Jul 27 19:01:55.790389 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:55.790292    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99bb6059-85aa-42a7-a845-9f50951db1db-maas-api-service-ca\") pod \"99bb6059-85aa-42a7-a845-9f50951db1db\" (UID: \"99bb6059-85aa-42a7-a845-9f50951db1db\") "
Jul 27 19:01:55.790646 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:55.790621    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99bb6059-85aa-42a7-a845-9f50951db1db-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "99bb6059-85aa-42a7-a845-9f50951db1db" (UID: "99bb6059-85aa-42a7-a845-9f50951db1db"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:01:55.891273 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:55.891225    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99bb6059-85aa-42a7-a845-9f50951db1db-maas-api-service-ca\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:01:56.491840 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:56.491807    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf4c6eb2-fd9jx"
Jul 27 19:01:56.492034 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:56.491840    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf4c6eb2-fd9jx" event={"ID":"99bb6059-85aa-42a7-a845-9f50951db1db","Type":"ContainerDied","Data":"8ae2401eb925819ba87afaedd3f936476f58ee05e4bae8916d90e0ded0284615"}
Jul 27 19:01:56.492034 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:01:56.491938    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ae2401eb925819ba87afaedd3f936476f58ee05e4bae8916d90e0ded0284615"
Jul 27 19:02:07.648361 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:07.648310    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf4c6eb2-fd9jx"]
Jul 27 19:02:07.650751 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:07.650718    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf4c6eb2-fd9jx"]
Jul 27 19:02:07.938078 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:07.937991    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99bb6059-85aa-42a7-a845-9f50951db1db" path="/var/lib/kubelet/pods/99bb6059-85aa-42a7-a845-9f50951db1db/volumes"
Jul 27 19:02:36.901179 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:36.901140    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-27df8036-zmbhk"]
Jul 27 19:02:36.901686 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:36.901495    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99bb6059-85aa-42a7-a845-9f50951db1db" containerName="revoke-keys"
Jul 27 19:02:36.901686 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:36.901506    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="99bb6059-85aa-42a7-a845-9f50951db1db" containerName="revoke-keys"
Jul 27 19:02:36.901686 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:36.901564    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="99bb6059-85aa-42a7-a845-9f50951db1db" containerName="revoke-keys"
Jul 27 19:02:36.904408 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:36.904387    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-27df8036-zmbhk"
Jul 27 19:02:36.906561 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:36.906533    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 19:02:36.906716 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:36.906596    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gwtlm\""
Jul 27 19:02:36.910977 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:36.910944    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-27df8036-zmbhk"]
Jul 27 19:02:37.073967 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:37.073907    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f155e9a-c938-40b9-8b7d-6b2b6be90a14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-27df8036-zmbhk\" (UID: \"8f155e9a-c938-40b9-8b7d-6b2b6be90a14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-27df8036-zmbhk"
Jul 27 19:02:37.174975 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:37.174841    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f155e9a-c938-40b9-8b7d-6b2b6be90a14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-27df8036-zmbhk\" (UID: \"8f155e9a-c938-40b9-8b7d-6b2b6be90a14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-27df8036-zmbhk"
Jul 27 19:02:37.175482 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:37.175463    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f155e9a-c938-40b9-8b7d-6b2b6be90a14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-27df8036-zmbhk\" (UID: \"8f155e9a-c938-40b9-8b7d-6b2b6be90a14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-27df8036-zmbhk"
Jul 27 19:02:37.216519 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:37.216475    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-27df8036-zmbhk"
Jul 27 19:02:37.354551 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:37.354527    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-27df8036-zmbhk"]
Jul 27 19:02:37.356973 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:02:37.356940    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f155e9a_c938_40b9_8b7d_6b2b6be90a14.slice/crio-457a4982352713184a59766abd52d44df34c3c4248f840a42eb2396039e1475f WatchSource:0}: Error finding container 457a4982352713184a59766abd52d44df34c3c4248f840a42eb2396039e1475f: Status 404 returned error can't find the container with id 457a4982352713184a59766abd52d44df34c3c4248f840a42eb2396039e1475f
Jul 27 19:02:37.673824 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:37.673786    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-27df8036-zmbhk" event={"ID":"8f155e9a-c938-40b9-8b7d-6b2b6be90a14","Type":"ContainerStarted","Data":"ff675bb9146f08772f9fa2671e0a3b026a0d3f61737ef8f26fa9c82f1fdd0efc"}
Jul 27 19:02:37.674091 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:37.673849    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-27df8036-zmbhk" event={"ID":"8f155e9a-c938-40b9-8b7d-6b2b6be90a14","Type":"ContainerStarted","Data":"457a4982352713184a59766abd52d44df34c3c4248f840a42eb2396039e1475f"}
Jul 27 19:02:37.689259 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:37.689198    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-27df8036-zmbhk" podStartSLOduration=1.689180213 podStartE2EDuration="1.689180213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:02:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:02:37.68798877 +0000 UTC m=+1866.238282403" watchObservedRunningTime="2026-07-27 19:02:37.689180213 +0000 UTC m=+1866.239473846"
Jul 27 19:02:38.680033 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:38.679993    2562 generic.go:356] "Generic (PLEG): container finished" podID="8f155e9a-c938-40b9-8b7d-6b2b6be90a14" containerID="ff675bb9146f08772f9fa2671e0a3b026a0d3f61737ef8f26fa9c82f1fdd0efc" exitCode=0
Jul 27 19:02:38.680419 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:38.680063    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-27df8036-zmbhk" event={"ID":"8f155e9a-c938-40b9-8b7d-6b2b6be90a14","Type":"ContainerDied","Data":"ff675bb9146f08772f9fa2671e0a3b026a0d3f61737ef8f26fa9c82f1fdd0efc"}
Jul 27 19:02:39.816377 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:39.816348    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-27df8036-zmbhk"
Jul 27 19:02:39.900894 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:39.900834    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f155e9a-c938-40b9-8b7d-6b2b6be90a14-maas-api-service-ca\") pod \"8f155e9a-c938-40b9-8b7d-6b2b6be90a14\" (UID: \"8f155e9a-c938-40b9-8b7d-6b2b6be90a14\") "
Jul 27 19:02:39.901246 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:39.901211    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f155e9a-c938-40b9-8b7d-6b2b6be90a14-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8f155e9a-c938-40b9-8b7d-6b2b6be90a14" (UID: "8f155e9a-c938-40b9-8b7d-6b2b6be90a14"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:02:40.002039 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:40.001994    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f155e9a-c938-40b9-8b7d-6b2b6be90a14-maas-api-service-ca\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:02:40.691642 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:40.691604    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-27df8036-zmbhk" event={"ID":"8f155e9a-c938-40b9-8b7d-6b2b6be90a14","Type":"ContainerDied","Data":"457a4982352713184a59766abd52d44df34c3c4248f840a42eb2396039e1475f"}
Jul 27 19:02:40.691642 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:40.691643    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-27df8036-zmbhk"
Jul 27 19:02:40.691829 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:40.691682    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="457a4982352713184a59766abd52d44df34c3c4248f840a42eb2396039e1475f"
Jul 27 19:02:51.971248 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:51.971204    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-27df8036-zmbhk"]
Jul 27 19:02:51.973281 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:51.973255    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-27df8036-zmbhk"]
Jul 27 19:02:53.936544 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:02:53.936504    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f155e9a-c938-40b9-8b7d-6b2b6be90a14" path="/var/lib/kubelet/pods/8f155e9a-c938-40b9-8b7d-6b2b6be90a14/volumes"
Jul 27 19:04:46.435685 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:46.435646    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5863a6bc-7b792"]
Jul 27 19:04:46.436196 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:46.435976    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f155e9a-c938-40b9-8b7d-6b2b6be90a14" containerName="revoke-keys"
Jul 27 19:04:46.436196 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:46.435987    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f155e9a-c938-40b9-8b7d-6b2b6be90a14" containerName="revoke-keys"
Jul 27 19:04:46.436196 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:46.436038    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f155e9a-c938-40b9-8b7d-6b2b6be90a14" containerName="revoke-keys"
Jul 27 19:04:46.438988 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:46.438961    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5863a6bc-7b792"
Jul 27 19:04:46.441493 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:46.441470    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gwtlm\""
Jul 27 19:04:46.441606 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:46.441507    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 19:04:46.447541 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:46.447509    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5863a6bc-7b792"]
Jul 27 19:04:46.532167 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:46.532131    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6672cb6f-4a67-456f-93cf-cba83ff3da0a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5863a6bc-7b792\" (UID: \"6672cb6f-4a67-456f-93cf-cba83ff3da0a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5863a6bc-7b792"
Jul 27 19:04:46.633461 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:46.633418    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6672cb6f-4a67-456f-93cf-cba83ff3da0a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5863a6bc-7b792\" (UID: \"6672cb6f-4a67-456f-93cf-cba83ff3da0a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5863a6bc-7b792"
Jul 27 19:04:46.634121 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:46.634100    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6672cb6f-4a67-456f-93cf-cba83ff3da0a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5863a6bc-7b792\" (UID: \"6672cb6f-4a67-456f-93cf-cba83ff3da0a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5863a6bc-7b792"
Jul 27 19:04:46.757780 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:46.757728    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5863a6bc-7b792"
Jul 27 19:04:46.895864 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:46.895828    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5863a6bc-7b792"]
Jul 27 19:04:46.898781 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:04:46.898754    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6672cb6f_4a67_456f_93cf_cba83ff3da0a.slice/crio-a93b4256958540a7e658a31342d4f63e1fc974ba99110db9ae0ad194fa1daa82 WatchSource:0}: Error finding container a93b4256958540a7e658a31342d4f63e1fc974ba99110db9ae0ad194fa1daa82: Status 404 returned error can't find the container with id a93b4256958540a7e658a31342d4f63e1fc974ba99110db9ae0ad194fa1daa82
Jul 27 19:04:47.240562 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:47.240521    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5863a6bc-7b792" event={"ID":"6672cb6f-4a67-456f-93cf-cba83ff3da0a","Type":"ContainerStarted","Data":"200c29200d340d33a64a5cbcbb4a54de433ee0f645bf7a4902dd95c1563c99c9"}
Jul 27 19:04:47.240707 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:47.240586    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5863a6bc-7b792" event={"ID":"6672cb6f-4a67-456f-93cf-cba83ff3da0a","Type":"ContainerStarted","Data":"a93b4256958540a7e658a31342d4f63e1fc974ba99110db9ae0ad194fa1daa82"}
Jul 27 19:04:47.257543 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:47.257490    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5863a6bc-7b792" podStartSLOduration=1.257473827 podStartE2EDuration="1.257473827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:04:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:04:47.256286489 +0000 UTC m=+1995.806580124" watchObservedRunningTime="2026-07-27 19:04:47.257473827 +0000 UTC m=+1995.807767461"
Jul 27 19:04:48.246563 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:48.246522    2562 generic.go:356] "Generic (PLEG): container finished" podID="6672cb6f-4a67-456f-93cf-cba83ff3da0a" containerID="200c29200d340d33a64a5cbcbb4a54de433ee0f645bf7a4902dd95c1563c99c9" exitCode=0
Jul 27 19:04:48.246985 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:48.246610    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5863a6bc-7b792" event={"ID":"6672cb6f-4a67-456f-93cf-cba83ff3da0a","Type":"ContainerDied","Data":"200c29200d340d33a64a5cbcbb4a54de433ee0f645bf7a4902dd95c1563c99c9"}
Jul 27 19:04:49.382892 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:49.382852    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5863a6bc-7b792"
Jul 27 19:04:49.461066 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:49.461019    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6672cb6f-4a67-456f-93cf-cba83ff3da0a-maas-api-service-ca\") pod \"6672cb6f-4a67-456f-93cf-cba83ff3da0a\" (UID: \"6672cb6f-4a67-456f-93cf-cba83ff3da0a\") "
Jul 27 19:04:49.461408 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:49.461383    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6672cb6f-4a67-456f-93cf-cba83ff3da0a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6672cb6f-4a67-456f-93cf-cba83ff3da0a" (UID: "6672cb6f-4a67-456f-93cf-cba83ff3da0a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:04:49.562043 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:49.561947    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6672cb6f-4a67-456f-93cf-cba83ff3da0a-maas-api-service-ca\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:04:50.257012 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:50.256983    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5863a6bc-7b792"
Jul 27 19:04:50.257163 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:50.257017    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5863a6bc-7b792" event={"ID":"6672cb6f-4a67-456f-93cf-cba83ff3da0a","Type":"ContainerDied","Data":"a93b4256958540a7e658a31342d4f63e1fc974ba99110db9ae0ad194fa1daa82"}
Jul 27 19:04:50.257163 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:04:50.257064    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a93b4256958540a7e658a31342d4f63e1fc974ba99110db9ae0ad194fa1daa82"
Jul 27 19:05:01.515677 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:01.515641    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5863a6bc-7b792"]
Jul 27 19:05:01.519458 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:01.519429    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5863a6bc-7b792"]
Jul 27 19:05:01.937141 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:01.937062    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6672cb6f-4a67-456f-93cf-cba83ff3da0a" path="/var/lib/kubelet/pods/6672cb6f-4a67-456f-93cf-cba83ff3da0a/volumes"
Jul 27 19:05:42.367937 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.367901    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"]
Jul 27 19:05:42.368485 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.368361    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6672cb6f-4a67-456f-93cf-cba83ff3da0a" containerName="revoke-keys"
Jul 27 19:05:42.368485 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.368379    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6672cb6f-4a67-456f-93cf-cba83ff3da0a" containerName="revoke-keys"
Jul 27 19:05:42.368485 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.368485    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6672cb6f-4a67-456f-93cf-cba83ff3da0a" containerName="revoke-keys"
Jul 27 19:05:42.371145 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.371122    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:42.373495 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.373467    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-dcc83a9c\"/\"openshift-service-ca.crt\""
Jul 27 19:05:42.374238 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.374212    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-dcc83a9c\"/\"e2e-lifecycle-model-dcc83a9c-kserve-self-signed-certs\""
Jul 27 19:05:42.374238 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.374225    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-dcc83a9c\"/\"default-dockercfg-x7k2n\""
Jul 27 19:05:42.374403 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.374225    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-dcc83a9c\"/\"kube-root-ca.crt\""
Jul 27 19:05:42.379132 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.379108    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"]
Jul 27 19:05:42.514589 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.514554    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-dshm\") pod \"e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") " pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:42.514765 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.514618    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-model-cache\") pod \"e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") " pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:42.514765 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.514658    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-home\") pod \"e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") " pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:42.514765 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.514682    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4xlx\" (UniqueName: \"kubernetes.io/projected/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-kube-api-access-f4xlx\") pod \"e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") " pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:42.514765 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.514752    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-tls-certs\") pod \"e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") " pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:42.514933 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.514779    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-tmp-dir\") pod \"e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") " pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:42.615712 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.615666    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-model-cache\") pod \"e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") " pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:42.615712 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.615717    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-home\") pod \"e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") " pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:42.616025 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.615740    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4xlx\" (UniqueName: \"kubernetes.io/projected/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-kube-api-access-f4xlx\") pod \"e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") " pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:42.616025 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.615788    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-tls-certs\") pod \"e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") " pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:42.616025 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.615809    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-tmp-dir\") pod \"e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") " pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:42.616025 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.615832    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-dshm\") pod \"e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") " pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:42.616258 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.616232    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-home\") pod \"e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") " pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:42.616307 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.616253    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-model-cache\") pod \"e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") " pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:42.616346 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.616310    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-tmp-dir\") pod \"e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") " pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:42.618360 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.618300    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-dshm\") pod \"e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") " pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:42.618589 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.618571    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-tls-certs\") pod \"e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") " pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:42.626713 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.626686    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4xlx\" (UniqueName: \"kubernetes.io/projected/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-kube-api-access-f4xlx\") pod \"e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") " pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:42.681682 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.681644    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:42.820214 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:42.820159    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"]
Jul 27 19:05:42.822507 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:05:42.822482    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb5e10b6_f988_49b4_8caa_c1a85d3b98aa.slice/crio-37642f777e3e7063131ed62a50d96ee20fd8474a7d3bb24779a6e9b1eded9272 WatchSource:0}: Error finding container 37642f777e3e7063131ed62a50d96ee20fd8474a7d3bb24779a6e9b1eded9272: Status 404 returned error can't find the container with id 37642f777e3e7063131ed62a50d96ee20fd8474a7d3bb24779a6e9b1eded9272
Jul 27 19:05:43.503270 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:43.503231    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph" event={"ID":"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa","Type":"ContainerStarted","Data":"85d492e26b48ef297279841581a610f76db817d4f8c355a4b7a3a9fec04db450"}
Jul 27 19:05:43.503270 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:43.503272    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph" event={"ID":"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa","Type":"ContainerStarted","Data":"37642f777e3e7063131ed62a50d96ee20fd8474a7d3bb24779a6e9b1eded9272"}
Jul 27 19:05:43.521542 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:43.521480    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph" podStartSLOduration=1.521462997 podStartE2EDuration="1.521462997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:05:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:05:43.520011141 +0000 UTC m=+2052.070304773" watchObservedRunningTime="2026-07-27 19:05:43.521462997 +0000 UTC m=+2052.071756631"
Jul 27 19:05:52.682644 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:52.682550    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:52.682644 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:52.682619    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:52.695271 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:52.695242    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:05:53.558589 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:05:53.558553    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:06:31.917757 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:06:31.917729    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 19:06:31.920461 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:06:31.920435    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 19:07:31.994622 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:07:31.994541    2562 scope.go:117] "RemoveContainer" containerID="0e979a04f979e88d212005086309be25f963f16007a8255622eab6104a10814b"
Jul 27 19:08:32.013473 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:08:32.013437    2562 scope.go:117] "RemoveContainer" containerID="f2772a8c3c599654a6a6d6221720a8bf0e3b03c22581168dcd04775409ea9f03"
Jul 27 19:09:32.032106 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:09:32.032067    2562 scope.go:117] "RemoveContainer" containerID="ff675bb9146f08772f9fa2671e0a3b026a0d3f61737ef8f26fa9c82f1fdd0efc"
Jul 27 19:09:44.585731 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:09:44.585691    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"]
Jul 27 19:09:59.605968 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:09:59.605921    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph" podUID="fb5e10b6-f988-49b4-8caa-c1a85d3b98aa" containerName="main" containerID="cri-o://85d492e26b48ef297279841581a610f76db817d4f8c355a4b7a3a9fec04db450" gracePeriod=45
Jul 27 19:09:59.863747 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:09:59.863675    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:09:59.926227 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:09:59.926194    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-tmp-dir\") pod \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") "
Jul 27 19:09:59.926227 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:09:59.926233    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-home\") pod \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") "
Jul 27 19:09:59.926565 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:09:59.926282    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4xlx\" (UniqueName: \"kubernetes.io/projected/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-kube-api-access-f4xlx\") pod \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") "
Jul 27 19:09:59.926565 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:09:59.926338    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-model-cache\") pod \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") "
Jul 27 19:09:59.926565 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:09:59.926385    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-dshm\") pod \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") "
Jul 27 19:09:59.926565 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:09:59.926408    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-tls-certs\") pod \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\" (UID: \"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa\") "
Jul 27 19:09:59.926765 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:09:59.926569    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fb5e10b6-f988-49b4-8caa-c1a85d3b98aa" (UID: "fb5e10b6-f988-49b4-8caa-c1a85d3b98aa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:09:59.926765 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:09:59.926584    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-home" (OuterVolumeSpecName: "home") pod "fb5e10b6-f988-49b4-8caa-c1a85d3b98aa" (UID: "fb5e10b6-f988-49b4-8caa-c1a85d3b98aa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:09:59.926765 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:09:59.926679    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-model-cache" (OuterVolumeSpecName: "model-cache") pod "fb5e10b6-f988-49b4-8caa-c1a85d3b98aa" (UID: "fb5e10b6-f988-49b4-8caa-c1a85d3b98aa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:09:59.926958 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:09:59.926830    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-tmp-dir\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:09:59.926958 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:09:59.926848    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-home\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:09:59.926958 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:09:59.926862    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-model-cache\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:09:59.928816 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:09:59.928788    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-kube-api-access-f4xlx" (OuterVolumeSpecName: "kube-api-access-f4xlx") pod "fb5e10b6-f988-49b4-8caa-c1a85d3b98aa" (UID: "fb5e10b6-f988-49b4-8caa-c1a85d3b98aa"). InnerVolumeSpecName "kube-api-access-f4xlx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:09:59.928987 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:09:59.928858    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fb5e10b6-f988-49b4-8caa-c1a85d3b98aa" (UID: "fb5e10b6-f988-49b4-8caa-c1a85d3b98aa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:09:59.928987 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:09:59.928919    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-dshm" (OuterVolumeSpecName: "dshm") pod "fb5e10b6-f988-49b4-8caa-c1a85d3b98aa" (UID: "fb5e10b6-f988-49b4-8caa-c1a85d3b98aa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:10:00.028245 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:10:00.028204    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4xlx\" (UniqueName: \"kubernetes.io/projected/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-kube-api-access-f4xlx\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:10:00.028245 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:10:00.028237    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-dshm\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:10:00.028245 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:10:00.028251    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa-tls-certs\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:10:00.651650 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:10:00.651611    2562 generic.go:356] "Generic (PLEG): container finished" podID="fb5e10b6-f988-49b4-8caa-c1a85d3b98aa" containerID="85d492e26b48ef297279841581a610f76db817d4f8c355a4b7a3a9fec04db450" exitCode=0
Jul 27 19:10:00.652148 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:10:00.651659    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"
Jul 27 19:10:00.652148 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:10:00.651705    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph" event={"ID":"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa","Type":"ContainerDied","Data":"85d492e26b48ef297279841581a610f76db817d4f8c355a4b7a3a9fec04db450"}
Jul 27 19:10:00.652148 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:10:00.651749    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph" event={"ID":"fb5e10b6-f988-49b4-8caa-c1a85d3b98aa","Type":"ContainerDied","Data":"37642f777e3e7063131ed62a50d96ee20fd8474a7d3bb24779a6e9b1eded9272"}
Jul 27 19:10:00.652148 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:10:00.651769    2562 scope.go:117] "RemoveContainer" containerID="85d492e26b48ef297279841581a610f76db817d4f8c355a4b7a3a9fec04db450"
Jul 27 19:10:00.660850 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:10:00.660826    2562 scope.go:117] "RemoveContainer" containerID="85d492e26b48ef297279841581a610f76db817d4f8c355a4b7a3a9fec04db450"
Jul 27 19:10:00.661180 ip-10-0-132-186 kubenswrapper[2562]: E0727 19:10:00.661159    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85d492e26b48ef297279841581a610f76db817d4f8c355a4b7a3a9fec04db450\": container with ID starting with 85d492e26b48ef297279841581a610f76db817d4f8c355a4b7a3a9fec04db450 not found: ID does not exist" containerID="85d492e26b48ef297279841581a610f76db817d4f8c355a4b7a3a9fec04db450"
Jul 27 19:10:00.661258 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:10:00.661189    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85d492e26b48ef297279841581a610f76db817d4f8c355a4b7a3a9fec04db450"} err="failed to get container status \"85d492e26b48ef297279841581a610f76db817d4f8c355a4b7a3a9fec04db450\": rpc error: code = NotFound desc = could not find container \"85d492e26b48ef297279841581a610f76db817d4f8c355a4b7a3a9fec04db450\": container with ID starting with 85d492e26b48ef297279841581a610f76db817d4f8c355a4b7a3a9fec04db450 not found: ID does not exist"
Jul 27 19:10:00.669267 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:10:00.669241    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"]
Jul 27 19:10:00.674879 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:10:00.674841    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-dcc83a9c/e2e-lifecycle-model-dcc83a9c-kserve-6d4cc7bd87-7kfph"]
Jul 27 19:10:01.936064 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:10:01.936028    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb5e10b6-f988-49b4-8caa-c1a85d3b98aa" path="/var/lib/kubelet/pods/fb5e10b6-f988-49b4-8caa-c1a85d3b98aa/volumes"
Jul 27 19:11:31.949361 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:11:31.949318    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 19:11:31.951747 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:11:31.951721    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 19:11:32.062497 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:11:32.062463    2562 scope.go:117] "RemoveContainer" containerID="200c29200d340d33a64a5cbcbb4a54de433ee0f645bf7a4902dd95c1563c99c9"
Jul 27 19:12:02.384308 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.384223    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"]
Jul 27 19:12:02.384756 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.384568    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb5e10b6-f988-49b4-8caa-c1a85d3b98aa" containerName="main"
Jul 27 19:12:02.384756 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.384581    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb5e10b6-f988-49b4-8caa-c1a85d3b98aa" containerName="main"
Jul 27 19:12:02.384756 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.384641    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb5e10b6-f988-49b4-8caa-c1a85d3b98aa" containerName="main"
Jul 27 19:12:02.387337 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.387322    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:02.389894 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.389856    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-7aabc5\"/\"kube-root-ca.crt\""
Jul 27 19:12:02.390633 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.390614    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-7aabc5\"/\"openshift-service-ca.crt\""
Jul 27 19:12:02.390683 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.390663    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-7aabc5\"/\"test-model-7aabc5-kserve-self-signed-certs\""
Jul 27 19:12:02.390967 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.390953    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-7aabc5\"/\"default-dockercfg-4svht\""
Jul 27 19:12:02.400384 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.400358    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"]
Jul 27 19:12:02.493444 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.493395    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-tmp-dir\") pod \"test-model-7aabc5-kserve-586bb9cd75-jxlv6\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") " pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:02.493641 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.493478    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-home\") pod \"test-model-7aabc5-kserve-586bb9cd75-jxlv6\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") " pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:02.493641 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.493542    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad2032de-7c4c-4293-9e82-952ae765cd8f-tls-certs\") pod \"test-model-7aabc5-kserve-586bb9cd75-jxlv6\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") " pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:02.493641 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.493576    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-model-cache\") pod \"test-model-7aabc5-kserve-586bb9cd75-jxlv6\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") " pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:02.493771 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.493661    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8skfb\" (UniqueName: \"kubernetes.io/projected/ad2032de-7c4c-4293-9e82-952ae765cd8f-kube-api-access-8skfb\") pod \"test-model-7aabc5-kserve-586bb9cd75-jxlv6\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") " pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:02.493771 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.493699    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-dshm\") pod \"test-model-7aabc5-kserve-586bb9cd75-jxlv6\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") " pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:02.594193 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.594149    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-model-cache\") pod \"test-model-7aabc5-kserve-586bb9cd75-jxlv6\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") " pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:02.594368 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.594229    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8skfb\" (UniqueName: \"kubernetes.io/projected/ad2032de-7c4c-4293-9e82-952ae765cd8f-kube-api-access-8skfb\") pod \"test-model-7aabc5-kserve-586bb9cd75-jxlv6\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") " pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:02.594368 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.594258    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-dshm\") pod \"test-model-7aabc5-kserve-586bb9cd75-jxlv6\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") " pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:02.594368 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.594298    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-tmp-dir\") pod \"test-model-7aabc5-kserve-586bb9cd75-jxlv6\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") " pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:02.594368 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.594323    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-home\") pod \"test-model-7aabc5-kserve-586bb9cd75-jxlv6\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") " pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:02.594368 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.594351    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad2032de-7c4c-4293-9e82-952ae765cd8f-tls-certs\") pod \"test-model-7aabc5-kserve-586bb9cd75-jxlv6\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") " pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:02.594630 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.594605    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-model-cache\") pod \"test-model-7aabc5-kserve-586bb9cd75-jxlv6\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") " pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:02.594753 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.594728    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-home\") pod \"test-model-7aabc5-kserve-586bb9cd75-jxlv6\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") " pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:02.594753 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.594730    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-tmp-dir\") pod \"test-model-7aabc5-kserve-586bb9cd75-jxlv6\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") " pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:02.596741 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.596723    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-dshm\") pod \"test-model-7aabc5-kserve-586bb9cd75-jxlv6\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") " pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:02.597124 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.597104    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad2032de-7c4c-4293-9e82-952ae765cd8f-tls-certs\") pod \"test-model-7aabc5-kserve-586bb9cd75-jxlv6\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") " pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:02.602106 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.602086    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8skfb\" (UniqueName: \"kubernetes.io/projected/ad2032de-7c4c-4293-9e82-952ae765cd8f-kube-api-access-8skfb\") pod \"test-model-7aabc5-kserve-586bb9cd75-jxlv6\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") " pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:02.696210 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.696117    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:02.833059 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:02.832999    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"]
Jul 27 19:12:02.835602 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:12:02.835575    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad2032de_7c4c_4293_9e82_952ae765cd8f.slice/crio-1bca6325ff8593df493a643c54b4e71728dcfeabae0156615d51707328229769 WatchSource:0}: Error finding container 1bca6325ff8593df493a643c54b4e71728dcfeabae0156615d51707328229769: Status 404 returned error can't find the container with id 1bca6325ff8593df493a643c54b4e71728dcfeabae0156615d51707328229769
Jul 27 19:12:03.180251 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:03.180211    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6" event={"ID":"ad2032de-7c4c-4293-9e82-952ae765cd8f","Type":"ContainerStarted","Data":"661f0dc69f2eddcd3d0209c9896197787e967f188f53563f53b6f328fecc1bd8"}
Jul 27 19:12:03.180439 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:03.180265    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6" event={"ID":"ad2032de-7c4c-4293-9e82-952ae765cd8f","Type":"ContainerStarted","Data":"1bca6325ff8593df493a643c54b4e71728dcfeabae0156615d51707328229769"}
Jul 27 19:12:03.197959 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:03.197909    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6" podStartSLOduration=1.197860491 podStartE2EDuration="1.197860491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:12:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:12:03.19713448 +0000 UTC m=+2431.747428114" watchObservedRunningTime="2026-07-27 19:12:03.197860491 +0000 UTC m=+2431.748154118"
Jul 27 19:12:12.697071 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:12.697026    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:12.697567 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:12.697122    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:12.709578 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:12.709552    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:13.235453 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:13.235421    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:12:29.392050 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.391962    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"]
Jul 27 19:12:29.395757 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.395733    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:29.398856 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.398823    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-954977\"/\"kube-root-ca.crt\""
Jul 27 19:12:29.398856 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.398846    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-954977\"/\"default-dockercfg-pd9v5\""
Jul 27 19:12:29.399059 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.398824    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-954977\"/\"test-model-954977-kserve-self-signed-certs\""
Jul 27 19:12:29.399059 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.398824    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-954977\"/\"openshift-service-ca.crt\""
Jul 27 19:12:29.405282 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.405257    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"]
Jul 27 19:12:29.536646 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.536609    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-tmp-dir\") pod \"test-model-954977-kserve-7dd76d9b9f-mjjtw\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") " pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:29.536883 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.536654    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-home\") pod \"test-model-954977-kserve-7dd76d9b9f-mjjtw\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") " pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:29.536883 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.536680    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-dshm\") pod \"test-model-954977-kserve-7dd76d9b9f-mjjtw\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") " pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:29.536883 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.536708    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c91dce2-6641-4885-84f8-5cefb61c7718-tls-certs\") pod \"test-model-954977-kserve-7dd76d9b9f-mjjtw\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") " pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:29.536883 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.536788    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k7gj\" (UniqueName: \"kubernetes.io/projected/2c91dce2-6641-4885-84f8-5cefb61c7718-kube-api-access-9k7gj\") pod \"test-model-954977-kserve-7dd76d9b9f-mjjtw\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") " pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:29.536883 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.536843    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-model-cache\") pod \"test-model-954977-kserve-7dd76d9b9f-mjjtw\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") " pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:29.638416 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.638379    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-tmp-dir\") pod \"test-model-954977-kserve-7dd76d9b9f-mjjtw\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") " pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:29.638416 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.638418    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-home\") pod \"test-model-954977-kserve-7dd76d9b9f-mjjtw\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") " pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:29.638662 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.638453    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-dshm\") pod \"test-model-954977-kserve-7dd76d9b9f-mjjtw\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") " pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:29.638662 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.638487    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c91dce2-6641-4885-84f8-5cefb61c7718-tls-certs\") pod \"test-model-954977-kserve-7dd76d9b9f-mjjtw\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") " pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:29.638662 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.638523    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k7gj\" (UniqueName: \"kubernetes.io/projected/2c91dce2-6641-4885-84f8-5cefb61c7718-kube-api-access-9k7gj\") pod \"test-model-954977-kserve-7dd76d9b9f-mjjtw\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") " pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:29.638662 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.638569    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-model-cache\") pod \"test-model-954977-kserve-7dd76d9b9f-mjjtw\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") " pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:29.638915 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.638855    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-home\") pod \"test-model-954977-kserve-7dd76d9b9f-mjjtw\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") " pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:29.639039 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.638954    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-tmp-dir\") pod \"test-model-954977-kserve-7dd76d9b9f-mjjtw\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") " pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:29.639039 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.638969    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-model-cache\") pod \"test-model-954977-kserve-7dd76d9b9f-mjjtw\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") " pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:29.640923 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.640901    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-dshm\") pod \"test-model-954977-kserve-7dd76d9b9f-mjjtw\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") " pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:29.641267 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.641250    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c91dce2-6641-4885-84f8-5cefb61c7718-tls-certs\") pod \"test-model-954977-kserve-7dd76d9b9f-mjjtw\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") " pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:29.647115 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.647055    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k7gj\" (UniqueName: \"kubernetes.io/projected/2c91dce2-6641-4885-84f8-5cefb61c7718-kube-api-access-9k7gj\") pod \"test-model-954977-kserve-7dd76d9b9f-mjjtw\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") " pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:29.706289 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.706246    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:29.845610 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:29.845586    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"]
Jul 27 19:12:29.848321 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:12:29.848292    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c91dce2_6641_4885_84f8_5cefb61c7718.slice/crio-02a95fc2e3c98b1b74f5cbd2b3be6bd1e045023a893c4feafac01ccc497a6e75 WatchSource:0}: Error finding container 02a95fc2e3c98b1b74f5cbd2b3be6bd1e045023a893c4feafac01ccc497a6e75: Status 404 returned error can't find the container with id 02a95fc2e3c98b1b74f5cbd2b3be6bd1e045023a893c4feafac01ccc497a6e75
Jul 27 19:12:30.295022 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:30.294979    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw" event={"ID":"2c91dce2-6641-4885-84f8-5cefb61c7718","Type":"ContainerStarted","Data":"322983db5d7d9a0d7e33f9ed5e26016436689697cdd8a02acbfafc07aca19421"}
Jul 27 19:12:30.295022 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:30.295022    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw" event={"ID":"2c91dce2-6641-4885-84f8-5cefb61c7718","Type":"ContainerStarted","Data":"02a95fc2e3c98b1b74f5cbd2b3be6bd1e045023a893c4feafac01ccc497a6e75"}
Jul 27 19:12:30.312951 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:30.312898    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw" podStartSLOduration=1.312863436 podStartE2EDuration="1.312863436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:12:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:12:30.312284443 +0000 UTC m=+2458.862578077" watchObservedRunningTime="2026-07-27 19:12:30.312863436 +0000 UTC m=+2458.863157108"
Jul 27 19:12:39.707267 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:39.707215    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:39.707802 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:39.707283    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:39.719792 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:39.719758    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:12:40.348695 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:12:40.348656    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:13:31.720767 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:31.720731    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"]
Jul 27 19:13:32.738114 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:32.738079    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"]
Jul 27 19:13:46.740449 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:46.740400    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6" podUID="ad2032de-7c4c-4293-9e82-952ae765cd8f" containerName="main" containerID="cri-o://661f0dc69f2eddcd3d0209c9896197787e967f188f53563f53b6f328fecc1bd8" gracePeriod=45
Jul 27 19:13:46.993703 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:46.993633    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:13:47.031589 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.031561    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad2032de-7c4c-4293-9e82-952ae765cd8f-tls-certs\") pod \"ad2032de-7c4c-4293-9e82-952ae765cd8f\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") "
Jul 27 19:13:47.031787 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.031614    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-dshm\") pod \"ad2032de-7c4c-4293-9e82-952ae765cd8f\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") "
Jul 27 19:13:47.031787 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.031664    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-model-cache\") pod \"ad2032de-7c4c-4293-9e82-952ae765cd8f\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") "
Jul 27 19:13:47.031787 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.031683    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8skfb\" (UniqueName: \"kubernetes.io/projected/ad2032de-7c4c-4293-9e82-952ae765cd8f-kube-api-access-8skfb\") pod \"ad2032de-7c4c-4293-9e82-952ae765cd8f\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") "
Jul 27 19:13:47.031787 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.031735    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-tmp-dir\") pod \"ad2032de-7c4c-4293-9e82-952ae765cd8f\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") "
Jul 27 19:13:47.031787 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.031770    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-home\") pod \"ad2032de-7c4c-4293-9e82-952ae765cd8f\" (UID: \"ad2032de-7c4c-4293-9e82-952ae765cd8f\") "
Jul 27 19:13:47.032085 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.031963    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-model-cache" (OuterVolumeSpecName: "model-cache") pod "ad2032de-7c4c-4293-9e82-952ae765cd8f" (UID: "ad2032de-7c4c-4293-9e82-952ae765cd8f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:13:47.032131 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.032109    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-model-cache\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:13:47.032199 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.032117    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ad2032de-7c4c-4293-9e82-952ae765cd8f" (UID: "ad2032de-7c4c-4293-9e82-952ae765cd8f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:13:47.032199 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.032134    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-home" (OuterVolumeSpecName: "home") pod "ad2032de-7c4c-4293-9e82-952ae765cd8f" (UID: "ad2032de-7c4c-4293-9e82-952ae765cd8f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:13:47.034082 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.034059    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-dshm" (OuterVolumeSpecName: "dshm") pod "ad2032de-7c4c-4293-9e82-952ae765cd8f" (UID: "ad2032de-7c4c-4293-9e82-952ae765cd8f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:13:47.034179 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.034059    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad2032de-7c4c-4293-9e82-952ae765cd8f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ad2032de-7c4c-4293-9e82-952ae765cd8f" (UID: "ad2032de-7c4c-4293-9e82-952ae765cd8f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:13:47.034179 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.034145    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad2032de-7c4c-4293-9e82-952ae765cd8f-kube-api-access-8skfb" (OuterVolumeSpecName: "kube-api-access-8skfb") pod "ad2032de-7c4c-4293-9e82-952ae765cd8f" (UID: "ad2032de-7c4c-4293-9e82-952ae765cd8f"). InnerVolumeSpecName "kube-api-access-8skfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:13:47.132945 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.132894    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-tmp-dir\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:13:47.132945 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.132938    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-home\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:13:47.132945 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.132948    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad2032de-7c4c-4293-9e82-952ae765cd8f-tls-certs\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:13:47.132945 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.132957    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad2032de-7c4c-4293-9e82-952ae765cd8f-dshm\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:13:47.132945 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.132966    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8skfb\" (UniqueName: \"kubernetes.io/projected/ad2032de-7c4c-4293-9e82-952ae765cd8f-kube-api-access-8skfb\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:13:47.635275 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.635236    2562 generic.go:356] "Generic (PLEG): container finished" podID="ad2032de-7c4c-4293-9e82-952ae765cd8f" containerID="661f0dc69f2eddcd3d0209c9896197787e967f188f53563f53b6f328fecc1bd8" exitCode=0
Jul 27 19:13:47.635484 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.635297    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"
Jul 27 19:13:47.635484 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.635336    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6" event={"ID":"ad2032de-7c4c-4293-9e82-952ae765cd8f","Type":"ContainerDied","Data":"661f0dc69f2eddcd3d0209c9896197787e967f188f53563f53b6f328fecc1bd8"}
Jul 27 19:13:47.635484 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.635390    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6" event={"ID":"ad2032de-7c4c-4293-9e82-952ae765cd8f","Type":"ContainerDied","Data":"1bca6325ff8593df493a643c54b4e71728dcfeabae0156615d51707328229769"}
Jul 27 19:13:47.635484 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.635414    2562 scope.go:117] "RemoveContainer" containerID="661f0dc69f2eddcd3d0209c9896197787e967f188f53563f53b6f328fecc1bd8"
Jul 27 19:13:47.644226 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.644207    2562 scope.go:117] "RemoveContainer" containerID="661f0dc69f2eddcd3d0209c9896197787e967f188f53563f53b6f328fecc1bd8"
Jul 27 19:13:47.644508 ip-10-0-132-186 kubenswrapper[2562]: E0727 19:13:47.644487    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"661f0dc69f2eddcd3d0209c9896197787e967f188f53563f53b6f328fecc1bd8\": container with ID starting with 661f0dc69f2eddcd3d0209c9896197787e967f188f53563f53b6f328fecc1bd8 not found: ID does not exist" containerID="661f0dc69f2eddcd3d0209c9896197787e967f188f53563f53b6f328fecc1bd8"
Jul 27 19:13:47.644562 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.644516    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"661f0dc69f2eddcd3d0209c9896197787e967f188f53563f53b6f328fecc1bd8"} err="failed to get container status \"661f0dc69f2eddcd3d0209c9896197787e967f188f53563f53b6f328fecc1bd8\": rpc error: code = NotFound desc = could not find container \"661f0dc69f2eddcd3d0209c9896197787e967f188f53563f53b6f328fecc1bd8\": container with ID starting with 661f0dc69f2eddcd3d0209c9896197787e967f188f53563f53b6f328fecc1bd8 not found: ID does not exist"
Jul 27 19:13:47.655255 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.655230    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"]
Jul 27 19:13:47.658763 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.658736    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-7aabc5/test-model-7aabc5-kserve-586bb9cd75-jxlv6"]
Jul 27 19:13:47.757606 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.757546    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw" podUID="2c91dce2-6641-4885-84f8-5cefb61c7718" containerName="main" containerID="cri-o://322983db5d7d9a0d7e33f9ed5e26016436689697cdd8a02acbfafc07aca19421" gracePeriod=45
Jul 27 19:13:47.937756 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:47.937721    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad2032de-7c4c-4293-9e82-952ae765cd8f" path="/var/lib/kubelet/pods/ad2032de-7c4c-4293-9e82-952ae765cd8f/volumes"
Jul 27 19:13:48.015584 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.015556    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:13:48.143437 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.143398    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c91dce2-6641-4885-84f8-5cefb61c7718-tls-certs\") pod \"2c91dce2-6641-4885-84f8-5cefb61c7718\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") "
Jul 27 19:13:48.143637 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.143448    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9k7gj\" (UniqueName: \"kubernetes.io/projected/2c91dce2-6641-4885-84f8-5cefb61c7718-kube-api-access-9k7gj\") pod \"2c91dce2-6641-4885-84f8-5cefb61c7718\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") "
Jul 27 19:13:48.143637 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.143496    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-tmp-dir\") pod \"2c91dce2-6641-4885-84f8-5cefb61c7718\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") "
Jul 27 19:13:48.143637 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.143526    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-model-cache\") pod \"2c91dce2-6641-4885-84f8-5cefb61c7718\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") "
Jul 27 19:13:48.143637 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.143598    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-home\") pod \"2c91dce2-6641-4885-84f8-5cefb61c7718\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") "
Jul 27 19:13:48.143637 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.143631    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-dshm\") pod \"2c91dce2-6641-4885-84f8-5cefb61c7718\" (UID: \"2c91dce2-6641-4885-84f8-5cefb61c7718\") "
Jul 27 19:13:48.143939 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.143740    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2c91dce2-6641-4885-84f8-5cefb61c7718" (UID: "2c91dce2-6641-4885-84f8-5cefb61c7718"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:13:48.143939 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.143896    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-model-cache" (OuterVolumeSpecName: "model-cache") pod "2c91dce2-6641-4885-84f8-5cefb61c7718" (UID: "2c91dce2-6641-4885-84f8-5cefb61c7718"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:13:48.143939 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.143904    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-home" (OuterVolumeSpecName: "home") pod "2c91dce2-6641-4885-84f8-5cefb61c7718" (UID: "2c91dce2-6641-4885-84f8-5cefb61c7718"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:13:48.144042 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.143984    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-tmp-dir\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:13:48.144042 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.144000    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-model-cache\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:13:48.144042 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.144008    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-home\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:13:48.146050 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.146026    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-dshm" (OuterVolumeSpecName: "dshm") pod "2c91dce2-6641-4885-84f8-5cefb61c7718" (UID: "2c91dce2-6641-4885-84f8-5cefb61c7718"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:13:48.146050 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.146032    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c91dce2-6641-4885-84f8-5cefb61c7718-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2c91dce2-6641-4885-84f8-5cefb61c7718" (UID: "2c91dce2-6641-4885-84f8-5cefb61c7718"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:13:48.146178 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.146137    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c91dce2-6641-4885-84f8-5cefb61c7718-kube-api-access-9k7gj" (OuterVolumeSpecName: "kube-api-access-9k7gj") pod "2c91dce2-6641-4885-84f8-5cefb61c7718" (UID: "2c91dce2-6641-4885-84f8-5cefb61c7718"). InnerVolumeSpecName "kube-api-access-9k7gj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:13:48.245127 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.245099    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c91dce2-6641-4885-84f8-5cefb61c7718-dshm\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:13:48.245127 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.245127    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c91dce2-6641-4885-84f8-5cefb61c7718-tls-certs\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:13:48.245322 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.245141    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9k7gj\" (UniqueName: \"kubernetes.io/projected/2c91dce2-6641-4885-84f8-5cefb61c7718-kube-api-access-9k7gj\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:13:48.640671 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.640576    2562 generic.go:356] "Generic (PLEG): container finished" podID="2c91dce2-6641-4885-84f8-5cefb61c7718" containerID="322983db5d7d9a0d7e33f9ed5e26016436689697cdd8a02acbfafc07aca19421" exitCode=0
Jul 27 19:13:48.640671 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.640628    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"
Jul 27 19:13:48.640671 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.640657    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw" event={"ID":"2c91dce2-6641-4885-84f8-5cefb61c7718","Type":"ContainerDied","Data":"322983db5d7d9a0d7e33f9ed5e26016436689697cdd8a02acbfafc07aca19421"}
Jul 27 19:13:48.640980 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.640695    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw" event={"ID":"2c91dce2-6641-4885-84f8-5cefb61c7718","Type":"ContainerDied","Data":"02a95fc2e3c98b1b74f5cbd2b3be6bd1e045023a893c4feafac01ccc497a6e75"}
Jul 27 19:13:48.640980 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.640714    2562 scope.go:117] "RemoveContainer" containerID="322983db5d7d9a0d7e33f9ed5e26016436689697cdd8a02acbfafc07aca19421"
Jul 27 19:13:48.651607 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.651583    2562 scope.go:117] "RemoveContainer" containerID="322983db5d7d9a0d7e33f9ed5e26016436689697cdd8a02acbfafc07aca19421"
Jul 27 19:13:48.651984 ip-10-0-132-186 kubenswrapper[2562]: E0727 19:13:48.651963    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"322983db5d7d9a0d7e33f9ed5e26016436689697cdd8a02acbfafc07aca19421\": container with ID starting with 322983db5d7d9a0d7e33f9ed5e26016436689697cdd8a02acbfafc07aca19421 not found: ID does not exist" containerID="322983db5d7d9a0d7e33f9ed5e26016436689697cdd8a02acbfafc07aca19421"
Jul 27 19:13:48.652061 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.651992    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"322983db5d7d9a0d7e33f9ed5e26016436689697cdd8a02acbfafc07aca19421"} err="failed to get container status \"322983db5d7d9a0d7e33f9ed5e26016436689697cdd8a02acbfafc07aca19421\": rpc error: code = NotFound desc = could not find container \"322983db5d7d9a0d7e33f9ed5e26016436689697cdd8a02acbfafc07aca19421\": container with ID starting with 322983db5d7d9a0d7e33f9ed5e26016436689697cdd8a02acbfafc07aca19421 not found: ID does not exist"
Jul 27 19:13:48.668783 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.668755    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"]
Jul 27 19:13:48.670611 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:48.670589    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-954977/test-model-954977-kserve-7dd76d9b9f-mjjtw"]
Jul 27 19:13:49.936853 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:49.936821    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c91dce2-6641-4885-84f8-5cefb61c7718" path="/var/lib/kubelet/pods/2c91dce2-6641-4885-84f8-5cefb61c7718/volumes"
Jul 27 19:13:56.133659 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:56.133619    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-954977-76tqs"]
Jul 27 19:13:56.134112 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:56.133957    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad2032de-7c4c-4293-9e82-952ae765cd8f" containerName="main"
Jul 27 19:13:56.134112 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:56.133969    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad2032de-7c4c-4293-9e82-952ae765cd8f" containerName="main"
Jul 27 19:13:56.134112 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:56.133986    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c91dce2-6641-4885-84f8-5cefb61c7718" containerName="main"
Jul 27 19:13:56.134112 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:56.133991    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c91dce2-6641-4885-84f8-5cefb61c7718" containerName="main"
Jul 27 19:13:56.134112 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:56.134042    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad2032de-7c4c-4293-9e82-952ae765cd8f" containerName="main"
Jul 27 19:13:56.134112 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:56.134049    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c91dce2-6641-4885-84f8-5cefb61c7718" containerName="main"
Jul 27 19:13:56.138384 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:56.138364    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-954977-76tqs"
Jul 27 19:13:56.141295 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:56.141268    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gwtlm\""
Jul 27 19:13:56.141542 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:56.141525    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 19:13:56.151744 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:56.151711    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-954977-76tqs"]
Jul 27 19:13:56.211959 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:56.211917    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9314efbf-dd12-4997-9b02-3ab1405279a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-954977-76tqs\" (UID: \"9314efbf-dd12-4997-9b02-3ab1405279a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-954977-76tqs"
Jul 27 19:13:56.312795 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:56.312755    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9314efbf-dd12-4997-9b02-3ab1405279a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-954977-76tqs\" (UID: \"9314efbf-dd12-4997-9b02-3ab1405279a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-954977-76tqs"
Jul 27 19:13:56.313502 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:56.313475    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9314efbf-dd12-4997-9b02-3ab1405279a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-954977-76tqs\" (UID: \"9314efbf-dd12-4997-9b02-3ab1405279a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-954977-76tqs"
Jul 27 19:13:56.448720 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:56.448619    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-954977-76tqs"
Jul 27 19:13:56.585671 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:56.585647    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-954977-76tqs"]
Jul 27 19:13:56.588107 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:13:56.588064    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9314efbf_dd12_4997_9b02_3ab1405279a5.slice/crio-76296c6fdcb212d5374062908df2f83f346c249a08bfba1fbcb70146d13409f2 WatchSource:0}: Error finding container 76296c6fdcb212d5374062908df2f83f346c249a08bfba1fbcb70146d13409f2: Status 404 returned error can't find the container with id 76296c6fdcb212d5374062908df2f83f346c249a08bfba1fbcb70146d13409f2
Jul 27 19:13:56.681930 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:56.681862    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-954977-76tqs" event={"ID":"9314efbf-dd12-4997-9b02-3ab1405279a5","Type":"ContainerStarted","Data":"76296c6fdcb212d5374062908df2f83f346c249a08bfba1fbcb70146d13409f2"}
Jul 27 19:13:57.687833 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:57.687798    2562 generic.go:356] "Generic (PLEG): container finished" podID="9314efbf-dd12-4997-9b02-3ab1405279a5" containerID="6cb4235908626225ca89151fc1dd479751c482a9bbdcf7770659949e420938bf" exitCode=0
Jul 27 19:13:57.688286 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:57.687904    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-954977-76tqs" event={"ID":"9314efbf-dd12-4997-9b02-3ab1405279a5","Type":"ContainerDied","Data":"6cb4235908626225ca89151fc1dd479751c482a9bbdcf7770659949e420938bf"}
Jul 27 19:13:58.835714 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:58.835687    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-954977-76tqs"
Jul 27 19:13:58.933764 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:58.933729    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9314efbf-dd12-4997-9b02-3ab1405279a5-maas-api-service-ca\") pod \"9314efbf-dd12-4997-9b02-3ab1405279a5\" (UID: \"9314efbf-dd12-4997-9b02-3ab1405279a5\") "
Jul 27 19:13:58.934141 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:58.934115    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9314efbf-dd12-4997-9b02-3ab1405279a5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9314efbf-dd12-4997-9b02-3ab1405279a5" (UID: "9314efbf-dd12-4997-9b02-3ab1405279a5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:13:59.034394 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:59.034356    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9314efbf-dd12-4997-9b02-3ab1405279a5-maas-api-service-ca\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:13:59.697257 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:59.697224    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-954977-76tqs"
Jul 27 19:13:59.697257 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:59.697237    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-954977-76tqs" event={"ID":"9314efbf-dd12-4997-9b02-3ab1405279a5","Type":"ContainerDied","Data":"76296c6fdcb212d5374062908df2f83f346c249a08bfba1fbcb70146d13409f2"}
Jul 27 19:13:59.697503 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:13:59.697289    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="76296c6fdcb212d5374062908df2f83f346c249a08bfba1fbcb70146d13409f2"
Jul 27 19:14:11.229039 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:11.229006    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-954977-76tqs"]
Jul 27 19:14:11.232394 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:11.232367    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-954977-76tqs"]
Jul 27 19:14:11.936012 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:11.935980    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9314efbf-dd12-4997-9b02-3ab1405279a5" path="/var/lib/kubelet/pods/9314efbf-dd12-4997-9b02-3ab1405279a5/volumes"
Jul 27 19:14:44.628687 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:44.628647    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cad7f7-wcw67"]
Jul 27 19:14:44.629160 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:44.628983    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9314efbf-dd12-4997-9b02-3ab1405279a5" containerName="revoke-keys"
Jul 27 19:14:44.629160 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:44.628994    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9314efbf-dd12-4997-9b02-3ab1405279a5" containerName="revoke-keys"
Jul 27 19:14:44.629160 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:44.629061    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9314efbf-dd12-4997-9b02-3ab1405279a5" containerName="revoke-keys"
Jul 27 19:14:44.631800 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:44.631782    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cad7f7-wcw67"
Jul 27 19:14:44.636545 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:44.636515    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gwtlm\""
Jul 27 19:14:44.636707 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:44.636524    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 19:14:44.652977 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:44.652926    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cad7f7-wcw67"]
Jul 27 19:14:44.753726 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:44.753684    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/890f5b27-8bb2-402f-879b-e3f81d9b3abb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-cad7f7-wcw67\" (UID: \"890f5b27-8bb2-402f-879b-e3f81d9b3abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cad7f7-wcw67"
Jul 27 19:14:44.855217 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:44.855171    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/890f5b27-8bb2-402f-879b-e3f81d9b3abb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-cad7f7-wcw67\" (UID: \"890f5b27-8bb2-402f-879b-e3f81d9b3abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cad7f7-wcw67"
Jul 27 19:14:44.855801 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:44.855777    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/890f5b27-8bb2-402f-879b-e3f81d9b3abb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-cad7f7-wcw67\" (UID: \"890f5b27-8bb2-402f-879b-e3f81d9b3abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cad7f7-wcw67"
Jul 27 19:14:44.942051 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:44.941955    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cad7f7-wcw67"
Jul 27 19:14:45.083810 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:45.083782    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cad7f7-wcw67"]
Jul 27 19:14:45.086377 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:14:45.086346    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod890f5b27_8bb2_402f_879b_e3f81d9b3abb.slice/crio-aaf70cca6df2f831825b0f4dd635e3268c0edc25efc9e281b8a2bf40ffa09a01 WatchSource:0}: Error finding container aaf70cca6df2f831825b0f4dd635e3268c0edc25efc9e281b8a2bf40ffa09a01: Status 404 returned error can't find the container with id aaf70cca6df2f831825b0f4dd635e3268c0edc25efc9e281b8a2bf40ffa09a01
Jul 27 19:14:45.906423 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:45.906386    2562 generic.go:356] "Generic (PLEG): container finished" podID="890f5b27-8bb2-402f-879b-e3f81d9b3abb" containerID="0dc9f86be958d40b1b7157a68251b462fac9116a47bba34ef1338a46095b0159" exitCode=0
Jul 27 19:14:45.906808 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:45.906447    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cad7f7-wcw67" event={"ID":"890f5b27-8bb2-402f-879b-e3f81d9b3abb","Type":"ContainerDied","Data":"0dc9f86be958d40b1b7157a68251b462fac9116a47bba34ef1338a46095b0159"}
Jul 27 19:14:45.906808 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:45.906498    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cad7f7-wcw67" event={"ID":"890f5b27-8bb2-402f-879b-e3f81d9b3abb","Type":"ContainerStarted","Data":"aaf70cca6df2f831825b0f4dd635e3268c0edc25efc9e281b8a2bf40ffa09a01"}
Jul 27 19:14:47.047212 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:47.047182    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cad7f7-wcw67"
Jul 27 19:14:47.179892 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:47.179768    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/890f5b27-8bb2-402f-879b-e3f81d9b3abb-maas-api-service-ca\") pod \"890f5b27-8bb2-402f-879b-e3f81d9b3abb\" (UID: \"890f5b27-8bb2-402f-879b-e3f81d9b3abb\") "
Jul 27 19:14:47.180184 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:47.180158    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/890f5b27-8bb2-402f-879b-e3f81d9b3abb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "890f5b27-8bb2-402f-879b-e3f81d9b3abb" (UID: "890f5b27-8bb2-402f-879b-e3f81d9b3abb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:14:47.281400 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:47.281351    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/890f5b27-8bb2-402f-879b-e3f81d9b3abb-maas-api-service-ca\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:14:47.915611 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:47.915576    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cad7f7-wcw67"
Jul 27 19:14:47.915764 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:47.915608    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cad7f7-wcw67" event={"ID":"890f5b27-8bb2-402f-879b-e3f81d9b3abb","Type":"ContainerDied","Data":"aaf70cca6df2f831825b0f4dd635e3268c0edc25efc9e281b8a2bf40ffa09a01"}
Jul 27 19:14:47.915764 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:14:47.915689    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aaf70cca6df2f831825b0f4dd635e3268c0edc25efc9e281b8a2bf40ffa09a01"
Jul 27 19:15:04.705602 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:04.705515    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cad7f7-wcw67"]
Jul 27 19:15:04.709195 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:04.709152    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cad7f7-wcw67"]
Jul 27 19:15:05.936779 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:05.936744    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="890f5b27-8bb2-402f-879b-e3f81d9b3abb" path="/var/lib/kubelet/pods/890f5b27-8bb2-402f-879b-e3f81d9b3abb/volumes"
Jul 27 19:15:11.861990 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:11.861952    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5b0fac-9m8rq"]
Jul 27 19:15:11.862391 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:11.862294    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="890f5b27-8bb2-402f-879b-e3f81d9b3abb" containerName="revoke-keys"
Jul 27 19:15:11.862391 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:11.862305    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="890f5b27-8bb2-402f-879b-e3f81d9b3abb" containerName="revoke-keys"
Jul 27 19:15:11.862391 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:11.862357    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="890f5b27-8bb2-402f-879b-e3f81d9b3abb" containerName="revoke-keys"
Jul 27 19:15:11.865227 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:11.865207    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5b0fac-9m8rq"
Jul 27 19:15:11.867900 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:11.867851    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 19:15:11.868049 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:11.867852    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gwtlm\""
Jul 27 19:15:11.874631 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:11.874583    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5b0fac-9m8rq"]
Jul 27 19:15:11.878986 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:11.878951    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbec21cc-2548-4af0-8093-ba493838926c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-5b0fac-9m8rq\" (UID: \"fbec21cc-2548-4af0-8093-ba493838926c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5b0fac-9m8rq"
Jul 27 19:15:11.979982 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:11.979942    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbec21cc-2548-4af0-8093-ba493838926c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-5b0fac-9m8rq\" (UID: \"fbec21cc-2548-4af0-8093-ba493838926c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5b0fac-9m8rq"
Jul 27 19:15:11.980564 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:11.980545    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbec21cc-2548-4af0-8093-ba493838926c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-5b0fac-9m8rq\" (UID: \"fbec21cc-2548-4af0-8093-ba493838926c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5b0fac-9m8rq"
Jul 27 19:15:12.177036 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:12.176944    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5b0fac-9m8rq"
Jul 27 19:15:12.323889 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:12.323841    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5b0fac-9m8rq"]
Jul 27 19:15:12.327102 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:15:12.327071    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbec21cc_2548_4af0_8093_ba493838926c.slice/crio-e89bd1b98317a342899cd59eb2f1ab6e6386b9ee6df9eddd3b15c15fcc64518a WatchSource:0}: Error finding container e89bd1b98317a342899cd59eb2f1ab6e6386b9ee6df9eddd3b15c15fcc64518a: Status 404 returned error can't find the container with id e89bd1b98317a342899cd59eb2f1ab6e6386b9ee6df9eddd3b15c15fcc64518a
Jul 27 19:15:13.026008 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:13.025974    2562 generic.go:356] "Generic (PLEG): container finished" podID="fbec21cc-2548-4af0-8093-ba493838926c" containerID="6dc3d02ec0def198ba5bdb6d8253da78d871c2679aab5c5cc9454c0656fb1854" exitCode=0
Jul 27 19:15:13.026391 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:13.026046    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5b0fac-9m8rq" event={"ID":"fbec21cc-2548-4af0-8093-ba493838926c","Type":"ContainerDied","Data":"6dc3d02ec0def198ba5bdb6d8253da78d871c2679aab5c5cc9454c0656fb1854"}
Jul 27 19:15:13.026391 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:13.026085    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5b0fac-9m8rq" event={"ID":"fbec21cc-2548-4af0-8093-ba493838926c","Type":"ContainerStarted","Data":"e89bd1b98317a342899cd59eb2f1ab6e6386b9ee6df9eddd3b15c15fcc64518a"}
Jul 27 19:15:14.164924 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:14.164894    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5b0fac-9m8rq"
Jul 27 19:15:14.194911 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:14.194863    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbec21cc-2548-4af0-8093-ba493838926c-maas-api-service-ca\") pod \"fbec21cc-2548-4af0-8093-ba493838926c\" (UID: \"fbec21cc-2548-4af0-8093-ba493838926c\") "
Jul 27 19:15:14.195312 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:14.195275    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbec21cc-2548-4af0-8093-ba493838926c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fbec21cc-2548-4af0-8093-ba493838926c" (UID: "fbec21cc-2548-4af0-8093-ba493838926c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:15:14.295751 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:14.295652    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbec21cc-2548-4af0-8093-ba493838926c-maas-api-service-ca\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:15:15.036754 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:15.036726    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5b0fac-9m8rq"
Jul 27 19:15:15.036754 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:15.036740    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5b0fac-9m8rq" event={"ID":"fbec21cc-2548-4af0-8093-ba493838926c","Type":"ContainerDied","Data":"e89bd1b98317a342899cd59eb2f1ab6e6386b9ee6df9eddd3b15c15fcc64518a"}
Jul 27 19:15:15.036996 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:15.036784    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e89bd1b98317a342899cd59eb2f1ab6e6386b9ee6df9eddd3b15c15fcc64518a"
Jul 27 19:15:31.957150 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:31.957100    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5b0fac-9m8rq"]
Jul 27 19:15:31.960729 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:31.960692    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5b0fac-9m8rq"]
Jul 27 19:15:33.936829 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:15:33.936791    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbec21cc-2548-4af0-8093-ba493838926c" path="/var/lib/kubelet/pods/fbec21cc-2548-4af0-8093-ba493838926c/volumes"
Jul 27 19:16:11.181602 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.181563    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"]
Jul 27 19:16:11.182250 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.182069    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbec21cc-2548-4af0-8093-ba493838926c" containerName="revoke-keys"
Jul 27 19:16:11.182250 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.182090    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbec21cc-2548-4af0-8093-ba493838926c" containerName="revoke-keys"
Jul 27 19:16:11.182250 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.182181    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbec21cc-2548-4af0-8093-ba493838926c" containerName="revoke-keys"
Jul 27 19:16:11.185248 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.185223    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:11.188575 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.188549    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-de344b\"/\"auth-test-model-de344b-kserve-self-signed-certs\""
Jul 27 19:16:11.188699 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.188601    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-de344b\"/\"default-dockercfg-kcnmj\""
Jul 27 19:16:11.188699 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.188553    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-de344b\"/\"kube-root-ca.crt\""
Jul 27 19:16:11.188699 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.188564    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-de344b\"/\"openshift-service-ca.crt\""
Jul 27 19:16:11.194852 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.194822    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"]
Jul 27 19:16:11.199347 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.199324    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-home\") pod \"auth-test-model-de344b-kserve-fd488b5bc-cgsnp\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") " pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:11.199460 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.199354    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-tmp-dir\") pod \"auth-test-model-de344b-kserve-fd488b5bc-cgsnp\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") " pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:11.199460 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.199428    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-dshm\") pod \"auth-test-model-de344b-kserve-fd488b5bc-cgsnp\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") " pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:11.199599 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.199468    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-model-cache\") pod \"auth-test-model-de344b-kserve-fd488b5bc-cgsnp\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") " pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:11.199599 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.199493    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2mcj\" (UniqueName: \"kubernetes.io/projected/db0fb12c-5d60-4df5-a555-227dbd9018e4-kube-api-access-f2mcj\") pod \"auth-test-model-de344b-kserve-fd488b5bc-cgsnp\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") " pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:11.199599 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.199528    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db0fb12c-5d60-4df5-a555-227dbd9018e4-tls-certs\") pod \"auth-test-model-de344b-kserve-fd488b5bc-cgsnp\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") " pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:11.300563 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.300521    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db0fb12c-5d60-4df5-a555-227dbd9018e4-tls-certs\") pod \"auth-test-model-de344b-kserve-fd488b5bc-cgsnp\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") " pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:11.300769 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.300578    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-home\") pod \"auth-test-model-de344b-kserve-fd488b5bc-cgsnp\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") " pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:11.300769 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.300599    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-tmp-dir\") pod \"auth-test-model-de344b-kserve-fd488b5bc-cgsnp\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") " pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:11.300769 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.300633    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-dshm\") pod \"auth-test-model-de344b-kserve-fd488b5bc-cgsnp\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") " pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:11.300769 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.300652    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-model-cache\") pod \"auth-test-model-de344b-kserve-fd488b5bc-cgsnp\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") " pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:11.300769 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.300668    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2mcj\" (UniqueName: \"kubernetes.io/projected/db0fb12c-5d60-4df5-a555-227dbd9018e4-kube-api-access-f2mcj\") pod \"auth-test-model-de344b-kserve-fd488b5bc-cgsnp\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") " pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:11.301123 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.301102    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-tmp-dir\") pod \"auth-test-model-de344b-kserve-fd488b5bc-cgsnp\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") " pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:11.301190 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.301112    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-home\") pod \"auth-test-model-de344b-kserve-fd488b5bc-cgsnp\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") " pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:11.301190 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.301175    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-model-cache\") pod \"auth-test-model-de344b-kserve-fd488b5bc-cgsnp\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") " pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:11.303118 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.303085    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-dshm\") pod \"auth-test-model-de344b-kserve-fd488b5bc-cgsnp\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") " pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:11.303363 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.303343    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db0fb12c-5d60-4df5-a555-227dbd9018e4-tls-certs\") pod \"auth-test-model-de344b-kserve-fd488b5bc-cgsnp\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") " pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:11.308852 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.308833    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2mcj\" (UniqueName: \"kubernetes.io/projected/db0fb12c-5d60-4df5-a555-227dbd9018e4-kube-api-access-f2mcj\") pod \"auth-test-model-de344b-kserve-fd488b5bc-cgsnp\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") " pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:11.496205 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.496171    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:11.634423 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:11.634297    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"]
Jul 27 19:16:11.637158 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:16:11.637122    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb0fb12c_5d60_4df5_a555_227dbd9018e4.slice/crio-1894576cb852f5fb2fe4bc63bbe23a6d04fb4dfa076be150a0f307571c6bafa2 WatchSource:0}: Error finding container 1894576cb852f5fb2fe4bc63bbe23a6d04fb4dfa076be150a0f307571c6bafa2: Status 404 returned error can't find the container with id 1894576cb852f5fb2fe4bc63bbe23a6d04fb4dfa076be150a0f307571c6bafa2
Jul 27 19:16:12.296183 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:12.296140    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp" event={"ID":"db0fb12c-5d60-4df5-a555-227dbd9018e4","Type":"ContainerStarted","Data":"915a300c733dc18687c2b652b054d4d9a45b1d754dcd9c18c3d0d57a6e3bebe9"}
Jul 27 19:16:12.296183 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:12.296183    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp" event={"ID":"db0fb12c-5d60-4df5-a555-227dbd9018e4","Type":"ContainerStarted","Data":"1894576cb852f5fb2fe4bc63bbe23a6d04fb4dfa076be150a0f307571c6bafa2"}
Jul 27 19:16:12.311909 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:12.311838    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp" podStartSLOduration=1.311821916 podStartE2EDuration="1.311821916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:16:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:16:12.310706018 +0000 UTC m=+2680.860999665" watchObservedRunningTime="2026-07-27 19:16:12.311821916 +0000 UTC m=+2680.862115549"
Jul 27 19:16:21.496358 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:21.496310    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:21.496849 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:21.496378    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:21.509527 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:21.509493    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:22.351360 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:22.351275    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:16:31.974550 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:31.974518    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 19:16:31.977209 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:31.977190    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 19:16:38.089081 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.089039    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"]
Jul 27 19:16:38.092521 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.092496    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:38.095666 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.095636    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ab99e9\"/\"auth-test-model-ab99e9-kserve-self-signed-certs\""
Jul 27 19:16:38.095784 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.095751    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ab99e9\"/\"default-dockercfg-mnjw9\""
Jul 27 19:16:38.095916 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.095900    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ab99e9\"/\"kube-root-ca.crt\""
Jul 27 19:16:38.095991 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.095977    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ab99e9\"/\"openshift-service-ca.crt\""
Jul 27 19:16:38.102057 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.102029    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"]
Jul 27 19:16:38.243109 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.243069    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-tmp-dir\") pod \"auth-test-model-ab99e9-kserve-5d47b59547-qrxv2\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") " pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:38.243287 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.243127    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-model-cache\") pod \"auth-test-model-ab99e9-kserve-5d47b59547-qrxv2\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") " pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:38.243287 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.243150    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-dshm\") pod \"auth-test-model-ab99e9-kserve-5d47b59547-qrxv2\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") " pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:38.243287 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.243189    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4255c531-1a41-4666-9987-52b17f58d916-tls-certs\") pod \"auth-test-model-ab99e9-kserve-5d47b59547-qrxv2\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") " pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:38.243287 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.243233    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-home\") pod \"auth-test-model-ab99e9-kserve-5d47b59547-qrxv2\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") " pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:38.243422 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.243300    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh4fx\" (UniqueName: \"kubernetes.io/projected/4255c531-1a41-4666-9987-52b17f58d916-kube-api-access-gh4fx\") pod \"auth-test-model-ab99e9-kserve-5d47b59547-qrxv2\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") " pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:38.344619 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.344524    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-model-cache\") pod \"auth-test-model-ab99e9-kserve-5d47b59547-qrxv2\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") " pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:38.344619 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.344573    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-dshm\") pod \"auth-test-model-ab99e9-kserve-5d47b59547-qrxv2\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") " pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:38.344619 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.344588    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4255c531-1a41-4666-9987-52b17f58d916-tls-certs\") pod \"auth-test-model-ab99e9-kserve-5d47b59547-qrxv2\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") " pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:38.344619 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.344609    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-home\") pod \"auth-test-model-ab99e9-kserve-5d47b59547-qrxv2\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") " pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:38.344907 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.344660    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh4fx\" (UniqueName: \"kubernetes.io/projected/4255c531-1a41-4666-9987-52b17f58d916-kube-api-access-gh4fx\") pod \"auth-test-model-ab99e9-kserve-5d47b59547-qrxv2\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") " pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:38.344907 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.344695    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-tmp-dir\") pod \"auth-test-model-ab99e9-kserve-5d47b59547-qrxv2\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") " pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:38.345129 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.345104    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-tmp-dir\") pod \"auth-test-model-ab99e9-kserve-5d47b59547-qrxv2\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") " pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:38.345198 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.345138    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-model-cache\") pod \"auth-test-model-ab99e9-kserve-5d47b59547-qrxv2\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") " pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:38.345198 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.345170    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-home\") pod \"auth-test-model-ab99e9-kserve-5d47b59547-qrxv2\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") " pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:38.347197 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.347174    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-dshm\") pod \"auth-test-model-ab99e9-kserve-5d47b59547-qrxv2\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") " pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:38.347404 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.347385    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4255c531-1a41-4666-9987-52b17f58d916-tls-certs\") pod \"auth-test-model-ab99e9-kserve-5d47b59547-qrxv2\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") " pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:38.354392 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.354363    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh4fx\" (UniqueName: \"kubernetes.io/projected/4255c531-1a41-4666-9987-52b17f58d916-kube-api-access-gh4fx\") pod \"auth-test-model-ab99e9-kserve-5d47b59547-qrxv2\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") " pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:38.403280 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.403234    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:38.542143 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:38.542115    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"]
Jul 27 19:16:38.545094 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:16:38.545065    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4255c531_1a41_4666_9987_52b17f58d916.slice/crio-98fea4aad7f6812a12a22567fb726178ffa6352cbe8862604b3f7e9c0d21a3c3 WatchSource:0}: Error finding container 98fea4aad7f6812a12a22567fb726178ffa6352cbe8862604b3f7e9c0d21a3c3: Status 404 returned error can't find the container with id 98fea4aad7f6812a12a22567fb726178ffa6352cbe8862604b3f7e9c0d21a3c3
Jul 27 19:16:39.413776 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:39.413730    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2" event={"ID":"4255c531-1a41-4666-9987-52b17f58d916","Type":"ContainerStarted","Data":"7cf280d6898d137cf94baeb9836dee222ef8478f373a6175786ad71d181b5d72"}
Jul 27 19:16:39.413776 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:39.413790    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2" event={"ID":"4255c531-1a41-4666-9987-52b17f58d916","Type":"ContainerStarted","Data":"98fea4aad7f6812a12a22567fb726178ffa6352cbe8862604b3f7e9c0d21a3c3"}
Jul 27 19:16:39.431016 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:39.430949    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2" podStartSLOduration=1.4309128819999999 podStartE2EDuration="1.430912882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:16:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:16:39.430391425 +0000 UTC m=+2707.980685058" watchObservedRunningTime="2026-07-27 19:16:39.430912882 +0000 UTC m=+2707.981206516"
Jul 27 19:16:48.403531 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:48.403464    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:48.403531 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:48.403544    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:48.416330 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:48.416300    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:16:48.466841 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:16:48.466808    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:20:32.149681 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:20:32.149649    2562 scope.go:117] "RemoveContainer" containerID="6cb4235908626225ca89151fc1dd479751c482a9bbdcf7770659949e420938bf"
Jul 27 19:21:21.706010 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:21.705975    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"]
Jul 27 19:21:22.589121 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:22.589082    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"]
Jul 27 19:21:32.000032 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:31.999925    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 19:21:32.004895 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:32.002332    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 19:21:32.169505 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:32.169471    2562 scope.go:117] "RemoveContainer" containerID="6dc3d02ec0def198ba5bdb6d8253da78d871c2679aab5c5cc9454c0656fb1854"
Jul 27 19:21:32.177770 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:32.177748    2562 scope.go:117] "RemoveContainer" containerID="0dc9f86be958d40b1b7157a68251b462fac9116a47bba34ef1338a46095b0159"
Jul 27 19:21:36.725297 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:36.725227    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp" podUID="db0fb12c-5d60-4df5-a555-227dbd9018e4" containerName="main" containerID="cri-o://915a300c733dc18687c2b652b054d4d9a45b1d754dcd9c18c3d0d57a6e3bebe9" gracePeriod=45
Jul 27 19:21:36.974517 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:36.974493    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:21:37.083643 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.083559    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-model-cache\") pod \"db0fb12c-5d60-4df5-a555-227dbd9018e4\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") "
Jul 27 19:21:37.083643 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.083602    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-tmp-dir\") pod \"db0fb12c-5d60-4df5-a555-227dbd9018e4\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") "
Jul 27 19:21:37.083643 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.083641    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-dshm\") pod \"db0fb12c-5d60-4df5-a555-227dbd9018e4\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") "
Jul 27 19:21:37.083950 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.083668    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2mcj\" (UniqueName: \"kubernetes.io/projected/db0fb12c-5d60-4df5-a555-227dbd9018e4-kube-api-access-f2mcj\") pod \"db0fb12c-5d60-4df5-a555-227dbd9018e4\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") "
Jul 27 19:21:37.083950 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.083688    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-home\") pod \"db0fb12c-5d60-4df5-a555-227dbd9018e4\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") "
Jul 27 19:21:37.083950 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.083763    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db0fb12c-5d60-4df5-a555-227dbd9018e4-tls-certs\") pod \"db0fb12c-5d60-4df5-a555-227dbd9018e4\" (UID: \"db0fb12c-5d60-4df5-a555-227dbd9018e4\") "
Jul 27 19:21:37.083950 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.083921    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-model-cache" (OuterVolumeSpecName: "model-cache") pod "db0fb12c-5d60-4df5-a555-227dbd9018e4" (UID: "db0fb12c-5d60-4df5-a555-227dbd9018e4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:21:37.084173 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.084022    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "db0fb12c-5d60-4df5-a555-227dbd9018e4" (UID: "db0fb12c-5d60-4df5-a555-227dbd9018e4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:21:37.084229 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.084188    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-tmp-dir\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:21:37.084229 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.084203    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-model-cache\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:21:37.084229 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.084191    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-home" (OuterVolumeSpecName: "home") pod "db0fb12c-5d60-4df5-a555-227dbd9018e4" (UID: "db0fb12c-5d60-4df5-a555-227dbd9018e4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:21:37.086131 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.086108    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-dshm" (OuterVolumeSpecName: "dshm") pod "db0fb12c-5d60-4df5-a555-227dbd9018e4" (UID: "db0fb12c-5d60-4df5-a555-227dbd9018e4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:21:37.086131 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.086113    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db0fb12c-5d60-4df5-a555-227dbd9018e4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "db0fb12c-5d60-4df5-a555-227dbd9018e4" (UID: "db0fb12c-5d60-4df5-a555-227dbd9018e4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:21:37.086243 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.086109    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db0fb12c-5d60-4df5-a555-227dbd9018e4-kube-api-access-f2mcj" (OuterVolumeSpecName: "kube-api-access-f2mcj") pod "db0fb12c-5d60-4df5-a555-227dbd9018e4" (UID: "db0fb12c-5d60-4df5-a555-227dbd9018e4"). InnerVolumeSpecName "kube-api-access-f2mcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:21:37.184801 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.184763    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db0fb12c-5d60-4df5-a555-227dbd9018e4-tls-certs\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:21:37.184801 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.184792    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-dshm\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:21:37.184801 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.184802    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2mcj\" (UniqueName: \"kubernetes.io/projected/db0fb12c-5d60-4df5-a555-227dbd9018e4-kube-api-access-f2mcj\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:21:37.184801 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.184810    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db0fb12c-5d60-4df5-a555-227dbd9018e4-home\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:21:37.608599 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.608526    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2" podUID="4255c531-1a41-4666-9987-52b17f58d916" containerName="main" containerID="cri-o://7cf280d6898d137cf94baeb9836dee222ef8478f373a6175786ad71d181b5d72" gracePeriod=45
Jul 27 19:21:37.761460 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.761428    2562 generic.go:356] "Generic (PLEG): container finished" podID="4255c531-1a41-4666-9987-52b17f58d916" containerID="7cf280d6898d137cf94baeb9836dee222ef8478f373a6175786ad71d181b5d72" exitCode=0
Jul 27 19:21:37.761924 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.761502    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2" event={"ID":"4255c531-1a41-4666-9987-52b17f58d916","Type":"ContainerDied","Data":"7cf280d6898d137cf94baeb9836dee222ef8478f373a6175786ad71d181b5d72"}
Jul 27 19:21:37.762810 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.762775    2562 generic.go:356] "Generic (PLEG): container finished" podID="db0fb12c-5d60-4df5-a555-227dbd9018e4" containerID="915a300c733dc18687c2b652b054d4d9a45b1d754dcd9c18c3d0d57a6e3bebe9" exitCode=0
Jul 27 19:21:37.762941 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.762827    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"
Jul 27 19:21:37.762941 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.762819    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp" event={"ID":"db0fb12c-5d60-4df5-a555-227dbd9018e4","Type":"ContainerDied","Data":"915a300c733dc18687c2b652b054d4d9a45b1d754dcd9c18c3d0d57a6e3bebe9"}
Jul 27 19:21:37.762941 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.762922    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp" event={"ID":"db0fb12c-5d60-4df5-a555-227dbd9018e4","Type":"ContainerDied","Data":"1894576cb852f5fb2fe4bc63bbe23a6d04fb4dfa076be150a0f307571c6bafa2"}
Jul 27 19:21:37.763066 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.762947    2562 scope.go:117] "RemoveContainer" containerID="915a300c733dc18687c2b652b054d4d9a45b1d754dcd9c18c3d0d57a6e3bebe9"
Jul 27 19:21:37.773022 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.773002    2562 scope.go:117] "RemoveContainer" containerID="915a300c733dc18687c2b652b054d4d9a45b1d754dcd9c18c3d0d57a6e3bebe9"
Jul 27 19:21:37.773362 ip-10-0-132-186 kubenswrapper[2562]: E0727 19:21:37.773343    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"915a300c733dc18687c2b652b054d4d9a45b1d754dcd9c18c3d0d57a6e3bebe9\": container with ID starting with 915a300c733dc18687c2b652b054d4d9a45b1d754dcd9c18c3d0d57a6e3bebe9 not found: ID does not exist" containerID="915a300c733dc18687c2b652b054d4d9a45b1d754dcd9c18c3d0d57a6e3bebe9"
Jul 27 19:21:37.773435 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.773371    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"915a300c733dc18687c2b652b054d4d9a45b1d754dcd9c18c3d0d57a6e3bebe9"} err="failed to get container status \"915a300c733dc18687c2b652b054d4d9a45b1d754dcd9c18c3d0d57a6e3bebe9\": rpc error: code = NotFound desc = could not find container \"915a300c733dc18687c2b652b054d4d9a45b1d754dcd9c18c3d0d57a6e3bebe9\": container with ID starting with 915a300c733dc18687c2b652b054d4d9a45b1d754dcd9c18c3d0d57a6e3bebe9 not found: ID does not exist"
Jul 27 19:21:37.785983 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.785953    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"]
Jul 27 19:21:37.790086 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.790062    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-de344b/auth-test-model-de344b-kserve-fd488b5bc-cgsnp"]
Jul 27 19:21:37.873462 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.873389    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:21:37.937597 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.937565    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db0fb12c-5d60-4df5-a555-227dbd9018e4" path="/var/lib/kubelet/pods/db0fb12c-5d60-4df5-a555-227dbd9018e4/volumes"
Jul 27 19:21:37.990331 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.990288    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gh4fx\" (UniqueName: \"kubernetes.io/projected/4255c531-1a41-4666-9987-52b17f58d916-kube-api-access-gh4fx\") pod \"4255c531-1a41-4666-9987-52b17f58d916\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") "
Jul 27 19:21:37.990526 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.990344    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4255c531-1a41-4666-9987-52b17f58d916-tls-certs\") pod \"4255c531-1a41-4666-9987-52b17f58d916\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") "
Jul 27 19:21:37.990526 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.990369    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-home\") pod \"4255c531-1a41-4666-9987-52b17f58d916\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") "
Jul 27 19:21:37.990526 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.990401    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-tmp-dir\") pod \"4255c531-1a41-4666-9987-52b17f58d916\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") "
Jul 27 19:21:37.990526 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.990425    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-dshm\") pod \"4255c531-1a41-4666-9987-52b17f58d916\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") "
Jul 27 19:21:37.990526 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.990445    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-model-cache\") pod \"4255c531-1a41-4666-9987-52b17f58d916\" (UID: \"4255c531-1a41-4666-9987-52b17f58d916\") "
Jul 27 19:21:37.990769 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.990677    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-home" (OuterVolumeSpecName: "home") pod "4255c531-1a41-4666-9987-52b17f58d916" (UID: "4255c531-1a41-4666-9987-52b17f58d916"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:21:37.990769 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.990714    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4255c531-1a41-4666-9987-52b17f58d916" (UID: "4255c531-1a41-4666-9987-52b17f58d916"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:21:37.990899 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.990792    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-model-cache" (OuterVolumeSpecName: "model-cache") pod "4255c531-1a41-4666-9987-52b17f58d916" (UID: "4255c531-1a41-4666-9987-52b17f58d916"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:21:37.992758 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.992729    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-dshm" (OuterVolumeSpecName: "dshm") pod "4255c531-1a41-4666-9987-52b17f58d916" (UID: "4255c531-1a41-4666-9987-52b17f58d916"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:21:37.992758 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.992743    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4255c531-1a41-4666-9987-52b17f58d916-kube-api-access-gh4fx" (OuterVolumeSpecName: "kube-api-access-gh4fx") pod "4255c531-1a41-4666-9987-52b17f58d916" (UID: "4255c531-1a41-4666-9987-52b17f58d916"). InnerVolumeSpecName "kube-api-access-gh4fx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:21:37.992948 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:37.992781    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4255c531-1a41-4666-9987-52b17f58d916-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4255c531-1a41-4666-9987-52b17f58d916" (UID: "4255c531-1a41-4666-9987-52b17f58d916"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:21:38.091546 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:38.091493    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gh4fx\" (UniqueName: \"kubernetes.io/projected/4255c531-1a41-4666-9987-52b17f58d916-kube-api-access-gh4fx\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:21:38.091546 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:38.091541    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4255c531-1a41-4666-9987-52b17f58d916-tls-certs\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:21:38.091546 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:38.091553    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-home\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:21:38.091841 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:38.091565    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-tmp-dir\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:21:38.091841 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:38.091577    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-dshm\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:21:38.091841 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:38.091588    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4255c531-1a41-4666-9987-52b17f58d916-model-cache\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:21:38.768711 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:38.768667    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2" event={"ID":"4255c531-1a41-4666-9987-52b17f58d916","Type":"ContainerDied","Data":"98fea4aad7f6812a12a22567fb726178ffa6352cbe8862604b3f7e9c0d21a3c3"}
Jul 27 19:21:38.769185 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:38.769011    2562 scope.go:117] "RemoveContainer" containerID="7cf280d6898d137cf94baeb9836dee222ef8478f373a6175786ad71d181b5d72"
Jul 27 19:21:38.769185 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:38.769069    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"
Jul 27 19:21:38.796509 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:38.796477    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"]
Jul 27 19:21:38.800554 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:38.800528    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ab99e9/auth-test-model-ab99e9-kserve-5d47b59547-qrxv2"]
Jul 27 19:21:39.937058 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:39.937017    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4255c531-1a41-4666-9987-52b17f58d916" path="/var/lib/kubelet/pods/4255c531-1a41-4666-9987-52b17f58d916/volumes"
Jul 27 19:21:48.556944 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.556902    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"]
Jul 27 19:21:48.557352 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.557205    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db0fb12c-5d60-4df5-a555-227dbd9018e4" containerName="main"
Jul 27 19:21:48.557352 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.557216    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="db0fb12c-5d60-4df5-a555-227dbd9018e4" containerName="main"
Jul 27 19:21:48.557352 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.557226    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4255c531-1a41-4666-9987-52b17f58d916" containerName="main"
Jul 27 19:21:48.557352 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.557232    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4255c531-1a41-4666-9987-52b17f58d916" containerName="main"
Jul 27 19:21:48.557352 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.557282    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4255c531-1a41-4666-9987-52b17f58d916" containerName="main"
Jul 27 19:21:48.557352 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.557291    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="db0fb12c-5d60-4df5-a555-227dbd9018e4" containerName="main"
Jul 27 19:21:48.560221 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.560201    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:48.562898 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.562845    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-de344b\"/\"default-dockercfg-kcnmj\""
Jul 27 19:21:48.563662 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.563633    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-de344b\"/\"kube-root-ca.crt\""
Jul 27 19:21:48.563662 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.563654    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-de344b\"/\"openshift-service-ca.crt\""
Jul 27 19:21:48.563805 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.563656    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-de344b\"/\"sub-test-model-de344b-kserve-self-signed-certs\""
Jul 27 19:21:48.579917 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.579886    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"]
Jul 27 19:21:48.685202 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.685159    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-dshm\") pod \"sub-test-model-de344b-kserve-5fdbb975bc-gn679\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") " pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:48.685392 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.685215    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-home\") pod \"sub-test-model-de344b-kserve-5fdbb975bc-gn679\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") " pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:48.685392 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.685247    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-model-cache\") pod \"sub-test-model-de344b-kserve-5fdbb975bc-gn679\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") " pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:48.685392 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.685288    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fcab3eb-7858-4683-8140-751246ea958c-tls-certs\") pod \"sub-test-model-de344b-kserve-5fdbb975bc-gn679\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") " pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:48.685392 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.685351    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-tmp-dir\") pod \"sub-test-model-de344b-kserve-5fdbb975bc-gn679\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") " pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:48.685392 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.685386    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z5rj\" (UniqueName: \"kubernetes.io/projected/9fcab3eb-7858-4683-8140-751246ea958c-kube-api-access-4z5rj\") pod \"sub-test-model-de344b-kserve-5fdbb975bc-gn679\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") " pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:48.786012 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.785967    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-dshm\") pod \"sub-test-model-de344b-kserve-5fdbb975bc-gn679\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") " pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:48.786227 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.786022    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-home\") pod \"sub-test-model-de344b-kserve-5fdbb975bc-gn679\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") " pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:48.786227 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.786047    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-model-cache\") pod \"sub-test-model-de344b-kserve-5fdbb975bc-gn679\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") " pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:48.786227 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.786084    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fcab3eb-7858-4683-8140-751246ea958c-tls-certs\") pod \"sub-test-model-de344b-kserve-5fdbb975bc-gn679\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") " pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:48.786227 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.786111    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-tmp-dir\") pod \"sub-test-model-de344b-kserve-5fdbb975bc-gn679\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") " pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:48.786445 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.786234    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z5rj\" (UniqueName: \"kubernetes.io/projected/9fcab3eb-7858-4683-8140-751246ea958c-kube-api-access-4z5rj\") pod \"sub-test-model-de344b-kserve-5fdbb975bc-gn679\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") " pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:48.786503 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.786450    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-home\") pod \"sub-test-model-de344b-kserve-5fdbb975bc-gn679\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") " pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:48.786544 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.786504    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-model-cache\") pod \"sub-test-model-de344b-kserve-5fdbb975bc-gn679\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") " pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:48.786612 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.786587    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-tmp-dir\") pod \"sub-test-model-de344b-kserve-5fdbb975bc-gn679\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") " pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:48.788563 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.788538    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-dshm\") pod \"sub-test-model-de344b-kserve-5fdbb975bc-gn679\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") " pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:48.788784 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.788766    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fcab3eb-7858-4683-8140-751246ea958c-tls-certs\") pod \"sub-test-model-de344b-kserve-5fdbb975bc-gn679\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") " pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:48.799340 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.799312    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z5rj\" (UniqueName: \"kubernetes.io/projected/9fcab3eb-7858-4683-8140-751246ea958c-kube-api-access-4z5rj\") pod \"sub-test-model-de344b-kserve-5fdbb975bc-gn679\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") " pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:48.869152 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:48.869052    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:49.008512 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:49.008484    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"]
Jul 27 19:21:49.010697 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:21:49.010673    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9fcab3eb_7858_4683_8140_751246ea958c.slice/crio-2ccc757140071f2281ecb4d03dcd8e37c94936649d8ef4ed63fa68b55f3829bb WatchSource:0}: Error finding container 2ccc757140071f2281ecb4d03dcd8e37c94936649d8ef4ed63fa68b55f3829bb: Status 404 returned error can't find the container with id 2ccc757140071f2281ecb4d03dcd8e37c94936649d8ef4ed63fa68b55f3829bb
Jul 27 19:21:49.826641 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:49.826605    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679" event={"ID":"9fcab3eb-7858-4683-8140-751246ea958c","Type":"ContainerStarted","Data":"beec758342e3af37186a74f893c9cfa5646821801d4dd54311a26d1fb2ffe703"}
Jul 27 19:21:49.826641 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:49.826647    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679" event={"ID":"9fcab3eb-7858-4683-8140-751246ea958c","Type":"ContainerStarted","Data":"2ccc757140071f2281ecb4d03dcd8e37c94936649d8ef4ed63fa68b55f3829bb"}
Jul 27 19:21:49.860062 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:49.860005    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679" podStartSLOduration=1.859988151 podStartE2EDuration="1.859988151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:21:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:21:49.857912082 +0000 UTC m=+3018.408205716" watchObservedRunningTime="2026-07-27 19:21:49.859988151 +0000 UTC m=+3018.410281784"
Jul 27 19:21:58.869180 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:58.869143    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:58.869827 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:58.869479    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:58.882355 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:58.882325    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:21:59.881657 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:21:59.881628    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:22:11.506132 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.506097    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"]
Jul 27 19:22:11.509330 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.509305    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:11.512046 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.512016    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ab99e9\"/\"openshift-service-ca.crt\""
Jul 27 19:22:11.513117 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.513089    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ab99e9\"/\"kube-root-ca.crt\""
Jul 27 19:22:11.513228 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.513142    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ab99e9\"/\"sub-test-model-ab99e9-kserve-self-signed-certs\""
Jul 27 19:22:11.513228 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.513148    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ab99e9\"/\"default-dockercfg-mnjw9\""
Jul 27 19:22:11.525780 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.525750    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"]
Jul 27 19:22:11.576363 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.576325    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-tmp-dir\") pod \"sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") " pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:11.576534 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.576377    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdn4w\" (UniqueName: \"kubernetes.io/projected/96bed640-35c1-45c6-8c2f-a9ac134d9642-kube-api-access-bdn4w\") pod \"sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") " pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:11.576534 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.576481    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-home\") pod \"sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") " pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:11.576534 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.576514    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96bed640-35c1-45c6-8c2f-a9ac134d9642-tls-certs\") pod \"sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") " pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:11.576646 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.576548    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-dshm\") pod \"sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") " pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:11.576646 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.576586    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-model-cache\") pod \"sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") " pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:11.677252 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.677210    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-tmp-dir\") pod \"sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") " pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:11.677252 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.677260    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdn4w\" (UniqueName: \"kubernetes.io/projected/96bed640-35c1-45c6-8c2f-a9ac134d9642-kube-api-access-bdn4w\") pod \"sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") " pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:11.677475 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.677287    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-home\") pod \"sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") " pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:11.677475 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.677405    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96bed640-35c1-45c6-8c2f-a9ac134d9642-tls-certs\") pod \"sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") " pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:11.677551 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.677474    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-dshm\") pod \"sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") " pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:11.677551 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.677517    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-model-cache\") pod \"sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") " pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:11.677678 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.677658    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-home\") pod \"sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") " pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:11.677801 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.677771    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-tmp-dir\") pod \"sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") " pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:11.677945 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.677813    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-model-cache\") pod \"sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") " pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:11.679917 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.679898    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-dshm\") pod \"sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") " pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:11.680197 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.680179    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96bed640-35c1-45c6-8c2f-a9ac134d9642-tls-certs\") pod \"sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") " pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:11.687595 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.687569    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdn4w\" (UniqueName: \"kubernetes.io/projected/96bed640-35c1-45c6-8c2f-a9ac134d9642-kube-api-access-bdn4w\") pod \"sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") " pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:11.820019 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.819918    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:11.967421 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:11.967384    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"]
Jul 27 19:22:11.971462 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:22:11.971430    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod96bed640_35c1_45c6_8c2f_a9ac134d9642.slice/crio-77d628034cfc5fb5a8b95b047b8e1d24abce2372f0be2c72ded4f26325260f98 WatchSource:0}: Error finding container 77d628034cfc5fb5a8b95b047b8e1d24abce2372f0be2c72ded4f26325260f98: Status 404 returned error can't find the container with id 77d628034cfc5fb5a8b95b047b8e1d24abce2372f0be2c72ded4f26325260f98
Jul 27 19:22:12.930013 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:12.929981    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk" event={"ID":"96bed640-35c1-45c6-8c2f-a9ac134d9642","Type":"ContainerStarted","Data":"20833ea569975bb23a4d85145a43006d126ce790e4ac6d4bcbf3fb73823d58c4"}
Jul 27 19:22:12.930421 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:12.930021    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk" event={"ID":"96bed640-35c1-45c6-8c2f-a9ac134d9642","Type":"ContainerStarted","Data":"77d628034cfc5fb5a8b95b047b8e1d24abce2372f0be2c72ded4f26325260f98"}
Jul 27 19:22:12.955737 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:12.955683    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk" podStartSLOduration=1.9556666809999999 podStartE2EDuration="1.955666681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:22:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:22:12.955471377 +0000 UTC m=+3041.505765013" watchObservedRunningTime="2026-07-27 19:22:12.955666681 +0000 UTC m=+3041.505960368"
Jul 27 19:22:21.821020 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:21.820965    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:21.821020 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:21.821023    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:21.834672 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:21.834639    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:22:21.977743 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:22:21.977706    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:23:12.294549 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:12.294511    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"]
Jul 27 19:23:13.285543 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:13.285510    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"]
Jul 27 19:23:22.926788 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:22.926751    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"]
Jul 27 19:23:22.929987 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:22.929968    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:22.933115 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:22.933086    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-de344b\"/\"rate-test-model-de344b-kserve-self-signed-certs\""
Jul 27 19:23:22.946298 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:22.946173    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"]
Jul 27 19:23:23.005352 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:23.005318    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-tmp-dir\") pod \"rate-test-model-de344b-kserve-54d7566f9f-75nj5\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") " pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:23.005541 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:23.005378    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-model-cache\") pod \"rate-test-model-de344b-kserve-54d7566f9f-75nj5\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") " pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:23.005541 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:23.005428    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-dshm\") pod \"rate-test-model-de344b-kserve-54d7566f9f-75nj5\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") " pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:23.005541 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:23.005452    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s77l6\" (UniqueName: \"kubernetes.io/projected/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-kube-api-access-s77l6\") pod \"rate-test-model-de344b-kserve-54d7566f9f-75nj5\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") " pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:23.005651 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:23.005574    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-home\") pod \"rate-test-model-de344b-kserve-54d7566f9f-75nj5\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") " pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:23.005651 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:23.005621    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-tls-certs\") pod \"rate-test-model-de344b-kserve-54d7566f9f-75nj5\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") " pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:23.106597 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:23.106557    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-home\") pod \"rate-test-model-de344b-kserve-54d7566f9f-75nj5\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") " pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:23.106774 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:23.106614    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-tls-certs\") pod \"rate-test-model-de344b-kserve-54d7566f9f-75nj5\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") " pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:23.106774 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:23.106655    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-tmp-dir\") pod \"rate-test-model-de344b-kserve-54d7566f9f-75nj5\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") " pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:23.106851 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:23.106797    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-model-cache\") pod \"rate-test-model-de344b-kserve-54d7566f9f-75nj5\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") " pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:23.106851 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:23.106836    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-dshm\") pod \"rate-test-model-de344b-kserve-54d7566f9f-75nj5\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") " pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:23.106944 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:23.106858    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s77l6\" (UniqueName: \"kubernetes.io/projected/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-kube-api-access-s77l6\") pod \"rate-test-model-de344b-kserve-54d7566f9f-75nj5\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") " pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:23.107067 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:23.107047    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-home\") pod \"rate-test-model-de344b-kserve-54d7566f9f-75nj5\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") " pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:23.107108 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:23.107062    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-tmp-dir\") pod \"rate-test-model-de344b-kserve-54d7566f9f-75nj5\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") " pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:23.107160 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:23.107137    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-model-cache\") pod \"rate-test-model-de344b-kserve-54d7566f9f-75nj5\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") " pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:23.109165 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:23.109135    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-dshm\") pod \"rate-test-model-de344b-kserve-54d7566f9f-75nj5\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") " pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:23.109332 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:23.109307    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-tls-certs\") pod \"rate-test-model-de344b-kserve-54d7566f9f-75nj5\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") " pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:23.116276 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:23.116250    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s77l6\" (UniqueName: \"kubernetes.io/projected/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-kube-api-access-s77l6\") pod \"rate-test-model-de344b-kserve-54d7566f9f-75nj5\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") " pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:23.239122 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:23.239084    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:23.388676 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:23.388638    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"]
Jul 27 19:23:23.392068 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:23:23.392032    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ecf6b97_f7a4_4e4a_b56f_937edf0c61ec.slice/crio-9407109136b73c519f802aa7482e025d1c22ad4ea15d20edb5e9929f68656829 WatchSource:0}: Error finding container 9407109136b73c519f802aa7482e025d1c22ad4ea15d20edb5e9929f68656829: Status 404 returned error can't find the container with id 9407109136b73c519f802aa7482e025d1c22ad4ea15d20edb5e9929f68656829
Jul 27 19:23:24.229411 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:24.229372    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5" event={"ID":"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec","Type":"ContainerStarted","Data":"4f53eb99d5e8ab64f59acccc387a4c1d31f23291c66c7fde75b0778564992cd3"}
Jul 27 19:23:24.229411 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:24.229418    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5" event={"ID":"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec","Type":"ContainerStarted","Data":"9407109136b73c519f802aa7482e025d1c22ad4ea15d20edb5e9929f68656829"}
Jul 27 19:23:24.251446 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:24.251390    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5" podStartSLOduration=2.251372043 podStartE2EDuration="2.251372043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:23:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:23:24.250505315 +0000 UTC m=+3112.800798946" watchObservedRunningTime="2026-07-27 19:23:24.251372043 +0000 UTC m=+3112.801665675"
Jul 27 19:23:27.313204 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:27.313153    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679" podUID="9fcab3eb-7858-4683-8140-751246ea958c" containerName="main" containerID="cri-o://beec758342e3af37186a74f893c9cfa5646821801d4dd54311a26d1fb2ffe703" gracePeriod=45
Jul 27 19:23:27.561158 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:27.561129    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:23:27.657239 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:27.657140    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-dshm\") pod \"9fcab3eb-7858-4683-8140-751246ea958c\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") "
Jul 27 19:23:27.657239 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:27.657177    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-tmp-dir\") pod \"9fcab3eb-7858-4683-8140-751246ea958c\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") "
Jul 27 19:23:27.657239 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:27.657226    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4z5rj\" (UniqueName: \"kubernetes.io/projected/9fcab3eb-7858-4683-8140-751246ea958c-kube-api-access-4z5rj\") pod \"9fcab3eb-7858-4683-8140-751246ea958c\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") "
Jul 27 19:23:27.657536 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:27.657256    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-model-cache\") pod \"9fcab3eb-7858-4683-8140-751246ea958c\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") "
Jul 27 19:23:27.657536 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:27.657318    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-home\") pod \"9fcab3eb-7858-4683-8140-751246ea958c\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") "
Jul 27 19:23:27.657536 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:27.657433    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fcab3eb-7858-4683-8140-751246ea958c-tls-certs\") pod \"9fcab3eb-7858-4683-8140-751246ea958c\" (UID: \"9fcab3eb-7858-4683-8140-751246ea958c\") "
Jul 27 19:23:27.657688 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:27.657537    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-model-cache" (OuterVolumeSpecName: "model-cache") pod "9fcab3eb-7858-4683-8140-751246ea958c" (UID: "9fcab3eb-7858-4683-8140-751246ea958c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:23:27.657688 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:27.657569    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9fcab3eb-7858-4683-8140-751246ea958c" (UID: "9fcab3eb-7858-4683-8140-751246ea958c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:23:27.657688 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:27.657581    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-home" (OuterVolumeSpecName: "home") pod "9fcab3eb-7858-4683-8140-751246ea958c" (UID: "9fcab3eb-7858-4683-8140-751246ea958c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:23:27.657829 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:27.657781    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-tmp-dir\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:23:27.657829 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:27.657803    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-model-cache\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:23:27.657829 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:27.657815    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-home\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:23:27.659589 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:27.659565    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-dshm" (OuterVolumeSpecName: "dshm") pod "9fcab3eb-7858-4683-8140-751246ea958c" (UID: "9fcab3eb-7858-4683-8140-751246ea958c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:23:27.659589 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:27.659570    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fcab3eb-7858-4683-8140-751246ea958c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9fcab3eb-7858-4683-8140-751246ea958c" (UID: "9fcab3eb-7858-4683-8140-751246ea958c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:23:27.659589 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:27.659568    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fcab3eb-7858-4683-8140-751246ea958c-kube-api-access-4z5rj" (OuterVolumeSpecName: "kube-api-access-4z5rj") pod "9fcab3eb-7858-4683-8140-751246ea958c" (UID: "9fcab3eb-7858-4683-8140-751246ea958c"). InnerVolumeSpecName "kube-api-access-4z5rj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:23:27.758564 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:27.758527    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fcab3eb-7858-4683-8140-751246ea958c-tls-certs\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:23:27.758564 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:27.758555    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fcab3eb-7858-4683-8140-751246ea958c-dshm\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:23:27.758564 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:27.758564    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4z5rj\" (UniqueName: \"kubernetes.io/projected/9fcab3eb-7858-4683-8140-751246ea958c-kube-api-access-4z5rj\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:23:28.247281 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.247244    2562 generic.go:356] "Generic (PLEG): container finished" podID="9fcab3eb-7858-4683-8140-751246ea958c" containerID="beec758342e3af37186a74f893c9cfa5646821801d4dd54311a26d1fb2ffe703" exitCode=0
Jul 27 19:23:28.247469 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.247291    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"
Jul 27 19:23:28.247469 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.247327    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679" event={"ID":"9fcab3eb-7858-4683-8140-751246ea958c","Type":"ContainerDied","Data":"beec758342e3af37186a74f893c9cfa5646821801d4dd54311a26d1fb2ffe703"}
Jul 27 19:23:28.247469 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.247378    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679" event={"ID":"9fcab3eb-7858-4683-8140-751246ea958c","Type":"ContainerDied","Data":"2ccc757140071f2281ecb4d03dcd8e37c94936649d8ef4ed63fa68b55f3829bb"}
Jul 27 19:23:28.247469 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.247402    2562 scope.go:117] "RemoveContainer" containerID="beec758342e3af37186a74f893c9cfa5646821801d4dd54311a26d1fb2ffe703"
Jul 27 19:23:28.255882 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.255848    2562 scope.go:117] "RemoveContainer" containerID="beec758342e3af37186a74f893c9cfa5646821801d4dd54311a26d1fb2ffe703"
Jul 27 19:23:28.256219 ip-10-0-132-186 kubenswrapper[2562]: E0727 19:23:28.256181    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"beec758342e3af37186a74f893c9cfa5646821801d4dd54311a26d1fb2ffe703\": container with ID starting with beec758342e3af37186a74f893c9cfa5646821801d4dd54311a26d1fb2ffe703 not found: ID does not exist" containerID="beec758342e3af37186a74f893c9cfa5646821801d4dd54311a26d1fb2ffe703"
Jul 27 19:23:28.256219 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.256210    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"beec758342e3af37186a74f893c9cfa5646821801d4dd54311a26d1fb2ffe703"} err="failed to get container status \"beec758342e3af37186a74f893c9cfa5646821801d4dd54311a26d1fb2ffe703\": rpc error: code = NotFound desc = could not find container \"beec758342e3af37186a74f893c9cfa5646821801d4dd54311a26d1fb2ffe703\": container with ID starting with beec758342e3af37186a74f893c9cfa5646821801d4dd54311a26d1fb2ffe703 not found: ID does not exist"
Jul 27 19:23:28.278409 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.278368    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"]
Jul 27 19:23:28.289171 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.289133    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-de344b/sub-test-model-de344b-kserve-5fdbb975bc-gn679"]
Jul 27 19:23:28.304279 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.304246    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk" podUID="96bed640-35c1-45c6-8c2f-a9ac134d9642" containerName="main" containerID="cri-o://20833ea569975bb23a4d85145a43006d126ce790e4ac6d4bcbf3fb73823d58c4" gracePeriod=45
Jul 27 19:23:28.560355 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.560329    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:23:28.666263 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.666226    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdn4w\" (UniqueName: \"kubernetes.io/projected/96bed640-35c1-45c6-8c2f-a9ac134d9642-kube-api-access-bdn4w\") pod \"96bed640-35c1-45c6-8c2f-a9ac134d9642\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") "
Jul 27 19:23:28.666445 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.666298    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-tmp-dir\") pod \"96bed640-35c1-45c6-8c2f-a9ac134d9642\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") "
Jul 27 19:23:28.666445 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.666316    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-model-cache\") pod \"96bed640-35c1-45c6-8c2f-a9ac134d9642\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") "
Jul 27 19:23:28.666445 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.666349    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96bed640-35c1-45c6-8c2f-a9ac134d9642-tls-certs\") pod \"96bed640-35c1-45c6-8c2f-a9ac134d9642\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") "
Jul 27 19:23:28.666445 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.666417    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-dshm\") pod \"96bed640-35c1-45c6-8c2f-a9ac134d9642\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") "
Jul 27 19:23:28.666632 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.666454    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-home\") pod \"96bed640-35c1-45c6-8c2f-a9ac134d9642\" (UID: \"96bed640-35c1-45c6-8c2f-a9ac134d9642\") "
Jul 27 19:23:28.666632 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.666616    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "96bed640-35c1-45c6-8c2f-a9ac134d9642" (UID: "96bed640-35c1-45c6-8c2f-a9ac134d9642"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:23:28.666747 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.666627    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-model-cache" (OuterVolumeSpecName: "model-cache") pod "96bed640-35c1-45c6-8c2f-a9ac134d9642" (UID: "96bed640-35c1-45c6-8c2f-a9ac134d9642"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:23:28.666825 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.666802    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-tmp-dir\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:23:28.666902 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.666831    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-model-cache\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:23:28.666902 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.666826    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-home" (OuterVolumeSpecName: "home") pod "96bed640-35c1-45c6-8c2f-a9ac134d9642" (UID: "96bed640-35c1-45c6-8c2f-a9ac134d9642"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:23:28.668581 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.668554    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96bed640-35c1-45c6-8c2f-a9ac134d9642-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "96bed640-35c1-45c6-8c2f-a9ac134d9642" (UID: "96bed640-35c1-45c6-8c2f-a9ac134d9642"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:23:28.668678 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.668600    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-dshm" (OuterVolumeSpecName: "dshm") pod "96bed640-35c1-45c6-8c2f-a9ac134d9642" (UID: "96bed640-35c1-45c6-8c2f-a9ac134d9642"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:23:28.668678 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.668645    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96bed640-35c1-45c6-8c2f-a9ac134d9642-kube-api-access-bdn4w" (OuterVolumeSpecName: "kube-api-access-bdn4w") pod "96bed640-35c1-45c6-8c2f-a9ac134d9642" (UID: "96bed640-35c1-45c6-8c2f-a9ac134d9642"). InnerVolumeSpecName "kube-api-access-bdn4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:23:28.768275 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.768234    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-dshm\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:23:28.768275 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.768266    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96bed640-35c1-45c6-8c2f-a9ac134d9642-home\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:23:28.768275 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.768275    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdn4w\" (UniqueName: \"kubernetes.io/projected/96bed640-35c1-45c6-8c2f-a9ac134d9642-kube-api-access-bdn4w\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:23:28.768275 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:28.768285    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96bed640-35c1-45c6-8c2f-a9ac134d9642-tls-certs\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:23:29.253447 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:29.253410    2562 generic.go:356] "Generic (PLEG): container finished" podID="96bed640-35c1-45c6-8c2f-a9ac134d9642" containerID="20833ea569975bb23a4d85145a43006d126ce790e4ac6d4bcbf3fb73823d58c4" exitCode=0
Jul 27 19:23:29.253659 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:29.253457    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"
Jul 27 19:23:29.253659 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:29.253501    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk" event={"ID":"96bed640-35c1-45c6-8c2f-a9ac134d9642","Type":"ContainerDied","Data":"20833ea569975bb23a4d85145a43006d126ce790e4ac6d4bcbf3fb73823d58c4"}
Jul 27 19:23:29.253659 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:29.253551    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk" event={"ID":"96bed640-35c1-45c6-8c2f-a9ac134d9642","Type":"ContainerDied","Data":"77d628034cfc5fb5a8b95b047b8e1d24abce2372f0be2c72ded4f26325260f98"}
Jul 27 19:23:29.253659 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:29.253573    2562 scope.go:117] "RemoveContainer" containerID="20833ea569975bb23a4d85145a43006d126ce790e4ac6d4bcbf3fb73823d58c4"
Jul 27 19:23:29.262660 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:29.262643    2562 scope.go:117] "RemoveContainer" containerID="20833ea569975bb23a4d85145a43006d126ce790e4ac6d4bcbf3fb73823d58c4"
Jul 27 19:23:29.262908 ip-10-0-132-186 kubenswrapper[2562]: E0727 19:23:29.262892    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20833ea569975bb23a4d85145a43006d126ce790e4ac6d4bcbf3fb73823d58c4\": container with ID starting with 20833ea569975bb23a4d85145a43006d126ce790e4ac6d4bcbf3fb73823d58c4 not found: ID does not exist" containerID="20833ea569975bb23a4d85145a43006d126ce790e4ac6d4bcbf3fb73823d58c4"
Jul 27 19:23:29.262965 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:29.262914    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20833ea569975bb23a4d85145a43006d126ce790e4ac6d4bcbf3fb73823d58c4"} err="failed to get container status \"20833ea569975bb23a4d85145a43006d126ce790e4ac6d4bcbf3fb73823d58c4\": rpc error: code = NotFound desc = could not find container \"20833ea569975bb23a4d85145a43006d126ce790e4ac6d4bcbf3fb73823d58c4\": container with ID starting with 20833ea569975bb23a4d85145a43006d126ce790e4ac6d4bcbf3fb73823d58c4 not found: ID does not exist"
Jul 27 19:23:29.274393 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:29.274365    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"]
Jul 27 19:23:29.282184 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:29.282162    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ab99e9/sub-test-model-ab99e9-kserve-64b67f7bcc-cdzdk"]
Jul 27 19:23:29.936531 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:29.936495    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96bed640-35c1-45c6-8c2f-a9ac134d9642" path="/var/lib/kubelet/pods/96bed640-35c1-45c6-8c2f-a9ac134d9642/volumes"
Jul 27 19:23:29.937136 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:29.937113    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fcab3eb-7858-4683-8140-751246ea958c" path="/var/lib/kubelet/pods/9fcab3eb-7858-4683-8140-751246ea958c/volumes"
Jul 27 19:23:33.239247 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:33.239205    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:33.239635 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:33.239289    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:33.251924 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:33.251893    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:23:33.282286 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:23:33.282256    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:24:02.791212 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.791123    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"]
Jul 27 19:24:02.791801 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.791572    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fcab3eb-7858-4683-8140-751246ea958c" containerName="main"
Jul 27 19:24:02.791801 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.791591    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fcab3eb-7858-4683-8140-751246ea958c" containerName="main"
Jul 27 19:24:02.791801 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.791608    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96bed640-35c1-45c6-8c2f-a9ac134d9642" containerName="main"
Jul 27 19:24:02.791801 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.791616    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="96bed640-35c1-45c6-8c2f-a9ac134d9642" containerName="main"
Jul 27 19:24:02.791801 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.791719    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fcab3eb-7858-4683-8140-751246ea958c" containerName="main"
Jul 27 19:24:02.791801 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.791734    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="96bed640-35c1-45c6-8c2f-a9ac134d9642" containerName="main"
Jul 27 19:24:02.794811 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.794785    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:02.798770 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.798739    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ab99e9\"/\"rate-test-model-ab99e9-kserve-self-signed-certs\""
Jul 27 19:24:02.798986 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.798743    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ab99e9\"/\"openshift-service-ca.crt\""
Jul 27 19:24:02.798986 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.798780    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ab99e9\"/\"kube-root-ca.crt\""
Jul 27 19:24:02.798986 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.798805    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ab99e9\"/\"default-dockercfg-mnjw9\""
Jul 27 19:24:02.809139 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.809112    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"]
Jul 27 19:24:02.859956 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.859908    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-model-cache\") pod \"rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") " pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:02.859956 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.859959    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f170a3a9-51ac-4e93-9c76-768eba106616-tls-certs\") pod \"rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") " pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:02.860191 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.860000    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-dshm\") pod \"rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") " pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:02.860191 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.860023    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-home\") pod \"rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") " pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:02.860191 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.860098    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-tmp-dir\") pod \"rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") " pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:02.860191 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.860137    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnw9w\" (UniqueName: \"kubernetes.io/projected/f170a3a9-51ac-4e93-9c76-768eba106616-kube-api-access-xnw9w\") pod \"rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") " pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:02.961416 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.961376    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-model-cache\") pod \"rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") " pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:02.961416 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.961427    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f170a3a9-51ac-4e93-9c76-768eba106616-tls-certs\") pod \"rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") " pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:02.961716 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.961463    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-dshm\") pod \"rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") " pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:02.961716 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.961483    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-home\") pod \"rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") " pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:02.961716 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.961512    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-tmp-dir\") pod \"rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") " pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:02.961716 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.961536    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnw9w\" (UniqueName: \"kubernetes.io/projected/f170a3a9-51ac-4e93-9c76-768eba106616-kube-api-access-xnw9w\") pod \"rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") " pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:02.961930 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.961864    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-model-cache\") pod \"rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") " pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:02.962017 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.961980    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-home\") pod \"rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") " pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:02.962017 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.961998    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-tmp-dir\") pod \"rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") " pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:02.963927 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.963906    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-dshm\") pod \"rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") " pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:02.964201 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.964181    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f170a3a9-51ac-4e93-9c76-768eba106616-tls-certs\") pod \"rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") " pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:02.973388 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:02.973361    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnw9w\" (UniqueName: \"kubernetes.io/projected/f170a3a9-51ac-4e93-9c76-768eba106616-kube-api-access-xnw9w\") pod \"rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") " pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:03.104505 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:03.104423    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:03.247035 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:03.246885    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"]
Jul 27 19:24:03.250748 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:24:03.250712    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf170a3a9_51ac_4e93_9c76_768eba106616.slice/crio-a343e21c39d08a6e16af44947df17d7539e8352e6532e04fce11a5e57475314e WatchSource:0}: Error finding container a343e21c39d08a6e16af44947df17d7539e8352e6532e04fce11a5e57475314e: Status 404 returned error can't find the container with id a343e21c39d08a6e16af44947df17d7539e8352e6532e04fce11a5e57475314e
Jul 27 19:24:03.396661 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:03.396562    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r" event={"ID":"f170a3a9-51ac-4e93-9c76-768eba106616","Type":"ContainerStarted","Data":"ec8cc6137eca7282222feab009c8263c103e359f514604a1d53b8c0e6cbc8880"}
Jul 27 19:24:03.396661 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:03.396602    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r" event={"ID":"f170a3a9-51ac-4e93-9c76-768eba106616","Type":"ContainerStarted","Data":"a343e21c39d08a6e16af44947df17d7539e8352e6532e04fce11a5e57475314e"}
Jul 27 19:24:03.417416 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:03.417352    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r" podStartSLOduration=1.417329979 podStartE2EDuration="1.417329979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:24:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:24:03.415746278 +0000 UTC m=+3151.966039911" watchObservedRunningTime="2026-07-27 19:24:03.417329979 +0000 UTC m=+3151.967624008"
Jul 27 19:24:13.104607 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:13.104563    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:13.105134 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:13.104643    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:13.117507 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:13.117477    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:13.452918 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:13.452791    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:24:38.547794 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:38.547737    2562 generic.go:356] "Generic (PLEG): container finished" podID="d0926f63-f6cc-41e9-a804-ca6ccd217df1" containerID="7e6497ff20718bd9e49cb02aea96f6870771ce8e96f7c66e523f5e2a87e3abbb" exitCode=1
Jul 27 19:24:38.548461 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:38.548430    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b" event={"ID":"d0926f63-f6cc-41e9-a804-ca6ccd217df1","Type":"ContainerDied","Data":"7e6497ff20718bd9e49cb02aea96f6870771ce8e96f7c66e523f5e2a87e3abbb"}
Jul 27 19:24:38.548907 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:38.548863    2562 scope.go:117] "RemoveContainer" containerID="7e6497ff20718bd9e49cb02aea96f6870771ce8e96f7c66e523f5e2a87e3abbb"
Jul 27 19:24:39.555126 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:39.555091    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b" event={"ID":"d0926f63-f6cc-41e9-a804-ca6ccd217df1","Type":"ContainerStarted","Data":"f43e0ecc63769ca0e6856c235f516bc8b311ea0c8476c348e9fc5907b9174495"}
Jul 27 19:24:39.555502 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:39.555298    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"
Jul 27 19:24:50.562355 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:24:50.562319    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5c45656c54-qhv7b"
Jul 27 19:25:25.008402 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:25.008309    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"]
Jul 27 19:25:25.991762 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:25.991715    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"]
Jul 27 19:25:40.027990 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.027926    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5" podUID="3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec" containerName="main" containerID="cri-o://4f53eb99d5e8ab64f59acccc387a4c1d31f23291c66c7fde75b0778564992cd3" gracePeriod=45
Jul 27 19:25:40.272164 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.272136    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:25:40.336846 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.336743    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-tmp-dir\") pod \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") "
Jul 27 19:25:40.336846 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.336788    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-home\") pod \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") "
Jul 27 19:25:40.336846 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.336835    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-model-cache\") pod \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") "
Jul 27 19:25:40.337171 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.336932    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-tls-certs\") pod \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") "
Jul 27 19:25:40.337171 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.336969    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s77l6\" (UniqueName: \"kubernetes.io/projected/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-kube-api-access-s77l6\") pod \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") "
Jul 27 19:25:40.337171 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.337013    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-dshm\") pod \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\" (UID: \"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec\") "
Jul 27 19:25:40.337171 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.337120    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec" (UID: "3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:25:40.337171 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.337156    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-home" (OuterVolumeSpecName: "home") pod "3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec" (UID: "3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:25:40.337400 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.337194    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-model-cache" (OuterVolumeSpecName: "model-cache") pod "3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec" (UID: "3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:25:40.337400 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.337345    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-model-cache\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:25:40.337400 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.337362    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-tmp-dir\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:25:40.337400 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.337370    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-home\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:25:40.339298 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.339258    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec" (UID: "3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:25:40.339298 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.339270    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-dshm" (OuterVolumeSpecName: "dshm") pod "3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec" (UID: "3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:25:40.339472 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.339318    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-kube-api-access-s77l6" (OuterVolumeSpecName: "kube-api-access-s77l6") pod "3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec" (UID: "3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec"). InnerVolumeSpecName "kube-api-access-s77l6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:25:40.438100 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.438058    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-tls-certs\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:25:40.438100 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.438091    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s77l6\" (UniqueName: \"kubernetes.io/projected/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-kube-api-access-s77l6\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:25:40.438100 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.438101    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec-dshm\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:25:40.816166 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.816123    2562 generic.go:356] "Generic (PLEG): container finished" podID="3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec" containerID="4f53eb99d5e8ab64f59acccc387a4c1d31f23291c66c7fde75b0778564992cd3" exitCode=0
Jul 27 19:25:40.816338 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.816184    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"
Jul 27 19:25:40.816338 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.816212    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5" event={"ID":"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec","Type":"ContainerDied","Data":"4f53eb99d5e8ab64f59acccc387a4c1d31f23291c66c7fde75b0778564992cd3"}
Jul 27 19:25:40.816338 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.816262    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5" event={"ID":"3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec","Type":"ContainerDied","Data":"9407109136b73c519f802aa7482e025d1c22ad4ea15d20edb5e9929f68656829"}
Jul 27 19:25:40.816338 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.816280    2562 scope.go:117] "RemoveContainer" containerID="4f53eb99d5e8ab64f59acccc387a4c1d31f23291c66c7fde75b0778564992cd3"
Jul 27 19:25:40.825262 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.825245    2562 scope.go:117] "RemoveContainer" containerID="4f53eb99d5e8ab64f59acccc387a4c1d31f23291c66c7fde75b0778564992cd3"
Jul 27 19:25:40.825563 ip-10-0-132-186 kubenswrapper[2562]: E0727 19:25:40.825541    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f53eb99d5e8ab64f59acccc387a4c1d31f23291c66c7fde75b0778564992cd3\": container with ID starting with 4f53eb99d5e8ab64f59acccc387a4c1d31f23291c66c7fde75b0778564992cd3 not found: ID does not exist" containerID="4f53eb99d5e8ab64f59acccc387a4c1d31f23291c66c7fde75b0778564992cd3"
Jul 27 19:25:40.825630 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.825573    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f53eb99d5e8ab64f59acccc387a4c1d31f23291c66c7fde75b0778564992cd3"} err="failed to get container status \"4f53eb99d5e8ab64f59acccc387a4c1d31f23291c66c7fde75b0778564992cd3\": rpc error: code = NotFound desc = could not find container \"4f53eb99d5e8ab64f59acccc387a4c1d31f23291c66c7fde75b0778564992cd3\": container with ID starting with 4f53eb99d5e8ab64f59acccc387a4c1d31f23291c66c7fde75b0778564992cd3 not found: ID does not exist"
Jul 27 19:25:40.837877 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.837838    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"]
Jul 27 19:25:40.843285 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:40.843259    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-de344b/rate-test-model-de344b-kserve-54d7566f9f-75nj5"]
Jul 27 19:25:41.017778 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.017724    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r" podUID="f170a3a9-51ac-4e93-9c76-768eba106616" containerName="main" containerID="cri-o://ec8cc6137eca7282222feab009c8263c103e359f514604a1d53b8c0e6cbc8880" gracePeriod=45
Jul 27 19:25:41.273565 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.273540    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:25:41.346605 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.346505    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f170a3a9-51ac-4e93-9c76-768eba106616-tls-certs\") pod \"f170a3a9-51ac-4e93-9c76-768eba106616\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") "
Jul 27 19:25:41.346605 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.346549    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-dshm\") pod \"f170a3a9-51ac-4e93-9c76-768eba106616\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") "
Jul 27 19:25:41.346605 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.346603    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-home\") pod \"f170a3a9-51ac-4e93-9c76-768eba106616\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") "
Jul 27 19:25:41.346912 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.346648    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-model-cache\") pod \"f170a3a9-51ac-4e93-9c76-768eba106616\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") "
Jul 27 19:25:41.346912 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.346698    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-tmp-dir\") pod \"f170a3a9-51ac-4e93-9c76-768eba106616\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") "
Jul 27 19:25:41.346912 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.346724    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnw9w\" (UniqueName: \"kubernetes.io/projected/f170a3a9-51ac-4e93-9c76-768eba106616-kube-api-access-xnw9w\") pod \"f170a3a9-51ac-4e93-9c76-768eba106616\" (UID: \"f170a3a9-51ac-4e93-9c76-768eba106616\") "
Jul 27 19:25:41.347073 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.346902    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-home" (OuterVolumeSpecName: "home") pod "f170a3a9-51ac-4e93-9c76-768eba106616" (UID: "f170a3a9-51ac-4e93-9c76-768eba106616"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:25:41.347073 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.346975    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f170a3a9-51ac-4e93-9c76-768eba106616" (UID: "f170a3a9-51ac-4e93-9c76-768eba106616"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:25:41.347073 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.346984    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-model-cache" (OuterVolumeSpecName: "model-cache") pod "f170a3a9-51ac-4e93-9c76-768eba106616" (UID: "f170a3a9-51ac-4e93-9c76-768eba106616"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:25:41.347073 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.347044    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-home\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:25:41.348960 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.348927    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f170a3a9-51ac-4e93-9c76-768eba106616-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f170a3a9-51ac-4e93-9c76-768eba106616" (UID: "f170a3a9-51ac-4e93-9c76-768eba106616"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:25:41.349084 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.348962    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-dshm" (OuterVolumeSpecName: "dshm") pod "f170a3a9-51ac-4e93-9c76-768eba106616" (UID: "f170a3a9-51ac-4e93-9c76-768eba106616"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:25:41.349084 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.349006    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f170a3a9-51ac-4e93-9c76-768eba106616-kube-api-access-xnw9w" (OuterVolumeSpecName: "kube-api-access-xnw9w") pod "f170a3a9-51ac-4e93-9c76-768eba106616" (UID: "f170a3a9-51ac-4e93-9c76-768eba106616"). InnerVolumeSpecName "kube-api-access-xnw9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:25:41.447743 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.447702    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-model-cache\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:25:41.447743 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.447735    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-tmp-dir\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:25:41.447743 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.447745    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnw9w\" (UniqueName: \"kubernetes.io/projected/f170a3a9-51ac-4e93-9c76-768eba106616-kube-api-access-xnw9w\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:25:41.447743 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.447755    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f170a3a9-51ac-4e93-9c76-768eba106616-tls-certs\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:25:41.448065 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.447767    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f170a3a9-51ac-4e93-9c76-768eba106616-dshm\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:25:41.699342 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.699262    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7574-140135784500224"]
Jul 27 19:25:41.699602 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.699588    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f170a3a9-51ac-4e93-9c76-768eba106616" containerName="main"
Jul 27 19:25:41.699647 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.699603    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f170a3a9-51ac-4e93-9c76-768eba106616" containerName="main"
Jul 27 19:25:41.699647 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.699624    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec" containerName="main"
Jul 27 19:25:41.699647 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.699629    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec" containerName="main"
Jul 27 19:25:41.699746 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.699678    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f170a3a9-51ac-4e93-9c76-768eba106616" containerName="main"
Jul 27 19:25:41.699746 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.699690    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec" containerName="main"
Jul 27 19:25:41.702648 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.702632    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140135784500224"
Jul 27 19:25:41.705841 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.705821    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-w5rhc\""
Jul 27 19:25:41.714983 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.714954    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-140135784500224"]
Jul 27 19:25:41.751852 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.751810    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk856\" (UniqueName: \"kubernetes.io/projected/ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed-kube-api-access-tk856\") pod \"test-curl-7574-140135784500224\" (UID: \"ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed\") " pod="opendatahub/test-curl-7574-140135784500224"
Jul 27 19:25:41.823826 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.823784    2562 generic.go:356] "Generic (PLEG): container finished" podID="f170a3a9-51ac-4e93-9c76-768eba106616" containerID="ec8cc6137eca7282222feab009c8263c103e359f514604a1d53b8c0e6cbc8880" exitCode=0
Jul 27 19:25:41.823826 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.823829    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"
Jul 27 19:25:41.824103 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.823842    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r" event={"ID":"f170a3a9-51ac-4e93-9c76-768eba106616","Type":"ContainerDied","Data":"ec8cc6137eca7282222feab009c8263c103e359f514604a1d53b8c0e6cbc8880"}
Jul 27 19:25:41.824103 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.823903    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r" event={"ID":"f170a3a9-51ac-4e93-9c76-768eba106616","Type":"ContainerDied","Data":"a343e21c39d08a6e16af44947df17d7539e8352e6532e04fce11a5e57475314e"}
Jul 27 19:25:41.824103 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.823927    2562 scope.go:117] "RemoveContainer" containerID="ec8cc6137eca7282222feab009c8263c103e359f514604a1d53b8c0e6cbc8880"
Jul 27 19:25:41.832849 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.832830    2562 scope.go:117] "RemoveContainer" containerID="ec8cc6137eca7282222feab009c8263c103e359f514604a1d53b8c0e6cbc8880"
Jul 27 19:25:41.833154 ip-10-0-132-186 kubenswrapper[2562]: E0727 19:25:41.833135    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec8cc6137eca7282222feab009c8263c103e359f514604a1d53b8c0e6cbc8880\": container with ID starting with ec8cc6137eca7282222feab009c8263c103e359f514604a1d53b8c0e6cbc8880 not found: ID does not exist" containerID="ec8cc6137eca7282222feab009c8263c103e359f514604a1d53b8c0e6cbc8880"
Jul 27 19:25:41.833208 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.833165    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec8cc6137eca7282222feab009c8263c103e359f514604a1d53b8c0e6cbc8880"} err="failed to get container status \"ec8cc6137eca7282222feab009c8263c103e359f514604a1d53b8c0e6cbc8880\": rpc error: code = NotFound desc = could not find container \"ec8cc6137eca7282222feab009c8263c103e359f514604a1d53b8c0e6cbc8880\": container with ID starting with ec8cc6137eca7282222feab009c8263c103e359f514604a1d53b8c0e6cbc8880 not found: ID does not exist"
Jul 27 19:25:41.848562 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.848534    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"]
Jul 27 19:25:41.852089 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.852065    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ab99e9/rate-test-model-ab99e9-kserve-76c6cbb895-d5m4r"]
Jul 27 19:25:41.852412 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.852391    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk856\" (UniqueName: \"kubernetes.io/projected/ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed-kube-api-access-tk856\") pod \"test-curl-7574-140135784500224\" (UID: \"ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed\") " pod="opendatahub/test-curl-7574-140135784500224"
Jul 27 19:25:41.862318 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.862293    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk856\" (UniqueName: \"kubernetes.io/projected/ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed-kube-api-access-tk856\") pod \"test-curl-7574-140135784500224\" (UID: \"ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed\") " pod="opendatahub/test-curl-7574-140135784500224"
Jul 27 19:25:41.936945 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.936914    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec" path="/var/lib/kubelet/pods/3ecf6b97-f7a4-4e4a-b56f-937edf0c61ec/volumes"
Jul 27 19:25:41.937316 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:41.937303    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f170a3a9-51ac-4e93-9c76-768eba106616" path="/var/lib/kubelet/pods/f170a3a9-51ac-4e93-9c76-768eba106616/volumes"
Jul 27 19:25:42.012473 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:42.012439    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140135784500224"
Jul 27 19:25:42.152434 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:42.152385    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-140135784500224"]
Jul 27 19:25:42.156235 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:25:42.156201    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec9a64c8_ebe6_46d0_82c6_97fab9afe6ed.slice/crio-0f6a7da4bca767fb94c39cb2dc0a07935d0267861b8cb2c77cf16b967229c6f6 WatchSource:0}: Error finding container 0f6a7da4bca767fb94c39cb2dc0a07935d0267861b8cb2c77cf16b967229c6f6: Status 404 returned error can't find the container with id 0f6a7da4bca767fb94c39cb2dc0a07935d0267861b8cb2c77cf16b967229c6f6
Jul 27 19:25:42.157611 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:42.157591    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 19:25:42.829611 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:42.829577    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-140135784500224" event={"ID":"ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed","Type":"ContainerStarted","Data":"0f6a7da4bca767fb94c39cb2dc0a07935d0267861b8cb2c77cf16b967229c6f6"}
Jul 27 19:25:43.841530 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:43.841488    2562 generic.go:356] "Generic (PLEG): container finished" podID="ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed" containerID="7aa899ff529e14905809bed27267259ef913bd9eecde4f1138309338e5c7ad7d" exitCode=0
Jul 27 19:25:43.841968 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:43.841577    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-140135784500224" event={"ID":"ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed","Type":"ContainerDied","Data":"7aa899ff529e14905809bed27267259ef913bd9eecde4f1138309338e5c7ad7d"}
Jul 27 19:25:44.983285 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:44.983257    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140135784500224"
Jul 27 19:25:45.030984 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:45.030932    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7574-140135784500224_ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed/test-curl-7574-140135784500224/0.log"
Jul 27 19:25:45.056286 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:45.056252    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7574-140135784500224"]
Jul 27 19:25:45.059213 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:45.059185    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7574-140135784500224"]
Jul 27 19:25:45.080663 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:45.080629    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tk856\" (UniqueName: \"kubernetes.io/projected/ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed-kube-api-access-tk856\") pod \"ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed\" (UID: \"ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed\") "
Jul 27 19:25:45.083082 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:45.083051    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed-kube-api-access-tk856" (OuterVolumeSpecName: "kube-api-access-tk856") pod "ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed" (UID: "ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed"). InnerVolumeSpecName "kube-api-access-tk856". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:25:45.181557 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:45.181468    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tk856\" (UniqueName: \"kubernetes.io/projected/ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed-kube-api-access-tk856\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:25:45.852045 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:45.852013    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f6a7da4bca767fb94c39cb2dc0a07935d0267861b8cb2c77cf16b967229c6f6"
Jul 27 19:25:45.852045 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:45.852049    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-140135784500224"
Jul 27 19:25:45.862997 ip-10-0-132-186 kubenswrapper[2562]: E0727 19:25:45.862957    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7574-140135784500224\" is forbidden: User \"system:node:ip-10-0-132-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-186.ec2.internal' and this object" podUID="ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed" pod="opendatahub/test-curl-7574-140135784500224"
Jul 27 19:25:45.936587 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:45.936550    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed" path="/var/lib/kubelet/pods/ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed/volumes"
Jul 27 19:25:46.441459 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.441426    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"]
Jul 27 19:25:46.441955 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.441847    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed" containerName="test-curl-7574-140135784500224"
Jul 27 19:25:46.441955 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.441888    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed" containerName="test-curl-7574-140135784500224"
Jul 27 19:25:46.442092 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.442002    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec9a64c8-ebe6-46d0-82c6-97fab9afe6ed" containerName="test-curl-7574-140135784500224"
Jul 27 19:25:46.445040 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.445020    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.452693 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.452673    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 27 19:25:46.452814 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.452749    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-d4857a-openshift-default-dockercfg-5l45d\""
Jul 27 19:25:46.460098 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.460068    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"]
Jul 27 19:25:46.594138 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.594093    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-workload-socket\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.594320 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.594155    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/444e89a7-263a-40cc-b844-88860c4bdda1-istio-token\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.594320 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.594234    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/444e89a7-263a-40cc-b844-88860c4bdda1-istiod-ca-cert\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.594320 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.594281    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-istio-data\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.594320 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.594302    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-istio-envoy\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.594462 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.594353    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-workload-certs\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.594462 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.594379    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/444e89a7-263a-40cc-b844-88860c4bdda1-istio-podinfo\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.594462 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.594421    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-credential-socket\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.594462 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.594442    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq667\" (UniqueName: \"kubernetes.io/projected/444e89a7-263a-40cc-b844-88860c4bdda1-kube-api-access-rq667\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.695478 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.695376    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-workload-certs\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.695478 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.695416    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/444e89a7-263a-40cc-b844-88860c4bdda1-istio-podinfo\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.695478 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.695448    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-credential-socket\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.695478 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.695467    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq667\" (UniqueName: \"kubernetes.io/projected/444e89a7-263a-40cc-b844-88860c4bdda1-kube-api-access-rq667\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.695813 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.695510    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-workload-socket\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.695813 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.695539    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/444e89a7-263a-40cc-b844-88860c4bdda1-istio-token\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.695813 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.695575    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/444e89a7-263a-40cc-b844-88860c4bdda1-istiod-ca-cert\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.695813 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.695629    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-istio-data\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.695813 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.695655    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-istio-envoy\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.696165 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.695908    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-workload-certs\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.696165 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.696083    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-credential-socket\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.696165 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.696108    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-workload-socket\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.696297 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.696180    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-istio-data\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.696430 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.696411    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/444e89a7-263a-40cc-b844-88860c4bdda1-istiod-ca-cert\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.698070 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.698054    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-istio-envoy\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.698345 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.698328    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/444e89a7-263a-40cc-b844-88860c4bdda1-istio-podinfo\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.704999 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.704974    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq667\" (UniqueName: \"kubernetes.io/projected/444e89a7-263a-40cc-b844-88860c4bdda1-kube-api-access-rq667\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.707199 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.707174    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/444e89a7-263a-40cc-b844-88860c4bdda1-istio-token\") pod \"e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") " pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.755517 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.755479    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:46.904236 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:46.904209    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"]
Jul 27 19:25:46.906199 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:25:46.906169    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod444e89a7_263a_40cc_b844_88860c4bdda1.slice/crio-eb864a36f7f0f0f6ea25c452b48879b492c6de258abebe0439ff6849d2006065 WatchSource:0}: Error finding container eb864a36f7f0f0f6ea25c452b48879b492c6de258abebe0439ff6849d2006065: Status 404 returned error can't find the container with id eb864a36f7f0f0f6ea25c452b48879b492c6de258abebe0439ff6849d2006065
Jul 27 19:25:47.861857 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:47.861814    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8" event={"ID":"444e89a7-263a-40cc-b844-88860c4bdda1","Type":"ContainerStarted","Data":"eb864a36f7f0f0f6ea25c452b48879b492c6de258abebe0439ff6849d2006065"}
Jul 27 19:25:50.166194 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:50.166151    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 27 19:25:50.166609 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:50.166245    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 27 19:25:50.166609 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:50.166290    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 27 19:25:50.879409 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:50.879377    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8" event={"ID":"444e89a7-263a-40cc-b844-88860c4bdda1","Type":"ContainerStarted","Data":"8b38eb48b3849ebcaac5cc26944876a3f34a00ed5cf3a9f3f400556b019b1633"}
Jul 27 19:25:50.915196 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:50.915138    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8" podStartSLOduration=1.657555115 podStartE2EDuration="4.915119796s" totalImagesPullingTime="3.257564681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:25:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:25:50.914135752 +0000 UTC m=+3259.464429406" watchObservedRunningTime="2026-07-27 19:25:50.915119796 +0000 UTC m=+3259.465413430"
Jul 27 19:25:51.756040 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:51.755993    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:51.760953 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:51.760927    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:51.884288 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:51.884256    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:25:51.885409 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:25:51.885390    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:26:05.948403 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:05.948371    2562 generic.go:356] "Generic (PLEG): container finished" podID="444e89a7-263a-40cc-b844-88860c4bdda1" containerID="8b38eb48b3849ebcaac5cc26944876a3f34a00ed5cf3a9f3f400556b019b1633" exitCode=0
Jul 27 19:26:05.948860 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:05.948449    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8" event={"ID":"444e89a7-263a-40cc-b844-88860c4bdda1","Type":"ContainerDied","Data":"8b38eb48b3849ebcaac5cc26944876a3f34a00ed5cf3a9f3f400556b019b1633"}
Jul 27 19:26:05.948943 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:05.948887    2562 scope.go:117] "RemoveContainer" containerID="8b38eb48b3849ebcaac5cc26944876a3f34a00ed5cf3a9f3f400556b019b1633"
Jul 27 19:26:05.951005 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:05.950973    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 27 19:26:05.951122 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:05.951059    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 27 19:26:05.951192 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:05.951106    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 27 19:26:06.885167 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:06.885127    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:26:06.955343 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:06.955311    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8" event={"ID":"444e89a7-263a-40cc-b844-88860c4bdda1","Type":"ContainerStarted","Data":"53182ee3426c2d9adad8eeba3073e7c87aeda69ac11902c04252cde292ceda54"}
Jul 27 19:26:07.756057 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:07.756008    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:26:07.757162 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:07.757122    2562 patch_prober.go:28] interesting pod/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 27 19:26:07.757301 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:07.757183    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8" podUID="444e89a7-263a-40cc-b844-88860c4bdda1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 27 19:26:08.712970 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.712929    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"]
Jul 27 19:26:08.719608 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.719565    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.722283 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.722242    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-2d7b97-openshift-default-dockercfg-sf7ts\""
Jul 27 19:26:08.731398 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.731364    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"]
Jul 27 19:26:08.756327 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.756293    2562 patch_prober.go:28] interesting pod/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 27 19:26:08.756483 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.756353    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8" podUID="444e89a7-263a-40cc-b844-88860c4bdda1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 27 19:26:08.804522 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.804464    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-data\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.804739 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.804542    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-workload-certs\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.804739 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.804581    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-credential-socket\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.804739 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.804610    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvb66\" (UniqueName: \"kubernetes.io/projected/4da4ae70-3dbb-4016-a410-c393df6438f5-kube-api-access-gvb66\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.804739 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.804727    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-podinfo\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.805016 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.804764    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-token\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.805016 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.804813    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-workload-socket\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.805016 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.804837    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-envoy\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.805016 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.804863    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4da4ae70-3dbb-4016-a410-c393df6438f5-istiod-ca-cert\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.905581 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.905537    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-podinfo\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.905581 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.905582    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-token\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.905854 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.905618    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-workload-socket\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.905854 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.905636    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-envoy\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.905854 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.905666    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4da4ae70-3dbb-4016-a410-c393df6438f5-istiod-ca-cert\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.905854 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.905837    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-data\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.906269 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.906224    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-data\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.906269 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.906237    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-workload-certs\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.906480 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.906274    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-workload-socket\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.906480 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.906290    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-credential-socket\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.906480 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.906332    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvb66\" (UniqueName: \"kubernetes.io/projected/4da4ae70-3dbb-4016-a410-c393df6438f5-kube-api-access-gvb66\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.906639 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.906533    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-credential-socket\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.906639 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.906566    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4da4ae70-3dbb-4016-a410-c393df6438f5-istiod-ca-cert\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.906794 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.906759    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-workload-certs\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.908237 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.908216    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-envoy\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.908559 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.908541    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-podinfo\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.917020 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.916994    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-token\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:08.920166 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:08.920142    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvb66\" (UniqueName: \"kubernetes.io/projected/4da4ae70-3dbb-4016-a410-c393df6438f5-kube-api-access-gvb66\") pod \"e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") " pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:09.033834 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:09.033734    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:09.219973 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:09.219852    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"]
Jul 27 19:26:09.222728 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:26:09.222699    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4da4ae70_3dbb_4016_a410_c393df6438f5.slice/crio-c553eec03dad810afe77f1a960020db287914fb19f1e7ea1a031f0dbc283e308 WatchSource:0}: Error finding container c553eec03dad810afe77f1a960020db287914fb19f1e7ea1a031f0dbc283e308: Status 404 returned error can't find the container with id c553eec03dad810afe77f1a960020db287914fb19f1e7ea1a031f0dbc283e308
Jul 27 19:26:09.225082 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:09.225044    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 27 19:26:09.225189 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:09.225137    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 27 19:26:09.225260 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:09.225191    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 27 19:26:09.756470 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:09.756429    2562 patch_prober.go:28] interesting pod/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 27 19:26:09.756899 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:09.756497    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8" podUID="444e89a7-263a-40cc-b844-88860c4bdda1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 27 19:26:09.970779 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:09.970737    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g" event={"ID":"4da4ae70-3dbb-4016-a410-c393df6438f5","Type":"ContainerStarted","Data":"048854f198d810049a216efe8471ec503ed1f74f67297ebc871a3271e34e526b"}
Jul 27 19:26:09.970978 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:09.970799    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g" event={"ID":"4da4ae70-3dbb-4016-a410-c393df6438f5","Type":"ContainerStarted","Data":"c553eec03dad810afe77f1a960020db287914fb19f1e7ea1a031f0dbc283e308"}
Jul 27 19:26:09.992920 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:09.992834    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g" podStartSLOduration=1.992810287 podStartE2EDuration="1.992810287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:26:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:26:09.99236215 +0000 UTC m=+3278.542655785" watchObservedRunningTime="2026-07-27 19:26:09.992810287 +0000 UTC m=+3278.543103910"
Jul 27 19:26:10.034406 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:10.034317    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:10.039344 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:10.039315    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:10.756589 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:10.756547    2562 patch_prober.go:28] interesting pod/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 27 19:26:10.757108 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:10.756606    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8" podUID="444e89a7-263a-40cc-b844-88860c4bdda1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 27 19:26:10.978673 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:10.978640    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:10.979618 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:10.979597    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:11.759712 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:11.759682    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:26:11.760129 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:11.759993    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:26:11.760825 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:11.760806    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:26:13.079027 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:13.078986    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml"]
Jul 27 19:26:13.082266 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:13.082248    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml"
Jul 27 19:26:13.084603 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:13.084572    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-2d7b97\""
Jul 27 19:26:13.084742 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:13.084572    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-2d7b97-dockercfg-xpkkw\""
Jul 27 19:26:13.097384 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:13.097355    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml"]
Jul 27 19:26:13.148668 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:13.148628    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5af5df4d-1117-4c39-a915-f4e0567258e2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml\" (UID: \"5af5df4d-1117-4c39-a915-f4e0567258e2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml"
Jul 27 19:26:13.148668 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:13.148663    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffwh7\" (UniqueName: \"kubernetes.io/projected/5af5df4d-1117-4c39-a915-f4e0567258e2-kube-api-access-ffwh7\") pod \"payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml\" (UID: \"5af5df4d-1117-4c39-a915-f4e0567258e2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml"
Jul 27 19:26:13.249845 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:13.249803    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5af5df4d-1117-4c39-a915-f4e0567258e2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml\" (UID: \"5af5df4d-1117-4c39-a915-f4e0567258e2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml"
Jul 27 19:26:13.249845 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:13.249842    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffwh7\" (UniqueName: \"kubernetes.io/projected/5af5df4d-1117-4c39-a915-f4e0567258e2-kube-api-access-ffwh7\") pod \"payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml\" (UID: \"5af5df4d-1117-4c39-a915-f4e0567258e2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml"
Jul 27 19:26:13.250591 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:13.250565    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5af5df4d-1117-4c39-a915-f4e0567258e2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml\" (UID: \"5af5df4d-1117-4c39-a915-f4e0567258e2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml"
Jul 27 19:26:13.267897 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:13.267839    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffwh7\" (UniqueName: \"kubernetes.io/projected/5af5df4d-1117-4c39-a915-f4e0567258e2-kube-api-access-ffwh7\") pod \"payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml\" (UID: \"5af5df4d-1117-4c39-a915-f4e0567258e2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml"
Jul 27 19:26:13.392896 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:13.392773    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml"
Jul 27 19:26:13.548889 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:13.548829    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml"]
Jul 27 19:26:13.552357 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:26:13.552322    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5af5df4d_1117_4c39_a915_f4e0567258e2.slice/crio-43c38b51ce7c6beaa670eb9134ee2b82836de1dd39a219f9b9ee4747cedebfcf WatchSource:0}: Error finding container 43c38b51ce7c6beaa670eb9134ee2b82836de1dd39a219f9b9ee4747cedebfcf: Status 404 returned error can't find the container with id 43c38b51ce7c6beaa670eb9134ee2b82836de1dd39a219f9b9ee4747cedebfcf
Jul 27 19:26:13.993159 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:13.993121    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml" event={"ID":"5af5df4d-1117-4c39-a915-f4e0567258e2","Type":"ContainerStarted","Data":"43c38b51ce7c6beaa670eb9134ee2b82836de1dd39a219f9b9ee4747cedebfcf"}
Jul 27 19:26:16.003293 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:16.003254    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml" event={"ID":"5af5df4d-1117-4c39-a915-f4e0567258e2","Type":"ContainerStarted","Data":"6684814a0ec3c7105711211ba1fe41a454cffb7a9e61eca20754d0f66822d7c5"}
Jul 27 19:26:16.003710 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:16.003329    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml"
Jul 27 19:26:24.045131 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:24.045081    2562 generic.go:356] "Generic (PLEG): container finished" podID="4da4ae70-3dbb-4016-a410-c393df6438f5" containerID="048854f198d810049a216efe8471ec503ed1f74f67297ebc871a3271e34e526b" exitCode=0
Jul 27 19:26:24.045645 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:24.045144    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g" event={"ID":"4da4ae70-3dbb-4016-a410-c393df6438f5","Type":"ContainerDied","Data":"048854f198d810049a216efe8471ec503ed1f74f67297ebc871a3271e34e526b"}
Jul 27 19:26:24.045645 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:24.045580    2562 scope.go:117] "RemoveContainer" containerID="048854f198d810049a216efe8471ec503ed1f74f67297ebc871a3271e34e526b"
Jul 27 19:26:24.046789 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:24.046761    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 27 19:26:24.046882 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:24.046834    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 27 19:26:24.046933 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:24.046887    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 27 19:26:24.066140 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:24.066087    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml" podStartSLOduration=9.589098613 podStartE2EDuration="11.066071127s" totalImagesPullingTime="1.476972514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:26:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:26:16.032217787 +0000 UTC m=+3284.582511420" watchObservedRunningTime="2026-07-27 19:26:24.066071127 +0000 UTC m=+3292.616364761"
Jul 27 19:26:25.052091 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:25.052052    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g" event={"ID":"4da4ae70-3dbb-4016-a410-c393df6438f5","Type":"ContainerStarted","Data":"e9cfe014c77f9abd28ea2df902c08b22c9a77456124e35b5c0a5f40e40f85265"}
Jul 27 19:26:25.979582 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:25.979533    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:26.034612 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:26.034574    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:26.035824 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:26.035795    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 27 19:26:26.035971 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:26.035847    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g" podUID="4da4ae70-3dbb-4016-a410-c393df6438f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 27 19:26:27.011637 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:27.011607    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml"
Jul 27 19:26:27.035184 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:27.035142    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 27 19:26:27.035378 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:27.035217    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g" podUID="4da4ae70-3dbb-4016-a410-c393df6438f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 27 19:26:28.035042 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:28.035005    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 27 19:26:28.035460 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:28.035071    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g" podUID="4da4ae70-3dbb-4016-a410-c393df6438f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 27 19:26:29.039022 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:29.038990    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:29.040097 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:29.040076    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:26:32.027421 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:32.027390    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 19:26:32.029211 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:32.029185    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 19:26:42.313083 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.313046    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"]
Jul 27 19:26:42.317795 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.317773    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:42.321418 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.321386    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-d4857a\"/\"ipp-route-d4857a-kserve-self-signed-certs\""
Jul 27 19:26:42.321418 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.321411    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-d4857a\"/\"openshift-service-ca.crt\""
Jul 27 19:26:42.321689 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.321430    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-d4857a\"/\"default-dockercfg-qqp6l\""
Jul 27 19:26:42.321689 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.321386    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-d4857a\"/\"kube-root-ca.crt\""
Jul 27 19:26:42.329841 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.329814    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"]
Jul 27 19:26:42.331373 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.331347    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-home\") pod \"ipp-route-d4857a-kserve-5fd6ddd64c-zg68g\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") " pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:42.331484 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.331384    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-tmp-dir\") pod \"ipp-route-d4857a-kserve-5fd6ddd64c-zg68g\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") " pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:42.331526 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.331480    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-dshm\") pod \"ipp-route-d4857a-kserve-5fd6ddd64c-zg68g\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") " pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:42.331526 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.331513    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-tls-certs\") pod \"ipp-route-d4857a-kserve-5fd6ddd64c-zg68g\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") " pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:42.331607 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.331594    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dhwf\" (UniqueName: \"kubernetes.io/projected/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-kube-api-access-6dhwf\") pod \"ipp-route-d4857a-kserve-5fd6ddd64c-zg68g\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") " pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:42.331648 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.331621    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-model-cache\") pod \"ipp-route-d4857a-kserve-5fd6ddd64c-zg68g\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") " pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:42.432525 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.432484    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dhwf\" (UniqueName: \"kubernetes.io/projected/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-kube-api-access-6dhwf\") pod \"ipp-route-d4857a-kserve-5fd6ddd64c-zg68g\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") " pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:42.432734 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.432535    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-model-cache\") pod \"ipp-route-d4857a-kserve-5fd6ddd64c-zg68g\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") " pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:42.432734 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.432566    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-home\") pod \"ipp-route-d4857a-kserve-5fd6ddd64c-zg68g\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") " pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:42.432734 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.432598    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-tmp-dir\") pod \"ipp-route-d4857a-kserve-5fd6ddd64c-zg68g\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") " pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:42.432734 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.432680    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-dshm\") pod \"ipp-route-d4857a-kserve-5fd6ddd64c-zg68g\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") " pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:42.432734 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.432701    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-tls-certs\") pod \"ipp-route-d4857a-kserve-5fd6ddd64c-zg68g\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") " pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:42.433080 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.433061    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-model-cache\") pod \"ipp-route-d4857a-kserve-5fd6ddd64c-zg68g\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") " pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:42.433143 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.433077    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-tmp-dir\") pod \"ipp-route-d4857a-kserve-5fd6ddd64c-zg68g\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") " pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:42.433208 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.433168    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-home\") pod \"ipp-route-d4857a-kserve-5fd6ddd64c-zg68g\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") " pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:42.435234 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.435211    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-dshm\") pod \"ipp-route-d4857a-kserve-5fd6ddd64c-zg68g\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") " pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:42.435467 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.435448    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-tls-certs\") pod \"ipp-route-d4857a-kserve-5fd6ddd64c-zg68g\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") " pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:42.441570 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.441520    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dhwf\" (UniqueName: \"kubernetes.io/projected/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-kube-api-access-6dhwf\") pod \"ipp-route-d4857a-kserve-5fd6ddd64c-zg68g\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") " pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:42.627192 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.627094    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:42.769651 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:42.769615    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"]
Jul 27 19:26:42.772059 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:26:42.772030    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2b60e3a_6b34_44fb_8fee_5ae220b00ad9.slice/crio-33f5a6e8ccee09b6fe4c414757fd20bb650b27d1e1cd1c2515f9d7d40b12dbd0 WatchSource:0}: Error finding container 33f5a6e8ccee09b6fe4c414757fd20bb650b27d1e1cd1c2515f9d7d40b12dbd0: Status 404 returned error can't find the container with id 33f5a6e8ccee09b6fe4c414757fd20bb650b27d1e1cd1c2515f9d7d40b12dbd0
Jul 27 19:26:43.133580 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:43.133545    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g" event={"ID":"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9","Type":"ContainerStarted","Data":"1db91bcbe787a326a7b0de1add7056bbe6dea2d25d7d6d0ebf16abc142953b93"}
Jul 27 19:26:43.133765 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:43.133587    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g" event={"ID":"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9","Type":"ContainerStarted","Data":"33f5a6e8ccee09b6fe4c414757fd20bb650b27d1e1cd1c2515f9d7d40b12dbd0"}
Jul 27 19:26:43.152816 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:43.152759    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g" podStartSLOduration=1.152743452 podStartE2EDuration="1.152743452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:26:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:26:43.150505408 +0000 UTC m=+3311.700799041" watchObservedRunningTime="2026-07-27 19:26:43.152743452 +0000 UTC m=+3311.703037085"
Jul 27 19:26:52.628022 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:52.627919    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:52.628022 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:52.627990    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:52.642893 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:52.642853    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:26:53.190717 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:26:53.190687    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:27:30.228746 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.228706    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd"]
Jul 27 19:27:30.232430 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.232409    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd"
Jul 27 19:27:30.234916 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.234890    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-8eed25\""
Jul 27 19:27:30.235061 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.234947    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-8eed25-dockercfg-mvhjn\""
Jul 27 19:27:30.244335 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.244310    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd"]
Jul 27 19:27:30.281200 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.281157    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67489f46-eb45-4a57-8c90-b6b7a5b9fb2a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd\" (UID: \"67489f46-eb45-4a57-8c90-b6b7a5b9fb2a\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd"
Jul 27 19:27:30.281375 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.281286    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knsls\" (UniqueName: \"kubernetes.io/projected/67489f46-eb45-4a57-8c90-b6b7a5b9fb2a-kube-api-access-knsls\") pod \"payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd\" (UID: \"67489f46-eb45-4a57-8c90-b6b7a5b9fb2a\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd"
Jul 27 19:27:30.282730 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.282710    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v"]
Jul 27 19:27:30.286265 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.286249    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v"
Jul 27 19:27:30.296519 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.296489    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v"]
Jul 27 19:27:30.382267 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.382222    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgdx7\" (UniqueName: \"kubernetes.io/projected/1f22d854-a592-4542-9c34-6c3f85659464-kube-api-access-bgdx7\") pod \"payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v\" (UID: \"1f22d854-a592-4542-9c34-6c3f85659464\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v"
Jul 27 19:27:30.382267 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.382269    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67489f46-eb45-4a57-8c90-b6b7a5b9fb2a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd\" (UID: \"67489f46-eb45-4a57-8c90-b6b7a5b9fb2a\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd"
Jul 27 19:27:30.382519 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.382350    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f22d854-a592-4542-9c34-6c3f85659464-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v\" (UID: \"1f22d854-a592-4542-9c34-6c3f85659464\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v"
Jul 27 19:27:30.382519 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.382371    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knsls\" (UniqueName: \"kubernetes.io/projected/67489f46-eb45-4a57-8c90-b6b7a5b9fb2a-kube-api-access-knsls\") pod \"payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd\" (UID: \"67489f46-eb45-4a57-8c90-b6b7a5b9fb2a\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd"
Jul 27 19:27:30.383028 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.383007    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67489f46-eb45-4a57-8c90-b6b7a5b9fb2a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd\" (UID: \"67489f46-eb45-4a57-8c90-b6b7a5b9fb2a\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd"
Jul 27 19:27:30.391470 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.391437    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knsls\" (UniqueName: \"kubernetes.io/projected/67489f46-eb45-4a57-8c90-b6b7a5b9fb2a-kube-api-access-knsls\") pod \"payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd\" (UID: \"67489f46-eb45-4a57-8c90-b6b7a5b9fb2a\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd"
Jul 27 19:27:30.483288 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.483186    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgdx7\" (UniqueName: \"kubernetes.io/projected/1f22d854-a592-4542-9c34-6c3f85659464-kube-api-access-bgdx7\") pod \"payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v\" (UID: \"1f22d854-a592-4542-9c34-6c3f85659464\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v"
Jul 27 19:27:30.483455 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.483301    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f22d854-a592-4542-9c34-6c3f85659464-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v\" (UID: \"1f22d854-a592-4542-9c34-6c3f85659464\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v"
Jul 27 19:27:30.484041 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.484022    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f22d854-a592-4542-9c34-6c3f85659464-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v\" (UID: \"1f22d854-a592-4542-9c34-6c3f85659464\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v"
Jul 27 19:27:30.494500 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.494465    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgdx7\" (UniqueName: \"kubernetes.io/projected/1f22d854-a592-4542-9c34-6c3f85659464-kube-api-access-bgdx7\") pod \"payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v\" (UID: \"1f22d854-a592-4542-9c34-6c3f85659464\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v"
Jul 27 19:27:30.544527 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.544483    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd"
Jul 27 19:27:30.596810 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.596780    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v"
Jul 27 19:27:30.698534 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.698479    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd"]
Jul 27 19:27:30.702745 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:27:30.702713    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67489f46_eb45_4a57_8c90_b6b7a5b9fb2a.slice/crio-a71e0b07e0d83e712f907897308cd2b45c74642d43a651971864da81b86646dc WatchSource:0}: Error finding container a71e0b07e0d83e712f907897308cd2b45c74642d43a651971864da81b86646dc: Status 404 returned error can't find the container with id a71e0b07e0d83e712f907897308cd2b45c74642d43a651971864da81b86646dc
Jul 27 19:27:30.751355 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:30.751330    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v"]
Jul 27 19:27:30.756064 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:27:30.756031    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f22d854_a592_4542_9c34_6c3f85659464.slice/crio-2be57582b4826895f8de96f4a25382e8c233284d0391c8f2d9cbbe5802838490 WatchSource:0}: Error finding container 2be57582b4826895f8de96f4a25382e8c233284d0391c8f2d9cbbe5802838490: Status 404 returned error can't find the container with id 2be57582b4826895f8de96f4a25382e8c233284d0391c8f2d9cbbe5802838490
Jul 27 19:27:31.364744 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:31.364704    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd" event={"ID":"67489f46-eb45-4a57-8c90-b6b7a5b9fb2a","Type":"ContainerStarted","Data":"3bb5522e5f94b72bf985069ce082dca0d1028e4e7eec0a9dd508dac0f574b7b2"}
Jul 27 19:27:31.365239 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:31.364800    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd" event={"ID":"67489f46-eb45-4a57-8c90-b6b7a5b9fb2a","Type":"ContainerStarted","Data":"a71e0b07e0d83e712f907897308cd2b45c74642d43a651971864da81b86646dc"}
Jul 27 19:27:31.365239 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:31.364831    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd"
Jul 27 19:27:31.366343 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:31.366315    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v" event={"ID":"1f22d854-a592-4542-9c34-6c3f85659464","Type":"ContainerStarted","Data":"1ffd57afac7c0b9ba0e827ef031b248076caca765d13e123e5821b27037de68a"}
Jul 27 19:27:31.366478 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:31.366365    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v" event={"ID":"1f22d854-a592-4542-9c34-6c3f85659464","Type":"ContainerStarted","Data":"2be57582b4826895f8de96f4a25382e8c233284d0391c8f2d9cbbe5802838490"}
Jul 27 19:27:31.366478 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:31.366404    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v"
Jul 27 19:27:32.531498 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:32.531434    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd" podStartSLOduration=2.531413326 podStartE2EDuration="2.531413326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:32.528594859 +0000 UTC m=+3361.078888491" watchObservedRunningTime="2026-07-27 19:27:32.531413326 +0000 UTC m=+3361.081706964"
Jul 27 19:27:32.556581 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:32.556520    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v" podStartSLOduration=2.5565020240000003 podStartE2EDuration="2.556502024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:32.555345283 +0000 UTC m=+3361.105638917" watchObservedRunningTime="2026-07-27 19:27:32.556502024 +0000 UTC m=+3361.106795657"
Jul 27 19:27:42.374171 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:42.374134    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd"
Jul 27 19:27:42.374667 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:42.374215    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v"
Jul 27 19:27:54.145903 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.136983    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd"]
Jul 27 19:27:54.145903 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.137339    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd" podUID="67489f46-eb45-4a57-8c90-b6b7a5b9fb2a" containerName="payload-processing" containerID="cri-o://3bb5522e5f94b72bf985069ce082dca0d1028e4e7eec0a9dd508dac0f574b7b2" gracePeriod=30
Jul 27 19:27:54.151132 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.151095    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v"]
Jul 27 19:27:54.151710 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.151385    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v" podUID="1f22d854-a592-4542-9c34-6c3f85659464" containerName="payload-pre-processing" containerID="cri-o://1ffd57afac7c0b9ba0e827ef031b248076caca765d13e123e5821b27037de68a" gracePeriod=30
Jul 27 19:27:54.418202 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.418177    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd"
Jul 27 19:27:54.429849 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.429820    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v"
Jul 27 19:27:54.479209 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.479175    2562 generic.go:356] "Generic (PLEG): container finished" podID="67489f46-eb45-4a57-8c90-b6b7a5b9fb2a" containerID="3bb5522e5f94b72bf985069ce082dca0d1028e4e7eec0a9dd508dac0f574b7b2" exitCode=0
Jul 27 19:27:54.479414 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.479220    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd"
Jul 27 19:27:54.479414 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.479265    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd" event={"ID":"67489f46-eb45-4a57-8c90-b6b7a5b9fb2a","Type":"ContainerDied","Data":"3bb5522e5f94b72bf985069ce082dca0d1028e4e7eec0a9dd508dac0f574b7b2"}
Jul 27 19:27:54.479414 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.479324    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd" event={"ID":"67489f46-eb45-4a57-8c90-b6b7a5b9fb2a","Type":"ContainerDied","Data":"a71e0b07e0d83e712f907897308cd2b45c74642d43a651971864da81b86646dc"}
Jul 27 19:27:54.479414 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.479357    2562 scope.go:117] "RemoveContainer" containerID="3bb5522e5f94b72bf985069ce082dca0d1028e4e7eec0a9dd508dac0f574b7b2"
Jul 27 19:27:54.480575 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.480553    2562 generic.go:356] "Generic (PLEG): container finished" podID="1f22d854-a592-4542-9c34-6c3f85659464" containerID="1ffd57afac7c0b9ba0e827ef031b248076caca765d13e123e5821b27037de68a" exitCode=0
Jul 27 19:27:54.480665 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.480600    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v"
Jul 27 19:27:54.480665 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.480621    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v" event={"ID":"1f22d854-a592-4542-9c34-6c3f85659464","Type":"ContainerDied","Data":"1ffd57afac7c0b9ba0e827ef031b248076caca765d13e123e5821b27037de68a"}
Jul 27 19:27:54.480775 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.480678    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v" event={"ID":"1f22d854-a592-4542-9c34-6c3f85659464","Type":"ContainerDied","Data":"2be57582b4826895f8de96f4a25382e8c233284d0391c8f2d9cbbe5802838490"}
Jul 27 19:27:54.488471 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.488449    2562 scope.go:117] "RemoveContainer" containerID="3bb5522e5f94b72bf985069ce082dca0d1028e4e7eec0a9dd508dac0f574b7b2"
Jul 27 19:27:54.488795 ip-10-0-132-186 kubenswrapper[2562]: E0727 19:27:54.488776    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bb5522e5f94b72bf985069ce082dca0d1028e4e7eec0a9dd508dac0f574b7b2\": container with ID starting with 3bb5522e5f94b72bf985069ce082dca0d1028e4e7eec0a9dd508dac0f574b7b2 not found: ID does not exist" containerID="3bb5522e5f94b72bf985069ce082dca0d1028e4e7eec0a9dd508dac0f574b7b2"
Jul 27 19:27:54.488883 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.488809    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bb5522e5f94b72bf985069ce082dca0d1028e4e7eec0a9dd508dac0f574b7b2"} err="failed to get container status \"3bb5522e5f94b72bf985069ce082dca0d1028e4e7eec0a9dd508dac0f574b7b2\": rpc error: code = NotFound desc = could not find container \"3bb5522e5f94b72bf985069ce082dca0d1028e4e7eec0a9dd508dac0f574b7b2\": container with ID starting with 3bb5522e5f94b72bf985069ce082dca0d1028e4e7eec0a9dd508dac0f574b7b2 not found: ID does not exist"
Jul 27 19:27:54.488883 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.488834    2562 scope.go:117] "RemoveContainer" containerID="1ffd57afac7c0b9ba0e827ef031b248076caca765d13e123e5821b27037de68a"
Jul 27 19:27:54.497260 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.497239    2562 scope.go:117] "RemoveContainer" containerID="1ffd57afac7c0b9ba0e827ef031b248076caca765d13e123e5821b27037de68a"
Jul 27 19:27:54.497575 ip-10-0-132-186 kubenswrapper[2562]: E0727 19:27:54.497552    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ffd57afac7c0b9ba0e827ef031b248076caca765d13e123e5821b27037de68a\": container with ID starting with 1ffd57afac7c0b9ba0e827ef031b248076caca765d13e123e5821b27037de68a not found: ID does not exist" containerID="1ffd57afac7c0b9ba0e827ef031b248076caca765d13e123e5821b27037de68a"
Jul 27 19:27:54.497659 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.497588    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ffd57afac7c0b9ba0e827ef031b248076caca765d13e123e5821b27037de68a"} err="failed to get container status \"1ffd57afac7c0b9ba0e827ef031b248076caca765d13e123e5821b27037de68a\": rpc error: code = NotFound desc = could not find container \"1ffd57afac7c0b9ba0e827ef031b248076caca765d13e123e5821b27037de68a\": container with ID starting with 1ffd57afac7c0b9ba0e827ef031b248076caca765d13e123e5821b27037de68a not found: ID does not exist"
Jul 27 19:27:54.504948 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.504921    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67489f46-eb45-4a57-8c90-b6b7a5b9fb2a-plugins-config-volume\") pod \"67489f46-eb45-4a57-8c90-b6b7a5b9fb2a\" (UID: \"67489f46-eb45-4a57-8c90-b6b7a5b9fb2a\") "
Jul 27 19:27:54.505063 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.504982    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knsls\" (UniqueName: \"kubernetes.io/projected/67489f46-eb45-4a57-8c90-b6b7a5b9fb2a-kube-api-access-knsls\") pod \"67489f46-eb45-4a57-8c90-b6b7a5b9fb2a\" (UID: \"67489f46-eb45-4a57-8c90-b6b7a5b9fb2a\") "
Jul 27 19:27:54.505292 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.505270    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67489f46-eb45-4a57-8c90-b6b7a5b9fb2a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "67489f46-eb45-4a57-8c90-b6b7a5b9fb2a" (UID: "67489f46-eb45-4a57-8c90-b6b7a5b9fb2a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:27:54.507329 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.507303    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67489f46-eb45-4a57-8c90-b6b7a5b9fb2a-kube-api-access-knsls" (OuterVolumeSpecName: "kube-api-access-knsls") pod "67489f46-eb45-4a57-8c90-b6b7a5b9fb2a" (UID: "67489f46-eb45-4a57-8c90-b6b7a5b9fb2a"). InnerVolumeSpecName "kube-api-access-knsls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:27:54.606481 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.606440    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f22d854-a592-4542-9c34-6c3f85659464-plugins-config-volume\") pod \"1f22d854-a592-4542-9c34-6c3f85659464\" (UID: \"1f22d854-a592-4542-9c34-6c3f85659464\") "
Jul 27 19:27:54.606481 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.606484    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgdx7\" (UniqueName: \"kubernetes.io/projected/1f22d854-a592-4542-9c34-6c3f85659464-kube-api-access-bgdx7\") pod \"1f22d854-a592-4542-9c34-6c3f85659464\" (UID: \"1f22d854-a592-4542-9c34-6c3f85659464\") "
Jul 27 19:27:54.606755 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.606686    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knsls\" (UniqueName: \"kubernetes.io/projected/67489f46-eb45-4a57-8c90-b6b7a5b9fb2a-kube-api-access-knsls\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:27:54.606755 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.606699    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67489f46-eb45-4a57-8c90-b6b7a5b9fb2a-plugins-config-volume\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:27:54.606948 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.606922    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f22d854-a592-4542-9c34-6c3f85659464-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1f22d854-a592-4542-9c34-6c3f85659464" (UID: "1f22d854-a592-4542-9c34-6c3f85659464"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:27:54.608765 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.608741    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f22d854-a592-4542-9c34-6c3f85659464-kube-api-access-bgdx7" (OuterVolumeSpecName: "kube-api-access-bgdx7") pod "1f22d854-a592-4542-9c34-6c3f85659464" (UID: "1f22d854-a592-4542-9c34-6c3f85659464"). InnerVolumeSpecName "kube-api-access-bgdx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:27:54.707398 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.707365    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f22d854-a592-4542-9c34-6c3f85659464-plugins-config-volume\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:27:54.707398 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.707396    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgdx7\" (UniqueName: \"kubernetes.io/projected/1f22d854-a592-4542-9c34-6c3f85659464-kube-api-access-bgdx7\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:27:54.803231 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.803189    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd"]
Jul 27 19:27:54.807939 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.807903    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8eed25-86c8c54558-jfwgd"]
Jul 27 19:27:54.817627 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.817594    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v"]
Jul 27 19:27:54.822047 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:54.822008    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8eed25-65cc966f96-hsj4v"]
Jul 27 19:27:55.937035 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:55.937003    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f22d854-a592-4542-9c34-6c3f85659464" path="/var/lib/kubelet/pods/1f22d854-a592-4542-9c34-6c3f85659464/volumes"
Jul 27 19:27:55.937467 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:27:55.937377    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67489f46-eb45-4a57-8c90-b6b7a5b9fb2a" path="/var/lib/kubelet/pods/67489f46-eb45-4a57-8c90-b6b7a5b9fb2a/volumes"
Jul 27 19:28:35.500236 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:35.500199    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"]
Jul 27 19:28:50.519633 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.519580    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g" podUID="c2b60e3a-6b34-44fb-8fee-5ae220b00ad9" containerName="main" containerID="cri-o://1db91bcbe787a326a7b0de1add7056bbe6dea2d25d7d6d0ebf16abc142953b93" gracePeriod=45
Jul 27 19:28:50.734804 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.734762    2562 generic.go:356] "Generic (PLEG): container finished" podID="c2b60e3a-6b34-44fb-8fee-5ae220b00ad9" containerID="1db91bcbe787a326a7b0de1add7056bbe6dea2d25d7d6d0ebf16abc142953b93" exitCode=0
Jul 27 19:28:50.734991 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.734834    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g" event={"ID":"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9","Type":"ContainerDied","Data":"1db91bcbe787a326a7b0de1add7056bbe6dea2d25d7d6d0ebf16abc142953b93"}
Jul 27 19:28:50.765559 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.765533    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:28:50.812933 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.812832    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-tls-certs\") pod \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") "
Jul 27 19:28:50.812933 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.812892    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dhwf\" (UniqueName: \"kubernetes.io/projected/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-kube-api-access-6dhwf\") pod \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") "
Jul 27 19:28:50.812933 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.812926    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-home\") pod \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") "
Jul 27 19:28:50.813219 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.812942    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-tmp-dir\") pod \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") "
Jul 27 19:28:50.813219 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.812969    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-dshm\") pod \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") "
Jul 27 19:28:50.813219 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.812993    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-model-cache\") pod \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\" (UID: \"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9\") "
Jul 27 19:28:50.813391 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.813331    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-home" (OuterVolumeSpecName: "home") pod "c2b60e3a-6b34-44fb-8fee-5ae220b00ad9" (UID: "c2b60e3a-6b34-44fb-8fee-5ae220b00ad9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:28:50.813536 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.813500    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-model-cache" (OuterVolumeSpecName: "model-cache") pod "c2b60e3a-6b34-44fb-8fee-5ae220b00ad9" (UID: "c2b60e3a-6b34-44fb-8fee-5ae220b00ad9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:28:50.813683 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.813574    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c2b60e3a-6b34-44fb-8fee-5ae220b00ad9" (UID: "c2b60e3a-6b34-44fb-8fee-5ae220b00ad9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:28:50.815446 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.815415    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-dshm" (OuterVolumeSpecName: "dshm") pod "c2b60e3a-6b34-44fb-8fee-5ae220b00ad9" (UID: "c2b60e3a-6b34-44fb-8fee-5ae220b00ad9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:28:50.815575 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.815506    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c2b60e3a-6b34-44fb-8fee-5ae220b00ad9" (UID: "c2b60e3a-6b34-44fb-8fee-5ae220b00ad9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:28:50.815688 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.815645    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-kube-api-access-6dhwf" (OuterVolumeSpecName: "kube-api-access-6dhwf") pod "c2b60e3a-6b34-44fb-8fee-5ae220b00ad9" (UID: "c2b60e3a-6b34-44fb-8fee-5ae220b00ad9"). InnerVolumeSpecName "kube-api-access-6dhwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:28:50.914498 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.914464    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-tls-certs\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:28:50.914498 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.914495    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dhwf\" (UniqueName: \"kubernetes.io/projected/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-kube-api-access-6dhwf\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:28:50.914498 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.914505    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-home\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:28:50.914748 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.914515    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-tmp-dir\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:28:50.914748 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.914523    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-dshm\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:28:50.914748 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:50.914531    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9-model-cache\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:28:51.740547 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:51.740508    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"
Jul 27 19:28:51.741037 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:51.740543    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g" event={"ID":"c2b60e3a-6b34-44fb-8fee-5ae220b00ad9","Type":"ContainerDied","Data":"33f5a6e8ccee09b6fe4c414757fd20bb650b27d1e1cd1c2515f9d7d40b12dbd0"}
Jul 27 19:28:51.741037 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:51.740594    2562 scope.go:117] "RemoveContainer" containerID="1db91bcbe787a326a7b0de1add7056bbe6dea2d25d7d6d0ebf16abc142953b93"
Jul 27 19:28:51.764303 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:51.764264    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"]
Jul 27 19:28:51.771263 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:51.771232    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-d4857a/ipp-route-d4857a-kserve-5fd6ddd64c-zg68g"]
Jul 27 19:28:51.936829 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:28:51.936793    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2b60e3a-6b34-44fb-8fee-5ae220b00ad9" path="/var/lib/kubelet/pods/c2b60e3a-6b34-44fb-8fee-5ae220b00ad9/volumes"
Jul 27 19:29:35.601114 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:35.601079    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"]
Jul 27 19:29:35.601635 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:35.601354    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8" podUID="444e89a7-263a-40cc-b844-88860c4bdda1" containerName="istio-proxy" containerID="cri-o://53182ee3426c2d9adad8eeba3073e7c87aeda69ac11902c04252cde292ceda54" gracePeriod=30
Jul 27 19:29:35.945432 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:35.945340    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2d7b97-z6hjw"]
Jul 27 19:29:35.945672 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:35.945659    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67489f46-eb45-4a57-8c90-b6b7a5b9fb2a" containerName="payload-processing"
Jul 27 19:29:35.945715 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:35.945674    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="67489f46-eb45-4a57-8c90-b6b7a5b9fb2a" containerName="payload-processing"
Jul 27 19:29:35.945715 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:35.945683    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f22d854-a592-4542-9c34-6c3f85659464" containerName="payload-pre-processing"
Jul 27 19:29:35.945715 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:35.945689    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f22d854-a592-4542-9c34-6c3f85659464" containerName="payload-pre-processing"
Jul 27 19:29:35.945715 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:35.945704    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2b60e3a-6b34-44fb-8fee-5ae220b00ad9" containerName="main"
Jul 27 19:29:35.945715 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:35.945709    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2b60e3a-6b34-44fb-8fee-5ae220b00ad9" containerName="main"
Jul 27 19:29:35.945894 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:35.945757    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2b60e3a-6b34-44fb-8fee-5ae220b00ad9" containerName="main"
Jul 27 19:29:35.945894 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:35.945766    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f22d854-a592-4542-9c34-6c3f85659464" containerName="payload-pre-processing"
Jul 27 19:29:35.945894 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:35.945772    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="67489f46-eb45-4a57-8c90-b6b7a5b9fb2a" containerName="payload-processing"
Jul 27 19:29:35.949959 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:35.949941    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2d7b97-z6hjw"
Jul 27 19:29:35.952271 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:35.952245    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 19:29:35.952492 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:35.952474    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gwtlm\""
Jul 27 19:29:35.955639 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:35.955614    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2d7b97-z6hjw"]
Jul 27 19:29:36.024523 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:36.024482    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3099acb4-7e58-484a-a01a-c8902587363c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-2d7b97-z6hjw\" (UID: \"3099acb4-7e58-484a-a01a-c8902587363c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2d7b97-z6hjw"
Jul 27 19:29:36.125730 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:36.125689    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3099acb4-7e58-484a-a01a-c8902587363c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-2d7b97-z6hjw\" (UID: \"3099acb4-7e58-484a-a01a-c8902587363c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2d7b97-z6hjw"
Jul 27 19:29:36.126373 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:36.126350    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3099acb4-7e58-484a-a01a-c8902587363c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-2d7b97-z6hjw\" (UID: \"3099acb4-7e58-484a-a01a-c8902587363c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2d7b97-z6hjw"
Jul 27 19:29:36.262027 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:36.261992    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2d7b97-z6hjw"
Jul 27 19:29:36.404940 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:36.404856    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2d7b97-z6hjw"]
Jul 27 19:29:36.407309 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:29:36.407277    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3099acb4_7e58_484a_a01a_c8902587363c.slice/crio-c00096697fcd9c58eb2e97c07e96b51e2abc7ba3d0b5d8536cbd91c2a4fdd31f WatchSource:0}: Error finding container c00096697fcd9c58eb2e97c07e96b51e2abc7ba3d0b5d8536cbd91c2a4fdd31f: Status 404 returned error can't find the container with id c00096697fcd9c58eb2e97c07e96b51e2abc7ba3d0b5d8536cbd91c2a4fdd31f
Jul 27 19:29:36.944747 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:36.944646    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2d7b97-z6hjw" event={"ID":"3099acb4-7e58-484a-a01a-c8902587363c","Type":"ContainerStarted","Data":"4b845b37af2a7c7a3e14937fb08804dc4322c1d14be67fa41d7d01dbcb99eb99"}
Jul 27 19:29:36.944747 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:36.944704    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2d7b97-z6hjw" event={"ID":"3099acb4-7e58-484a-a01a-c8902587363c","Type":"ContainerStarted","Data":"c00096697fcd9c58eb2e97c07e96b51e2abc7ba3d0b5d8536cbd91c2a4fdd31f"}
Jul 27 19:29:37.274170 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:37.274109    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2d7b97-z6hjw" podStartSLOduration=2.274092773 podStartE2EDuration="2.274092773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:29:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:37.272542093 +0000 UTC m=+3485.822835725" watchObservedRunningTime="2026-07-27 19:29:37.274092773 +0000 UTC m=+3485.824386406"
Jul 27 19:29:37.950437 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:37.950401    2562 generic.go:356] "Generic (PLEG): container finished" podID="3099acb4-7e58-484a-a01a-c8902587363c" containerID="4b845b37af2a7c7a3e14937fb08804dc4322c1d14be67fa41d7d01dbcb99eb99" exitCode=0
Jul 27 19:29:37.950956 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:37.950499    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2d7b97-z6hjw" event={"ID":"3099acb4-7e58-484a-a01a-c8902587363c","Type":"ContainerDied","Data":"4b845b37af2a7c7a3e14937fb08804dc4322c1d14be67fa41d7d01dbcb99eb99"}
Jul 27 19:29:39.094598 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:39.094566    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2d7b97-z6hjw"
Jul 27 19:29:39.150542 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:39.150510    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3099acb4-7e58-484a-a01a-c8902587363c-maas-api-service-ca\") pod \"3099acb4-7e58-484a-a01a-c8902587363c\" (UID: \"3099acb4-7e58-484a-a01a-c8902587363c\") "
Jul 27 19:29:39.150923 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:39.150903    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3099acb4-7e58-484a-a01a-c8902587363c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3099acb4-7e58-484a-a01a-c8902587363c" (UID: "3099acb4-7e58-484a-a01a-c8902587363c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:29:39.251424 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:39.251330    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3099acb4-7e58-484a-a01a-c8902587363c-maas-api-service-ca\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:29:39.960969 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:39.960940    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2d7b97-z6hjw"
Jul 27 19:29:39.960969 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:39.960965    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2d7b97-z6hjw" event={"ID":"3099acb4-7e58-484a-a01a-c8902587363c","Type":"ContainerDied","Data":"c00096697fcd9c58eb2e97c07e96b51e2abc7ba3d0b5d8536cbd91c2a4fdd31f"}
Jul 27 19:29:39.961228 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:39.961024    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c00096697fcd9c58eb2e97c07e96b51e2abc7ba3d0b5d8536cbd91c2a4fdd31f"
Jul 27 19:29:40.859168 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.859143    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:29:40.862161 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.862137    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-workload-socket\") pod \"444e89a7-263a-40cc-b844-88860c4bdda1\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") "
Jul 27 19:29:40.862307 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.862169    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/444e89a7-263a-40cc-b844-88860c4bdda1-istiod-ca-cert\") pod \"444e89a7-263a-40cc-b844-88860c4bdda1\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") "
Jul 27 19:29:40.862307 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.862203    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/444e89a7-263a-40cc-b844-88860c4bdda1-istio-token\") pod \"444e89a7-263a-40cc-b844-88860c4bdda1\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") "
Jul 27 19:29:40.862307 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.862226    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-workload-certs\") pod \"444e89a7-263a-40cc-b844-88860c4bdda1\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") "
Jul 27 19:29:40.862307 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.862244    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/444e89a7-263a-40cc-b844-88860c4bdda1-istio-podinfo\") pod \"444e89a7-263a-40cc-b844-88860c4bdda1\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") "
Jul 27 19:29:40.862307 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.862274    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-istio-data\") pod \"444e89a7-263a-40cc-b844-88860c4bdda1\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") "
Jul 27 19:29:40.862557 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.862354    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-istio-envoy\") pod \"444e89a7-263a-40cc-b844-88860c4bdda1\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") "
Jul 27 19:29:40.862557 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.862404    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-credential-socket\") pod \"444e89a7-263a-40cc-b844-88860c4bdda1\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") "
Jul 27 19:29:40.862557 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.862415    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "444e89a7-263a-40cc-b844-88860c4bdda1" (UID: "444e89a7-263a-40cc-b844-88860c4bdda1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:29:40.862557 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.862451    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rq667\" (UniqueName: \"kubernetes.io/projected/444e89a7-263a-40cc-b844-88860c4bdda1-kube-api-access-rq667\") pod \"444e89a7-263a-40cc-b844-88860c4bdda1\" (UID: \"444e89a7-263a-40cc-b844-88860c4bdda1\") "
Jul 27 19:29:40.862761 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.862556    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "444e89a7-263a-40cc-b844-88860c4bdda1" (UID: "444e89a7-263a-40cc-b844-88860c4bdda1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:29:40.862761 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.862705    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "444e89a7-263a-40cc-b844-88860c4bdda1" (UID: "444e89a7-263a-40cc-b844-88860c4bdda1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:29:40.862761 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.862716    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/444e89a7-263a-40cc-b844-88860c4bdda1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "444e89a7-263a-40cc-b844-88860c4bdda1" (UID: "444e89a7-263a-40cc-b844-88860c4bdda1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:29:40.862938 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.862833    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-workload-socket\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:29:40.862938 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.862852    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/444e89a7-263a-40cc-b844-88860c4bdda1-istiod-ca-cert\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:29:40.862938 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.862891    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-workload-certs\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:29:40.862938 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.862906    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-credential-socket\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:29:40.863165 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.863142    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-istio-data" (OuterVolumeSpecName: "istio-data") pod "444e89a7-263a-40cc-b844-88860c4bdda1" (UID: "444e89a7-263a-40cc-b844-88860c4bdda1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:29:40.864983 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.864918    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/444e89a7-263a-40cc-b844-88860c4bdda1-kube-api-access-rq667" (OuterVolumeSpecName: "kube-api-access-rq667") pod "444e89a7-263a-40cc-b844-88860c4bdda1" (UID: "444e89a7-263a-40cc-b844-88860c4bdda1"). InnerVolumeSpecName "kube-api-access-rq667". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:29:40.865093 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.865008    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "444e89a7-263a-40cc-b844-88860c4bdda1" (UID: "444e89a7-263a-40cc-b844-88860c4bdda1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:29:40.865140 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.865103    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/444e89a7-263a-40cc-b844-88860c4bdda1-istio-token" (OuterVolumeSpecName: "istio-token") pod "444e89a7-263a-40cc-b844-88860c4bdda1" (UID: "444e89a7-263a-40cc-b844-88860c4bdda1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:29:40.865140 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.865097    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/444e89a7-263a-40cc-b844-88860c4bdda1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "444e89a7-263a-40cc-b844-88860c4bdda1" (UID: "444e89a7-263a-40cc-b844-88860c4bdda1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 19:29:40.964102 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.964047    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/444e89a7-263a-40cc-b844-88860c4bdda1-istio-token\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:29:40.964102 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.964082    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/444e89a7-263a-40cc-b844-88860c4bdda1-istio-podinfo\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:29:40.964102 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.964094    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-istio-data\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:29:40.964102 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.964106    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/444e89a7-263a-40cc-b844-88860c4bdda1-istio-envoy\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:29:40.964442 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.964118    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rq667\" (UniqueName: \"kubernetes.io/projected/444e89a7-263a-40cc-b844-88860c4bdda1-kube-api-access-rq667\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:29:40.971845 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.970983    2562 generic.go:356] "Generic (PLEG): container finished" podID="444e89a7-263a-40cc-b844-88860c4bdda1" containerID="53182ee3426c2d9adad8eeba3073e7c87aeda69ac11902c04252cde292ceda54" exitCode=0
Jul 27 19:29:40.971845 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.971126    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8" event={"ID":"444e89a7-263a-40cc-b844-88860c4bdda1","Type":"ContainerDied","Data":"53182ee3426c2d9adad8eeba3073e7c87aeda69ac11902c04252cde292ceda54"}
Jul 27 19:29:40.971845 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.971220    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8" event={"ID":"444e89a7-263a-40cc-b844-88860c4bdda1","Type":"ContainerDied","Data":"eb864a36f7f0f0f6ea25c452b48879b492c6de258abebe0439ff6849d2006065"}
Jul 27 19:29:40.971845 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.971259    2562 scope.go:117] "RemoveContainer" containerID="53182ee3426c2d9adad8eeba3073e7c87aeda69ac11902c04252cde292ceda54"
Jul 27 19:29:40.971845 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.971470    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"
Jul 27 19:29:40.982275 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.982255    2562 scope.go:117] "RemoveContainer" containerID="8b38eb48b3849ebcaac5cc26944876a3f34a00ed5cf3a9f3f400556b019b1633"
Jul 27 19:29:40.991330 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.991303    2562 scope.go:117] "RemoveContainer" containerID="53182ee3426c2d9adad8eeba3073e7c87aeda69ac11902c04252cde292ceda54"
Jul 27 19:29:40.991638 ip-10-0-132-186 kubenswrapper[2562]: E0727 19:29:40.991614    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53182ee3426c2d9adad8eeba3073e7c87aeda69ac11902c04252cde292ceda54\": container with ID starting with 53182ee3426c2d9adad8eeba3073e7c87aeda69ac11902c04252cde292ceda54 not found: ID does not exist" containerID="53182ee3426c2d9adad8eeba3073e7c87aeda69ac11902c04252cde292ceda54"
Jul 27 19:29:40.991713 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.991656    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53182ee3426c2d9adad8eeba3073e7c87aeda69ac11902c04252cde292ceda54"} err="failed to get container status \"53182ee3426c2d9adad8eeba3073e7c87aeda69ac11902c04252cde292ceda54\": rpc error: code = NotFound desc = could not find container \"53182ee3426c2d9adad8eeba3073e7c87aeda69ac11902c04252cde292ceda54\": container with ID starting with 53182ee3426c2d9adad8eeba3073e7c87aeda69ac11902c04252cde292ceda54 not found: ID does not exist"
Jul 27 19:29:40.991713 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.991684    2562 scope.go:117] "RemoveContainer" containerID="8b38eb48b3849ebcaac5cc26944876a3f34a00ed5cf3a9f3f400556b019b1633"
Jul 27 19:29:40.991969 ip-10-0-132-186 kubenswrapper[2562]: E0727 19:29:40.991950    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b38eb48b3849ebcaac5cc26944876a3f34a00ed5cf3a9f3f400556b019b1633\": container with ID starting with 8b38eb48b3849ebcaac5cc26944876a3f34a00ed5cf3a9f3f400556b019b1633 not found: ID does not exist" containerID="8b38eb48b3849ebcaac5cc26944876a3f34a00ed5cf3a9f3f400556b019b1633"
Jul 27 19:29:40.992016 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.991975    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b38eb48b3849ebcaac5cc26944876a3f34a00ed5cf3a9f3f400556b019b1633"} err="failed to get container status \"8b38eb48b3849ebcaac5cc26944876a3f34a00ed5cf3a9f3f400556b019b1633\": rpc error: code = NotFound desc = could not find container \"8b38eb48b3849ebcaac5cc26944876a3f34a00ed5cf3a9f3f400556b019b1633\": container with ID starting with 8b38eb48b3849ebcaac5cc26944876a3f34a00ed5cf3a9f3f400556b019b1633 not found: ID does not exist"
Jul 27 19:29:40.994502 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.994464    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"]
Jul 27 19:29:40.997196 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:40.997175    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-d4857a-openshift-default-6d455bc56d-l5vv8"]
Jul 27 19:29:41.937503 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:41.937472    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="444e89a7-263a-40cc-b844-88860c4bdda1" path="/var/lib/kubelet/pods/444e89a7-263a-40cc-b844-88860c4bdda1/volumes"
Jul 27 19:29:47.584703 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:47.584659    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml"]
Jul 27 19:29:47.585285 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:47.584967    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml" podUID="5af5df4d-1117-4c39-a915-f4e0567258e2" containerName="payload-pre-processing" containerID="cri-o://6684814a0ec3c7105711211ba1fe41a454cffb7a9e61eca20754d0f66822d7c5" gracePeriod=30
Jul 27 19:29:47.838107 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:47.838039    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml"
Jul 27 19:29:47.922423 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:47.922384    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffwh7\" (UniqueName: \"kubernetes.io/projected/5af5df4d-1117-4c39-a915-f4e0567258e2-kube-api-access-ffwh7\") pod \"5af5df4d-1117-4c39-a915-f4e0567258e2\" (UID: \"5af5df4d-1117-4c39-a915-f4e0567258e2\") "
Jul 27 19:29:47.922423 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:47.922428    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5af5df4d-1117-4c39-a915-f4e0567258e2-plugins-config-volume\") pod \"5af5df4d-1117-4c39-a915-f4e0567258e2\" (UID: \"5af5df4d-1117-4c39-a915-f4e0567258e2\") "
Jul 27 19:29:47.922897 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:47.922838    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5af5df4d-1117-4c39-a915-f4e0567258e2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5af5df4d-1117-4c39-a915-f4e0567258e2" (UID: "5af5df4d-1117-4c39-a915-f4e0567258e2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:29:47.924725 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:47.924702    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5af5df4d-1117-4c39-a915-f4e0567258e2-kube-api-access-ffwh7" (OuterVolumeSpecName: "kube-api-access-ffwh7") pod "5af5df4d-1117-4c39-a915-f4e0567258e2" (UID: "5af5df4d-1117-4c39-a915-f4e0567258e2"). InnerVolumeSpecName "kube-api-access-ffwh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:29:48.003207 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:48.003169    2562 generic.go:356] "Generic (PLEG): container finished" podID="5af5df4d-1117-4c39-a915-f4e0567258e2" containerID="6684814a0ec3c7105711211ba1fe41a454cffb7a9e61eca20754d0f66822d7c5" exitCode=0
Jul 27 19:29:48.003378 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:48.003214    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml"
Jul 27 19:29:48.003378 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:48.003255    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml" event={"ID":"5af5df4d-1117-4c39-a915-f4e0567258e2","Type":"ContainerDied","Data":"6684814a0ec3c7105711211ba1fe41a454cffb7a9e61eca20754d0f66822d7c5"}
Jul 27 19:29:48.003378 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:48.003309    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml" event={"ID":"5af5df4d-1117-4c39-a915-f4e0567258e2","Type":"ContainerDied","Data":"43c38b51ce7c6beaa670eb9134ee2b82836de1dd39a219f9b9ee4747cedebfcf"}
Jul 27 19:29:48.003378 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:48.003335    2562 scope.go:117] "RemoveContainer" containerID="6684814a0ec3c7105711211ba1fe41a454cffb7a9e61eca20754d0f66822d7c5"
Jul 27 19:29:48.012165 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:48.012147    2562 scope.go:117] "RemoveContainer" containerID="6684814a0ec3c7105711211ba1fe41a454cffb7a9e61eca20754d0f66822d7c5"
Jul 27 19:29:48.012453 ip-10-0-132-186 kubenswrapper[2562]: E0727 19:29:48.012432    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6684814a0ec3c7105711211ba1fe41a454cffb7a9e61eca20754d0f66822d7c5\": container with ID starting with 6684814a0ec3c7105711211ba1fe41a454cffb7a9e61eca20754d0f66822d7c5 not found: ID does not exist" containerID="6684814a0ec3c7105711211ba1fe41a454cffb7a9e61eca20754d0f66822d7c5"
Jul 27 19:29:48.012505 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:48.012463    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6684814a0ec3c7105711211ba1fe41a454cffb7a9e61eca20754d0f66822d7c5"} err="failed to get container status \"6684814a0ec3c7105711211ba1fe41a454cffb7a9e61eca20754d0f66822d7c5\": rpc error: code = NotFound desc = could not find container \"6684814a0ec3c7105711211ba1fe41a454cffb7a9e61eca20754d0f66822d7c5\": container with ID starting with 6684814a0ec3c7105711211ba1fe41a454cffb7a9e61eca20754d0f66822d7c5 not found: ID does not exist"
Jul 27 19:29:48.018348 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:48.018318    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml"]
Jul 27 19:29:48.022385 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:48.022360    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2d7b97-656d7fcbb4-dflml"]
Jul 27 19:29:48.023134 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:48.023110    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffwh7\" (UniqueName: \"kubernetes.io/projected/5af5df4d-1117-4c39-a915-f4e0567258e2-kube-api-access-ffwh7\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:29:48.023134 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:48.023132    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5af5df4d-1117-4c39-a915-f4e0567258e2-plugins-config-volume\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:29:49.936588 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:49.936549    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5af5df4d-1117-4c39-a915-f4e0567258e2" path="/var/lib/kubelet/pods/5af5df4d-1117-4c39-a915-f4e0567258e2/volumes"
Jul 27 19:29:51.031307 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:51.031268    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2d7b97-z6hjw"]
Jul 27 19:29:51.035965 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:51.035925    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2d7b97-z6hjw"]
Jul 27 19:29:51.943351 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:51.943313    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3099acb4-7e58-484a-a01a-c8902587363c" path="/var/lib/kubelet/pods/3099acb4-7e58-484a-a01a-c8902587363c/volumes"
Jul 27 19:29:58.403735 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:58.403693    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"]
Jul 27 19:29:58.404193 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:58.403994    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g" podUID="4da4ae70-3dbb-4016-a410-c393df6438f5" containerName="istio-proxy" containerID="cri-o://e9cfe014c77f9abd28ea2df902c08b22c9a77456124e35b5c0a5f40e40f85265" gracePeriod=30
Jul 27 19:29:59.041923 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:59.041863    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 27 19:29:59.042107 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:29:59.041951    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g" podUID="4da4ae70-3dbb-4016-a410-c393df6438f5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 19:30:03.651480 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.651451    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:30:03.766401 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.766367    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-workload-certs\") pod \"4da4ae70-3dbb-4016-a410-c393df6438f5\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") "
Jul 27 19:30:03.766401 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.766403    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4da4ae70-3dbb-4016-a410-c393df6438f5-istiod-ca-cert\") pod \"4da4ae70-3dbb-4016-a410-c393df6438f5\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") "
Jul 27 19:30:03.766660 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.766426    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-podinfo\") pod \"4da4ae70-3dbb-4016-a410-c393df6438f5\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") "
Jul 27 19:30:03.766660 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.766454    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvb66\" (UniqueName: \"kubernetes.io/projected/4da4ae70-3dbb-4016-a410-c393df6438f5-kube-api-access-gvb66\") pod \"4da4ae70-3dbb-4016-a410-c393df6438f5\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") "
Jul 27 19:30:03.766660 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.766497    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-credential-socket\") pod \"4da4ae70-3dbb-4016-a410-c393df6438f5\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") "
Jul 27 19:30:03.766660 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.766513    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-envoy\") pod \"4da4ae70-3dbb-4016-a410-c393df6438f5\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") "
Jul 27 19:30:03.766660 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.766546    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-workload-socket\") pod \"4da4ae70-3dbb-4016-a410-c393df6438f5\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") "
Jul 27 19:30:03.766660 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.766570    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-data\") pod \"4da4ae70-3dbb-4016-a410-c393df6438f5\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") "
Jul 27 19:30:03.766660 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.766623    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-token\") pod \"4da4ae70-3dbb-4016-a410-c393df6438f5\" (UID: \"4da4ae70-3dbb-4016-a410-c393df6438f5\") "
Jul 27 19:30:03.767058 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.766746    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4da4ae70-3dbb-4016-a410-c393df6438f5" (UID: "4da4ae70-3dbb-4016-a410-c393df6438f5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:30:03.767058 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.766906    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4da4ae70-3dbb-4016-a410-c393df6438f5" (UID: "4da4ae70-3dbb-4016-a410-c393df6438f5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:30:03.767058 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.766928    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4da4ae70-3dbb-4016-a410-c393df6438f5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4da4ae70-3dbb-4016-a410-c393df6438f5" (UID: "4da4ae70-3dbb-4016-a410-c393df6438f5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:30:03.767058 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.767004    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-workload-certs\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:30:03.767058 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.767025    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4da4ae70-3dbb-4016-a410-c393df6438f5-istiod-ca-cert\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:30:03.767058 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.767039    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-credential-socket\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:30:03.767343 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.767121    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4da4ae70-3dbb-4016-a410-c393df6438f5" (UID: "4da4ae70-3dbb-4016-a410-c393df6438f5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:30:03.768158 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.768131    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-data" (OuterVolumeSpecName: "istio-data") pod "4da4ae70-3dbb-4016-a410-c393df6438f5" (UID: "4da4ae70-3dbb-4016-a410-c393df6438f5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:30:03.768828 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.768799    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4da4ae70-3dbb-4016-a410-c393df6438f5" (UID: "4da4ae70-3dbb-4016-a410-c393df6438f5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 19:30:03.768979 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.768941    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4da4ae70-3dbb-4016-a410-c393df6438f5" (UID: "4da4ae70-3dbb-4016-a410-c393df6438f5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:30:03.769419 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.769393    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-token" (OuterVolumeSpecName: "istio-token") pod "4da4ae70-3dbb-4016-a410-c393df6438f5" (UID: "4da4ae70-3dbb-4016-a410-c393df6438f5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:30:03.769523 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.769413    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4da4ae70-3dbb-4016-a410-c393df6438f5-kube-api-access-gvb66" (OuterVolumeSpecName: "kube-api-access-gvb66") pod "4da4ae70-3dbb-4016-a410-c393df6438f5" (UID: "4da4ae70-3dbb-4016-a410-c393df6438f5"). InnerVolumeSpecName "kube-api-access-gvb66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:30:03.868310 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.868266    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-envoy\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:30:03.868310 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.868301    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-workload-socket\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:30:03.868310 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.868311    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-data\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:30:03.868310 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.868319    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-token\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:30:03.868595 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.868327    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4da4ae70-3dbb-4016-a410-c393df6438f5-istio-podinfo\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:30:03.868595 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:03.868335    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvb66\" (UniqueName: \"kubernetes.io/projected/4da4ae70-3dbb-4016-a410-c393df6438f5-kube-api-access-gvb66\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:30:04.077306 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:04.077222    2562 generic.go:356] "Generic (PLEG): container finished" podID="4da4ae70-3dbb-4016-a410-c393df6438f5" containerID="e9cfe014c77f9abd28ea2df902c08b22c9a77456124e35b5c0a5f40e40f85265" exitCode=0
Jul 27 19:30:04.077306 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:04.077275    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"
Jul 27 19:30:04.077306 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:04.077298    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g" event={"ID":"4da4ae70-3dbb-4016-a410-c393df6438f5","Type":"ContainerDied","Data":"e9cfe014c77f9abd28ea2df902c08b22c9a77456124e35b5c0a5f40e40f85265"}
Jul 27 19:30:04.077563 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:04.077341    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g" event={"ID":"4da4ae70-3dbb-4016-a410-c393df6438f5","Type":"ContainerDied","Data":"c553eec03dad810afe77f1a960020db287914fb19f1e7ea1a031f0dbc283e308"}
Jul 27 19:30:04.077563 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:04.077370    2562 scope.go:117] "RemoveContainer" containerID="e9cfe014c77f9abd28ea2df902c08b22c9a77456124e35b5c0a5f40e40f85265"
Jul 27 19:30:04.086461 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:04.086434    2562 scope.go:117] "RemoveContainer" containerID="048854f198d810049a216efe8471ec503ed1f74f67297ebc871a3271e34e526b"
Jul 27 19:30:04.094538 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:04.094502    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"]
Jul 27 19:30:04.097049 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:04.096939    2562 scope.go:117] "RemoveContainer" containerID="e9cfe014c77f9abd28ea2df902c08b22c9a77456124e35b5c0a5f40e40f85265"
Jul 27 19:30:04.097693 ip-10-0-132-186 kubenswrapper[2562]: E0727 19:30:04.097608    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9cfe014c77f9abd28ea2df902c08b22c9a77456124e35b5c0a5f40e40f85265\": container with ID starting with e9cfe014c77f9abd28ea2df902c08b22c9a77456124e35b5c0a5f40e40f85265 not found: ID does not exist" containerID="e9cfe014c77f9abd28ea2df902c08b22c9a77456124e35b5c0a5f40e40f85265"
Jul 27 19:30:04.097693 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:04.097653    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9cfe014c77f9abd28ea2df902c08b22c9a77456124e35b5c0a5f40e40f85265"} err="failed to get container status \"e9cfe014c77f9abd28ea2df902c08b22c9a77456124e35b5c0a5f40e40f85265\": rpc error: code = NotFound desc = could not find container \"e9cfe014c77f9abd28ea2df902c08b22c9a77456124e35b5c0a5f40e40f85265\": container with ID starting with e9cfe014c77f9abd28ea2df902c08b22c9a77456124e35b5c0a5f40e40f85265 not found: ID does not exist"
Jul 27 19:30:04.097693 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:04.097681    2562 scope.go:117] "RemoveContainer" containerID="048854f198d810049a216efe8471ec503ed1f74f67297ebc871a3271e34e526b"
Jul 27 19:30:04.102777 ip-10-0-132-186 kubenswrapper[2562]: E0727 19:30:04.102739    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"048854f198d810049a216efe8471ec503ed1f74f67297ebc871a3271e34e526b\": container with ID starting with 048854f198d810049a216efe8471ec503ed1f74f67297ebc871a3271e34e526b not found: ID does not exist" containerID="048854f198d810049a216efe8471ec503ed1f74f67297ebc871a3271e34e526b"
Jul 27 19:30:04.102913 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:04.102782    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"048854f198d810049a216efe8471ec503ed1f74f67297ebc871a3271e34e526b"} err="failed to get container status \"048854f198d810049a216efe8471ec503ed1f74f67297ebc871a3271e34e526b\": rpc error: code = NotFound desc = could not find container \"048854f198d810049a216efe8471ec503ed1f74f67297ebc871a3271e34e526b\": container with ID starting with 048854f198d810049a216efe8471ec503ed1f74f67297ebc871a3271e34e526b not found: ID does not exist"
Jul 27 19:30:04.102913 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:04.102894    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-2d7b97-openshift-default-9fbfb667d-mtw7g"]
Jul 27 19:30:05.936338 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:05.936300    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4da4ae70-3dbb-4016-a410-c393df6438f5" path="/var/lib/kubelet/pods/4da4ae70-3dbb-4016-a410-c393df6438f5/volumes"
Jul 27 19:30:21.436374 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.436334    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ab99e9-p2bxp"]
Jul 27 19:30:21.436898 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.436646    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3099acb4-7e58-484a-a01a-c8902587363c" containerName="revoke-keys"
Jul 27 19:30:21.436898 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.436656    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3099acb4-7e58-484a-a01a-c8902587363c" containerName="revoke-keys"
Jul 27 19:30:21.436898 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.436668    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4da4ae70-3dbb-4016-a410-c393df6438f5" containerName="istio-proxy"
Jul 27 19:30:21.436898 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.436674    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4da4ae70-3dbb-4016-a410-c393df6438f5" containerName="istio-proxy"
Jul 27 19:30:21.436898 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.436689    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5af5df4d-1117-4c39-a915-f4e0567258e2" containerName="payload-pre-processing"
Jul 27 19:30:21.436898 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.436697    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5af5df4d-1117-4c39-a915-f4e0567258e2" containerName="payload-pre-processing"
Jul 27 19:30:21.436898 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.436705    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="444e89a7-263a-40cc-b844-88860c4bdda1" containerName="istio-proxy"
Jul 27 19:30:21.436898 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.436711    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="444e89a7-263a-40cc-b844-88860c4bdda1" containerName="istio-proxy"
Jul 27 19:30:21.436898 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.436720    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="444e89a7-263a-40cc-b844-88860c4bdda1" containerName="istio-proxy"
Jul 27 19:30:21.436898 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.436725    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="444e89a7-263a-40cc-b844-88860c4bdda1" containerName="istio-proxy"
Jul 27 19:30:21.436898 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.436774    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="444e89a7-263a-40cc-b844-88860c4bdda1" containerName="istio-proxy"
Jul 27 19:30:21.436898 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.436781    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4da4ae70-3dbb-4016-a410-c393df6438f5" containerName="istio-proxy"
Jul 27 19:30:21.436898 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.436787    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3099acb4-7e58-484a-a01a-c8902587363c" containerName="revoke-keys"
Jul 27 19:30:21.436898 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.436792    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="444e89a7-263a-40cc-b844-88860c4bdda1" containerName="istio-proxy"
Jul 27 19:30:21.436898 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.436798    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4da4ae70-3dbb-4016-a410-c393df6438f5" containerName="istio-proxy"
Jul 27 19:30:21.436898 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.436804    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5af5df4d-1117-4c39-a915-f4e0567258e2" containerName="payload-pre-processing"
Jul 27 19:30:21.439844 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.439825    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ab99e9-p2bxp"
Jul 27 19:30:21.441917 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.441895    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gwtlm\""
Jul 27 19:30:21.442057 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.441926    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 19:30:21.448231 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.448203    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ab99e9-p2bxp"]
Jul 27 19:30:21.519811 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.519775    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/091cb8a9-74ca-4785-af5b-0bff03f4e7bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ab99e9-p2bxp\" (UID: \"091cb8a9-74ca-4785-af5b-0bff03f4e7bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ab99e9-p2bxp"
Jul 27 19:30:21.620250 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.620215    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/091cb8a9-74ca-4785-af5b-0bff03f4e7bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ab99e9-p2bxp\" (UID: \"091cb8a9-74ca-4785-af5b-0bff03f4e7bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ab99e9-p2bxp"
Jul 27 19:30:21.620889 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.620842    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/091cb8a9-74ca-4785-af5b-0bff03f4e7bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ab99e9-p2bxp\" (UID: \"091cb8a9-74ca-4785-af5b-0bff03f4e7bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ab99e9-p2bxp"
Jul 27 19:30:21.751628 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.751585    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ab99e9-p2bxp"
Jul 27 19:30:21.897067 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:21.897036    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ab99e9-p2bxp"]
Jul 27 19:30:21.899533 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:30:21.899507    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod091cb8a9_74ca_4785_af5b_0bff03f4e7bc.slice/crio-9674ce3447b0718e1a714ebeb08d499f33e944d957cc9edeaeddd605363672ee WatchSource:0}: Error finding container 9674ce3447b0718e1a714ebeb08d499f33e944d957cc9edeaeddd605363672ee: Status 404 returned error can't find the container with id 9674ce3447b0718e1a714ebeb08d499f33e944d957cc9edeaeddd605363672ee
Jul 27 19:30:22.160672 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:22.160633    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ab99e9-p2bxp" event={"ID":"091cb8a9-74ca-4785-af5b-0bff03f4e7bc","Type":"ContainerStarted","Data":"9674ce3447b0718e1a714ebeb08d499f33e944d957cc9edeaeddd605363672ee"}
Jul 27 19:30:23.172090 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:23.172050    2562 generic.go:356] "Generic (PLEG): container finished" podID="091cb8a9-74ca-4785-af5b-0bff03f4e7bc" containerID="f4897bb0939f5be8f62a2af242e032360bdba40d0df08eb312a1c38e5cbce6db" exitCode=0
Jul 27 19:30:23.172544 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:23.172133    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ab99e9-p2bxp" event={"ID":"091cb8a9-74ca-4785-af5b-0bff03f4e7bc","Type":"ContainerDied","Data":"f4897bb0939f5be8f62a2af242e032360bdba40d0df08eb312a1c38e5cbce6db"}
Jul 27 19:30:24.305570 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:24.305545    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ab99e9-p2bxp"
Jul 27 19:30:24.445228 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:24.445132    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/091cb8a9-74ca-4785-af5b-0bff03f4e7bc-maas-api-service-ca\") pod \"091cb8a9-74ca-4785-af5b-0bff03f4e7bc\" (UID: \"091cb8a9-74ca-4785-af5b-0bff03f4e7bc\") "
Jul 27 19:30:24.445526 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:24.445501    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/091cb8a9-74ca-4785-af5b-0bff03f4e7bc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "091cb8a9-74ca-4785-af5b-0bff03f4e7bc" (UID: "091cb8a9-74ca-4785-af5b-0bff03f4e7bc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:30:24.545828 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:24.545789    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/091cb8a9-74ca-4785-af5b-0bff03f4e7bc-maas-api-service-ca\") on node \"ip-10-0-132-186.ec2.internal\" DevicePath \"\""
Jul 27 19:30:25.182132 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:25.182082    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ab99e9-p2bxp" event={"ID":"091cb8a9-74ca-4785-af5b-0bff03f4e7bc","Type":"ContainerDied","Data":"9674ce3447b0718e1a714ebeb08d499f33e944d957cc9edeaeddd605363672ee"}
Jul 27 19:30:25.182132 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:25.182120    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ab99e9-p2bxp"
Jul 27 19:30:25.182132 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:25.182141    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9674ce3447b0718e1a714ebeb08d499f33e944d957cc9edeaeddd605363672ee"
Jul 27 19:30:36.554532 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:36.554488    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ab99e9-p2bxp"]
Jul 27 19:30:36.558393 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:36.558361    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ab99e9-p2bxp"]
Jul 27 19:30:37.936362 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:37.936324    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="091cb8a9-74ca-4785-af5b-0bff03f4e7bc" path="/var/lib/kubelet/pods/091cb8a9-74ca-4785-af5b-0bff03f4e7bc/volumes"
Jul 27 19:30:49.858486 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:49.858456    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk_760a81e6-82a4-4ae0-8407-e3536402cca0/copy-manifests/0.log"
Jul 27 19:30:49.874573 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:49.874547    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk_760a81e6-82a4-4ae0-8407-e3536402cca0/manager/0.log"
Jul 27 19:30:52.021224 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:52.021195    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-c4ml2_6aed4676-7892-4dd9-8e18-a15ab50cbaa2/kuadrant-console-plugin/0.log"
Jul 27 19:30:53.133894 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:53.133834    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55894cd494-n5g7d_6384df57-3269-47c8-8187-e65b70d8b57a/kube-auth-proxy/0.log"
Jul 27 19:30:53.920017 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:53.919986    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-tzhvf_5fc6b031-e7ad-41bf-952e-a532a490e8c6/main/0.log"
Jul 27 19:30:54.138559 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:54.138527    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-v82k8_0ebccf45-daca-4e84-a865-354a4ff658d9/main/0.log"
Jul 27 19:30:54.355811 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:54.355779    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-v4j68_c4a97a97-176b-4e28-9388-ed7c33b6e250/main/0.log"
Jul 27 19:30:54.467248 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:30:54.467218    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-9sgsb_fefdc4f8-7128-41fa-b0c4-c5d919415561/main/0.log"
Jul 27 19:31:02.255490 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:02.255449    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rmf9d_914d3bcb-4f6c-4153-aec8-e4547ff69068/global-pull-secret-syncer/0.log"
Jul 27 19:31:02.468155 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:02.468129    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-scl2m_a56a2199-4863-403e-bc3e-b601728e5406/konnectivity-agent/0.log"
Jul 27 19:31:02.554636 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:02.554553    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-186.ec2.internal_129a2108b5a3803a480ffe6e1a6b9b55/haproxy/0.log"
Jul 27 19:31:07.105632 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:07.105602    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-c4ml2_6aed4676-7892-4dd9-8e18-a15ab50cbaa2/kuadrant-console-plugin/0.log"
Jul 27 19:31:09.365115 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:09.365082    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-rhrcr_5d6f565b-25e3-4db4-89cc-28f0a0cbc890/cluster-monitoring-operator/0.log"
Jul 27 19:31:09.529933 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:09.529899    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-27krt_285f4d97-01a4-43a9-a067-abcec72f14a2/node-exporter/0.log"
Jul 27 19:31:09.553172 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:09.553142    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-27krt_285f4d97-01a4-43a9-a067-abcec72f14a2/kube-rbac-proxy/0.log"
Jul 27 19:31:09.579715 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:09.579682    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-27krt_285f4d97-01a4-43a9-a067-abcec72f14a2/init-textfile/0.log"
Jul 27 19:31:11.882679 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:11.882644    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-8jtmk_0e1a6428-a17e-485f-96b7-6b1a92afb72d/networking-console-plugin/0.log"
Jul 27 19:31:12.487085 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:12.487050    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/2.log"
Jul 27 19:31:12.490446 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:12.490427    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-rbtsd_2f2170b2-dbd3-4867-9eaf-291ba1ebe412/console-operator/3.log"
Jul 27 19:31:12.785035 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:12.784946    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"]
Jul 27 19:31:12.785263 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:12.785251    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091cb8a9-74ca-4785-af5b-0bff03f4e7bc" containerName="revoke-keys"
Jul 27 19:31:12.785320 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:12.785265    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="091cb8a9-74ca-4785-af5b-0bff03f4e7bc" containerName="revoke-keys"
Jul 27 19:31:12.785320 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:12.785276    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4da4ae70-3dbb-4016-a410-c393df6438f5" containerName="istio-proxy"
Jul 27 19:31:12.785320 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:12.785283    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4da4ae70-3dbb-4016-a410-c393df6438f5" containerName="istio-proxy"
Jul 27 19:31:12.785444 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:12.785349    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="091cb8a9-74ca-4785-af5b-0bff03f4e7bc" containerName="revoke-keys"
Jul 27 19:31:12.788118 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:12.788097    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"
Jul 27 19:31:12.790314 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:12.790285    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jdk7g\"/\"kube-root-ca.crt\""
Jul 27 19:31:12.790852 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:12.790831    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jdk7g\"/\"default-dockercfg-tdhxf\""
Jul 27 19:31:12.790990 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:12.790832    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jdk7g\"/\"openshift-service-ca.crt\""
Jul 27 19:31:12.799367 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:12.799339    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"]
Jul 27 19:31:12.974691 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:12.974642    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7cfa4e1b-b3a9-4eea-acaf-c50f38a37664-proc\") pod \"perf-node-gather-daemonset-4jdkw\" (UID: \"7cfa4e1b-b3a9-4eea-acaf-c50f38a37664\") " pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"
Jul 27 19:31:12.974691 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:12.974696    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7cfa4e1b-b3a9-4eea-acaf-c50f38a37664-podres\") pod \"perf-node-gather-daemonset-4jdkw\" (UID: \"7cfa4e1b-b3a9-4eea-acaf-c50f38a37664\") " pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"
Jul 27 19:31:12.975168 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:12.974721    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xdnk\" (UniqueName: \"kubernetes.io/projected/7cfa4e1b-b3a9-4eea-acaf-c50f38a37664-kube-api-access-8xdnk\") pod \"perf-node-gather-daemonset-4jdkw\" (UID: \"7cfa4e1b-b3a9-4eea-acaf-c50f38a37664\") " pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"
Jul 27 19:31:12.975168 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:12.974766    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7cfa4e1b-b3a9-4eea-acaf-c50f38a37664-lib-modules\") pod \"perf-node-gather-daemonset-4jdkw\" (UID: \"7cfa4e1b-b3a9-4eea-acaf-c50f38a37664\") " pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"
Jul 27 19:31:12.975168 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:12.974821    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7cfa4e1b-b3a9-4eea-acaf-c50f38a37664-sys\") pod \"perf-node-gather-daemonset-4jdkw\" (UID: \"7cfa4e1b-b3a9-4eea-acaf-c50f38a37664\") " pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"
Jul 27 19:31:13.076027 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:13.075933    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xdnk\" (UniqueName: \"kubernetes.io/projected/7cfa4e1b-b3a9-4eea-acaf-c50f38a37664-kube-api-access-8xdnk\") pod \"perf-node-gather-daemonset-4jdkw\" (UID: \"7cfa4e1b-b3a9-4eea-acaf-c50f38a37664\") " pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"
Jul 27 19:31:13.076027 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:13.076011    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7cfa4e1b-b3a9-4eea-acaf-c50f38a37664-lib-modules\") pod \"perf-node-gather-daemonset-4jdkw\" (UID: \"7cfa4e1b-b3a9-4eea-acaf-c50f38a37664\") " pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"
Jul 27 19:31:13.076262 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:13.076091    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7cfa4e1b-b3a9-4eea-acaf-c50f38a37664-sys\") pod \"perf-node-gather-daemonset-4jdkw\" (UID: \"7cfa4e1b-b3a9-4eea-acaf-c50f38a37664\") " pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"
Jul 27 19:31:13.076262 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:13.076132    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7cfa4e1b-b3a9-4eea-acaf-c50f38a37664-proc\") pod \"perf-node-gather-daemonset-4jdkw\" (UID: \"7cfa4e1b-b3a9-4eea-acaf-c50f38a37664\") " pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"
Jul 27 19:31:13.076262 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:13.076163    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7cfa4e1b-b3a9-4eea-acaf-c50f38a37664-podres\") pod \"perf-node-gather-daemonset-4jdkw\" (UID: \"7cfa4e1b-b3a9-4eea-acaf-c50f38a37664\") " pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"
Jul 27 19:31:13.076382 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:13.076255    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7cfa4e1b-b3a9-4eea-acaf-c50f38a37664-sys\") pod \"perf-node-gather-daemonset-4jdkw\" (UID: \"7cfa4e1b-b3a9-4eea-acaf-c50f38a37664\") " pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"
Jul 27 19:31:13.076382 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:13.076288    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7cfa4e1b-b3a9-4eea-acaf-c50f38a37664-podres\") pod \"perf-node-gather-daemonset-4jdkw\" (UID: \"7cfa4e1b-b3a9-4eea-acaf-c50f38a37664\") " pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"
Jul 27 19:31:13.076382 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:13.076289    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7cfa4e1b-b3a9-4eea-acaf-c50f38a37664-lib-modules\") pod \"perf-node-gather-daemonset-4jdkw\" (UID: \"7cfa4e1b-b3a9-4eea-acaf-c50f38a37664\") " pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"
Jul 27 19:31:13.076382 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:13.076255    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7cfa4e1b-b3a9-4eea-acaf-c50f38a37664-proc\") pod \"perf-node-gather-daemonset-4jdkw\" (UID: \"7cfa4e1b-b3a9-4eea-acaf-c50f38a37664\") " pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"
Jul 27 19:31:13.084295 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:13.084268    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xdnk\" (UniqueName: \"kubernetes.io/projected/7cfa4e1b-b3a9-4eea-acaf-c50f38a37664-kube-api-access-8xdnk\") pod \"perf-node-gather-daemonset-4jdkw\" (UID: \"7cfa4e1b-b3a9-4eea-acaf-c50f38a37664\") " pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"
Jul 27 19:31:13.098236 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:13.098197    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"
Jul 27 19:31:13.232826 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:13.232657    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"]
Jul 27 19:31:13.235904 ip-10-0-132-186 kubenswrapper[2562]: W0727 19:31:13.235858    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7cfa4e1b_b3a9_4eea_acaf_c50f38a37664.slice/crio-527434a1448416983af78d54bba83da42ae2b1c13037a6ae2fb8fcb94db33b9d WatchSource:0}: Error finding container 527434a1448416983af78d54bba83da42ae2b1c13037a6ae2fb8fcb94db33b9d: Status 404 returned error can't find the container with id 527434a1448416983af78d54bba83da42ae2b1c13037a6ae2fb8fcb94db33b9d
Jul 27 19:31:13.398493 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:13.398403    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw" event={"ID":"7cfa4e1b-b3a9-4eea-acaf-c50f38a37664","Type":"ContainerStarted","Data":"1471cc2572ad318b2b981669f798e3fc9b426b7bc1d276a5206c77539b0adc15"}
Jul 27 19:31:13.398493 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:13.398453    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw" event={"ID":"7cfa4e1b-b3a9-4eea-acaf-c50f38a37664","Type":"ContainerStarted","Data":"527434a1448416983af78d54bba83da42ae2b1c13037a6ae2fb8fcb94db33b9d"}
Jul 27 19:31:13.398679 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:13.398490    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"
Jul 27 19:31:13.414779 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:13.414725    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw" podStartSLOduration=1.4147078770000001 podStartE2EDuration="1.414707877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:31:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:31:13.41349965 +0000 UTC m=+3581.963793283" watchObservedRunningTime="2026-07-27 19:31:13.414707877 +0000 UTC m=+3581.965001564"
Jul 27 19:31:14.822673 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:14.822643    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wzhvk_2ea9768a-db15-4a87-b27c-1c5e69af52b8/dns/0.log"
Jul 27 19:31:14.842243 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:14.842219    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wzhvk_2ea9768a-db15-4a87-b27c-1c5e69af52b8/kube-rbac-proxy/0.log"
Jul 27 19:31:14.909709 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:14.909674    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8kqdg_05f049d3-8c6b-4155-8586-be26e91c5bb9/dns-node-resolver/0.log"
Jul 27 19:31:15.553764 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:15.553728    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jz5n4_2efd4785-cf26-4815-86af-86dd8dbd5108/node-ca/0.log"
Jul 27 19:31:16.859886 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:16.859850    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55894cd494-n5g7d_6384df57-3269-47c8-8187-e65b70d8b57a/kube-auth-proxy/0.log"
Jul 27 19:31:18.002289 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:18.002256    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xjh28_243eab96-1170-4ea0-98fa-d9cb4a68adb2/serve-healthcheck-canary/0.log"
Jul 27 19:31:18.477656 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:18.477622    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-fwf4j_476d7150-35a6-4b7d-beee-1fbac9faa00f/insights-operator/0.log"
Jul 27 19:31:18.478272 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:18.478256    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-fwf4j_476d7150-35a6-4b7d-beee-1fbac9faa00f/insights-operator/1.log"
Jul 27 19:31:18.632176 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:18.632144    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mpdtb_c71f0bfd-e960-46e5-b005-2978249db03a/kube-rbac-proxy/0.log"
Jul 27 19:31:18.652730 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:18.652701    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mpdtb_c71f0bfd-e960-46e5-b005-2978249db03a/exporter/0.log"
Jul 27 19:31:18.673199 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:18.673159    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mpdtb_c71f0bfd-e960-46e5-b005-2978249db03a/extractor/0.log"
Jul 27 19:31:19.412788 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:19.412761    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jdk7g/perf-node-gather-daemonset-4jdkw"
Jul 27 19:31:21.575600 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:21.575568    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk_760a81e6-82a4-4ae0-8407-e3536402cca0/manager/0.log"
Jul 27 19:31:21.599806 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:21.599779    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dfd8c75b7-hrwrk_760a81e6-82a4-4ae0-8407-e3536402cca0/copy-manifests/0.log"
Jul 27 19:31:23.137116 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:23.136881    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5c45656c54-qhv7b_d0926f63-f6cc-41e9-a804-ca6ccd217df1/manager/1.log"
Jul 27 19:31:23.153852 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:23.137258    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5c45656c54-qhv7b_d0926f63-f6cc-41e9-a804-ca6ccd217df1/manager/0.log"
Jul 27 19:31:23.535636 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:23.535589    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55894cd494-n5g7d_6384df57-3269-47c8-8187-e65b70d8b57a/kube-auth-proxy/0.log"
Jul 27 19:31:25.234310 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:25.234280    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55894cd494-n5g7d_6384df57-3269-47c8-8187-e65b70d8b57a/kube-auth-proxy/0.log"
Jul 27 19:31:29.570089 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:29.570054    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9gvlm_5cf7f534-634d-4eab-9577-7d03c7cdb9cb/kube-multus/0.log"
Jul 27 19:31:29.758561 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:29.758528    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q9k9p_be23ce78-126d-46a9-95d2-40d68c20f6f2/kube-multus-additional-cni-plugins/0.log"
Jul 27 19:31:29.783725 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:29.783638    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q9k9p_be23ce78-126d-46a9-95d2-40d68c20f6f2/egress-router-binary-copy/0.log"
Jul 27 19:31:29.805995 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:29.805968    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q9k9p_be23ce78-126d-46a9-95d2-40d68c20f6f2/cni-plugins/0.log"
Jul 27 19:31:29.827563 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:29.827529    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q9k9p_be23ce78-126d-46a9-95d2-40d68c20f6f2/bond-cni-plugin/0.log"
Jul 27 19:31:29.850001 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:29.849965    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q9k9p_be23ce78-126d-46a9-95d2-40d68c20f6f2/routeoverride-cni/0.log"
Jul 27 19:31:29.875364 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:29.875337    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q9k9p_be23ce78-126d-46a9-95d2-40d68c20f6f2/whereabouts-cni-bincopy/0.log"
Jul 27 19:31:29.897972 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:29.897938    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q9k9p_be23ce78-126d-46a9-95d2-40d68c20f6f2/whereabouts-cni/0.log"
Jul 27 19:31:30.209199 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:30.209168    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mhtmr_88f67acb-91af-4850-bb4d-81ff499bc672/network-metrics-daemon/0.log"
Jul 27 19:31:30.235424 ip-10-0-132-186 kubenswrapper[2562]: I0727 19:31:30.235390    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mhtmr_88f67acb-91af-4850-bb4d-81ff499bc672/kube-rbac-proxy/0.log"