Jul 07 22:12:50.572664 ip-10-0-128-159 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 07 22:12:50.572679 ip-10-0-128-159 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 07 22:12:50.572687 ip-10-0-128-159 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 07 22:12:50.572999 ip-10-0-128-159 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 07 22:13:00.769490 ip-10-0-128-159 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 07 22:13:00.769513 ip-10-0-128-159 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot a318490282b74bcc9211d99d12a705ad --
Jul 07 22:15:01.075076 ip-10-0-128-159 systemd[1]: Starting Kubernetes Kubelet...
Jul 07 22:15:02.650290 ip-10-0-128-159 kubenswrapper[2555]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 22:15:02.650290 ip-10-0-128-159 kubenswrapper[2555]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 07 22:15:02.650290 ip-10-0-128-159 kubenswrapper[2555]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 22:15:02.650290 ip-10-0-128-159 kubenswrapper[2555]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 07 22:15:02.650290 ip-10-0-128-159 kubenswrapper[2555]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 22:15:02.651195 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.651109    2555 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 07 22:15:02.657785 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657765    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 22:15:02.657785 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657782    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 22:15:02.657785 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657786    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 22:15:02.657785 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657789    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 22:15:02.657785 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657792    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 22:15:02.658028 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657795    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 22:15:02.658028 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657800    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 22:15:02.658028 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657802    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 22:15:02.658028 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657805    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 22:15:02.658028 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657808    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 22:15:02.658028 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657810    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 22:15:02.658028 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657813    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 22:15:02.658028 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657816    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 22:15:02.658028 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657819    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 22:15:02.658028 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657821    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 22:15:02.658028 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657824    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 22:15:02.658028 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657826    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 22:15:02.658028 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657829    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 22:15:02.658028 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657832    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 22:15:02.658028 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657834    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 22:15:02.658028 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657837    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 22:15:02.658028 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657839    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 22:15:02.658028 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657842    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 22:15:02.658028 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657844    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 22:15:02.658028 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657847    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 22:15:02.658502 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657849    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 22:15:02.658502 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657852    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 22:15:02.658502 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657854    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 22:15:02.658502 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657857    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 22:15:02.658502 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657859    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 22:15:02.658502 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657862    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 22:15:02.658502 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657884    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 22:15:02.658502 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657888    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 22:15:02.658502 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657890    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 22:15:02.658502 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657893    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 22:15:02.658502 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657895    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 22:15:02.658502 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657898    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 22:15:02.658502 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657900    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 22:15:02.658502 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657905    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 22:15:02.658502 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657907    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 22:15:02.658502 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657910    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 22:15:02.658502 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657912    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 22:15:02.658502 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657915    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 22:15:02.658502 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657917    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 22:15:02.658502 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657920    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 22:15:02.658992 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657922    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 22:15:02.658992 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657925    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 22:15:02.658992 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657928    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 22:15:02.658992 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657930    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 22:15:02.658992 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657933    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 22:15:02.658992 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657935    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 22:15:02.658992 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657938    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 22:15:02.658992 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657941    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 22:15:02.658992 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657943    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 22:15:02.658992 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657946    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 22:15:02.658992 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657948    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 22:15:02.658992 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657951    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 22:15:02.658992 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657954    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 22:15:02.658992 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657957    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 22:15:02.658992 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657959    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 22:15:02.658992 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657962    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 22:15:02.658992 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657964    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 22:15:02.658992 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657966    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 22:15:02.658992 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657969    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 22:15:02.658992 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657972    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 22:15:02.659459 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657974    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 22:15:02.659459 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657978    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 22:15:02.659459 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657983    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 22:15:02.659459 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657986    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 22:15:02.659459 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657989    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 22:15:02.659459 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657992    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 22:15:02.659459 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657995    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 22:15:02.659459 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.657998    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 22:15:02.659459 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658000    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 22:15:02.659459 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658003    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 22:15:02.659459 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658005    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 22:15:02.659459 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658009    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 22:15:02.659459 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658013    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 22:15:02.659459 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658015    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 22:15:02.659459 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658018    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 22:15:02.659459 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658021    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 22:15:02.659459 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658025    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 22:15:02.659459 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658029    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 22:15:02.659914 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658034    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 22:15:02.659914 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658037    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 22:15:02.659914 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658040    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 22:15:02.659914 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658043    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 22:15:02.659914 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658045    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 22:15:02.659914 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658048    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 22:15:02.659914 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658051    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 22:15:02.659914 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658053    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 22:15:02.659914 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658056    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 22:15:02.659914 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658058    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 22:15:02.659914 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658061    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 22:15:02.659914 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658064    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 22:15:02.659914 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658096    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 22:15:02.659914 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658101    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 22:15:02.659914 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658104    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 22:15:02.659914 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658108    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 22:15:02.659914 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658817    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 22:15:02.659914 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658824    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 22:15:02.659914 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658827    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 22:15:02.659914 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658830    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 22:15:02.660391 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658833    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 22:15:02.660391 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658836    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 22:15:02.660391 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658839    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 22:15:02.660391 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658842    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 22:15:02.660391 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658844    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 22:15:02.660391 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658847    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 22:15:02.660391 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658849    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 22:15:02.660391 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658852    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 22:15:02.660391 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658855    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 22:15:02.660391 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658857    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 22:15:02.660391 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658860    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 22:15:02.660391 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658865    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 22:15:02.660391 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658884    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 22:15:02.660391 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658888    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 22:15:02.660391 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658891    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 22:15:02.660391 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658895    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 22:15:02.660391 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658897    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 22:15:02.660391 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658900    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 22:15:02.660391 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658902    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 22:15:02.660391 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658905    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 22:15:02.660921 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658907    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 22:15:02.660921 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658910    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 22:15:02.660921 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658913    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 22:15:02.660921 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658915    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 22:15:02.660921 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658918    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 22:15:02.660921 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658920    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 22:15:02.660921 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658923    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 22:15:02.660921 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658925    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 22:15:02.660921 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658928    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 22:15:02.660921 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658931    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 22:15:02.660921 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658933    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 22:15:02.660921 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658936    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 22:15:02.660921 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658939    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 22:15:02.660921 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658941    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 22:15:02.660921 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658944    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 22:15:02.660921 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658946    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 22:15:02.660921 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658949    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 22:15:02.660921 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658951    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 22:15:02.660921 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658954    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 22:15:02.661378 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658956    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 22:15:02.661378 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658959    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 22:15:02.661378 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658961    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 22:15:02.661378 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658964    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 22:15:02.661378 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658967    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 22:15:02.661378 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658970    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 22:15:02.661378 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658973    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 22:15:02.661378 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658976    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 22:15:02.661378 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658978    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 22:15:02.661378 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658980    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 22:15:02.661378 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658983    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 22:15:02.661378 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658985    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 22:15:02.661378 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658988    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 22:15:02.661378 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658990    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 22:15:02.661378 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658992    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 22:15:02.661378 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658995    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 22:15:02.661378 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.658998    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 22:15:02.661378 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659001    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 22:15:02.661378 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659005    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 22:15:02.661378 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659010    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 22:15:02.661859 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659012    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 22:15:02.661859 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659015    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 22:15:02.661859 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659018    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 22:15:02.661859 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659020    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 22:15:02.661859 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659023    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 22:15:02.661859 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659025    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 22:15:02.661859 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659028    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 22:15:02.661859 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659030    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 22:15:02.661859 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659033    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 22:15:02.661859 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659035    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 22:15:02.661859 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659037    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 22:15:02.661859 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659040    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 22:15:02.661859 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659043    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 22:15:02.661859 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659045    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 22:15:02.661859 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659047    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 22:15:02.661859 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659056    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 22:15:02.661859 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659058    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 22:15:02.661859 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659061    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 22:15:02.661859 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659064    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 22:15:02.661859 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659067    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 22:15:02.662345 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659070    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 22:15:02.662345 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659072    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 22:15:02.662345 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659075    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 22:15:02.662345 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659077    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 22:15:02.662345 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659080    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 22:15:02.662345 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659082    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 22:15:02.662345 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659085    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 22:15:02.662345 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659089    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 22:15:02.662345 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659092    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 22:15:02.662345 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659095    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 22:15:02.662345 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659098    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 22:15:02.662345 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659101    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 22:15:02.662345 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659104    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 22:15:02.662345 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659106    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 22:15:02.662345 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659109    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 22:15:02.662345 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659111    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 22:15:02.662345 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659503    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 22:15:02.662345 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659509    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 22:15:02.662345 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659513    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 22:15:02.662792 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659517    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 22:15:02.662792 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659520    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 22:15:02.662792 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659523    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 22:15:02.662792 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659525    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 22:15:02.662792 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659528    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 22:15:02.662792 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659531    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 22:15:02.662792 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659533    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 22:15:02.662792 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659536    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 22:15:02.662792 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659539    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 22:15:02.662792 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659541    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 22:15:02.662792 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659544    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 22:15:02.662792 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659547    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 22:15:02.662792 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659549    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 22:15:02.662792 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659552    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 22:15:02.662792 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659555    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 22:15:02.662792 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659557    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 22:15:02.662792 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659560    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 22:15:02.662792 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659562    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 22:15:02.662792 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659565    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 22:15:02.663264 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659567    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 22:15:02.663264 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659570    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 22:15:02.663264 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659572    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 22:15:02.663264 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659575    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 22:15:02.663264 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659578    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 22:15:02.663264 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659582    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 22:15:02.663264 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659585    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 22:15:02.663264 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659588    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 22:15:02.663264 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659591    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 22:15:02.663264 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659593    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 22:15:02.663264 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659596    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 22:15:02.663264 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659599    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 22:15:02.663264 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659602    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 22:15:02.663264 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659605    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 22:15:02.663264 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659607    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 22:15:02.663264 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659611    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 22:15:02.663264 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659614    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 22:15:02.663264 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659617    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 22:15:02.663264 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659621    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 22:15:02.663735 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659624    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 22:15:02.663735 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659627    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 22:15:02.663735 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659630    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 22:15:02.663735 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659633    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 22:15:02.663735 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659636    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 22:15:02.663735 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659638    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 22:15:02.663735 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659641    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 22:15:02.663735 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659643    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 22:15:02.663735 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659647    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 22:15:02.663735 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659649    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 22:15:02.663735 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659652    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 22:15:02.663735 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659656    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 22:15:02.663735 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659659    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 22:15:02.663735 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659661    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 22:15:02.663735 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659664    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 22:15:02.663735 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659666    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 22:15:02.663735 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659668    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 22:15:02.663735 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659671    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 22:15:02.663735 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659674    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 22:15:02.663735 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659676    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 22:15:02.664237 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659679    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 22:15:02.664237 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659681    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 22:15:02.664237 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659683    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 22:15:02.664237 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659686    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 22:15:02.664237 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659688    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 22:15:02.664237 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659691    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 22:15:02.664237 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659694    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 22:15:02.664237 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659696    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 22:15:02.664237 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659698    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 22:15:02.664237 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659701    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 22:15:02.664237 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659703    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 22:15:02.664237 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659705    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 22:15:02.664237 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659708    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 22:15:02.664237 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659711    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 22:15:02.664237 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659713    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 22:15:02.664237 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659716    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 22:15:02.664237 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659719    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 22:15:02.664237 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659733    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 22:15:02.664237 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659736    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 22:15:02.664237 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659738    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 22:15:02.664700 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659741    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 22:15:02.664700 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659744    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 22:15:02.664700 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659747    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 22:15:02.664700 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659750    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 22:15:02.664700 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659752    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 22:15:02.664700 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659755    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 22:15:02.664700 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659757    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 22:15:02.664700 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659759    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 22:15:02.664700 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659762    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 22:15:02.664700 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659767    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 22:15:02.664700 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659770    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 22:15:02.664700 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659773    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 22:15:02.664700 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659775    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 22:15:02.664700 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659778    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 22:15:02.664700 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659781    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 22:15:02.664700 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659783    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 22:15:02.664700 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659786    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 22:15:02.664700 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.659788    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 22:15:02.664700 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659885    2555 flags.go:64] FLAG: --address="0.0.0.0"
Jul 07 22:15:02.664700 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659895    2555 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 07 22:15:02.664700 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659904    2555 flags.go:64] FLAG: --anonymous-auth="true"
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659910    2555 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659915    2555 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659918    2555 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659923    2555 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659927    2555 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659930    2555 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659933    2555 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659937    2555 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659940    2555 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659943    2555 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659945    2555 flags.go:64] FLAG: --cgroup-root=""
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659948    2555 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659951    2555 flags.go:64] FLAG: --client-ca-file=""
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659955    2555 flags.go:64] FLAG: --cloud-provider="external"
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659957    2555 flags.go:64] FLAG: --cluster-dns="[]"
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659963    2555 flags.go:64] FLAG: --cluster-domain=""
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659965    2555 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659968    2555 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659971    2555 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659975    2555 flags.go:64] FLAG: --container-log-max-files="5"
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659980    2555 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659983    2555 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 07 22:15:02.665218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659986    2555 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659989    2555 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659992    2555 flags.go:64] FLAG: --contention-profiling="false"
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659995    2555 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.659998    2555 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660001    2555 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660004    2555 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660008    2555 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660011    2555 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660013    2555 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660016    2555 flags.go:64] FLAG: --enable-load-reader="false"
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660019    2555 flags.go:64] FLAG: --enable-server="true"
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660021    2555 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660026    2555 flags.go:64] FLAG: --event-burst="100"
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660028    2555 flags.go:64] FLAG: --event-qps="50"
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660033    2555 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660036    2555 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660039    2555 flags.go:64] FLAG: --eviction-hard=""
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660042    2555 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660045    2555 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660048    2555 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660051    2555 flags.go:64] FLAG: --eviction-soft=""
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660053    2555 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660056    2555 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660060    2555 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 07 22:15:02.665797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660063    2555 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660066    2555 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660068    2555 flags.go:64] FLAG: --fail-swap-on="true"
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660071    2555 flags.go:64] FLAG: --feature-gates=""
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660075    2555 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660080    2555 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660083    2555 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660086    2555 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660089    2555 flags.go:64] FLAG: --healthz-port="10248"
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660092    2555 flags.go:64] FLAG: --help="false"
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660095    2555 flags.go:64] FLAG: --hostname-override="ip-10-0-128-159.ec2.internal"
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660097    2555 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660100    2555 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660103    2555 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660107    2555 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660110    2555 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660113    2555 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660116    2555 flags.go:64] FLAG: --image-service-endpoint=""
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660119    2555 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660121    2555 flags.go:64] FLAG: --kube-api-burst="100"
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660124    2555 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660127    2555 flags.go:64] FLAG: --kube-api-qps="50"
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660131    2555 flags.go:64] FLAG: --kube-reserved=""
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660134    2555 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 07 22:15:02.666433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660137    2555 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660140    2555 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660143    2555 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660145    2555 flags.go:64] FLAG: --lock-file=""
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660148    2555 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660151    2555 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660154    2555 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660159    2555 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660162    2555 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660165    2555 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660167    2555 flags.go:64] FLAG: --logging-format="text"
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660170    2555 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660173    2555 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660177    2555 flags.go:64] FLAG: --manifest-url=""
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660180    2555 flags.go:64] FLAG: --manifest-url-header=""
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660184    2555 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660187    2555 flags.go:64] FLAG: --max-open-files="1000000"
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660191    2555 flags.go:64] FLAG: --max-pods="110"
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660194    2555 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660197    2555 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660200    2555 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660202    2555 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660205    2555 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660208    2555 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660211    2555 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 07 22:15:02.667036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660218    2555 flags.go:64] FLAG: --node-status-max-images="50"
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660221    2555 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660224    2555 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660227    2555 flags.go:64] FLAG: --pod-cidr=""
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660229    2555 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660236    2555 flags.go:64] FLAG: --pod-manifest-path=""
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660239    2555 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660242    2555 flags.go:64] FLAG: --pods-per-core="0"
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660245    2555 flags.go:64] FLAG: --port="10250"
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660248    2555 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660251    2555 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06c01954c0bd61bc9"
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660254    2555 flags.go:64] FLAG: --qos-reserved=""
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660257    2555 flags.go:64] FLAG: --read-only-port="10255"
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660260    2555 flags.go:64] FLAG: --register-node="true"
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660262    2555 flags.go:64] FLAG: --register-with-taints=""
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660266    2555 flags.go:64] FLAG: --registry-burst="10"
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660269    2555 flags.go:64] FLAG: --registry-qps="5"
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660272    2555 flags.go:64] FLAG: --reserved-cpus=""
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660275    2555 flags.go:64] FLAG: --reserved-memory=""
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660278    2555 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660281    2555 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660285    2555 flags.go:64] FLAG: --rotate-certificates="false"
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660288    2555 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660291    2555 flags.go:64] FLAG: --runonce="false"
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660294    2555 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 07 22:15:02.668158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660296    2555 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660299    2555 flags.go:64] FLAG: --seccomp-default="false"
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660302    2555 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660305    2555 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660308    2555 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660311    2555 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660314    2555 flags.go:64] FLAG: --storage-driver-password="root"
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660316    2555 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660319    2555 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660322    2555 flags.go:64] FLAG: --storage-driver-user="root"
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660324    2555 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660327    2555 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660331    2555 flags.go:64] FLAG: --system-cgroups=""
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660334    2555 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660339    2555 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660342    2555 flags.go:64] FLAG: --tls-cert-file=""
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660344    2555 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660349    2555 flags.go:64] FLAG: --tls-min-version=""
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660352    2555 flags.go:64] FLAG: --tls-private-key-file=""
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660355    2555 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660358    2555 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660360    2555 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660363    2555 flags.go:64] FLAG: --v="2"
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660367    2555 flags.go:64] FLAG: --version="false"
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660371    2555 flags.go:64] FLAG: --vmodule=""
Jul 07 22:15:02.669228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660375    2555 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 07 22:15:02.670300 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.660378    2555 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 07 22:15:02.670300 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660493    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 22:15:02.670300 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660499    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 22:15:02.670300 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660501    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 22:15:02.670300 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660504    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 22:15:02.670300 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660507    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 22:15:02.670300 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660509    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 22:15:02.670300 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660512    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 22:15:02.670300 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660515    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 22:15:02.670300 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660517    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 22:15:02.670300 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660520    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 22:15:02.670300 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660522    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 22:15:02.670300 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660525    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 22:15:02.670300 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660527    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 22:15:02.670300 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660530    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 22:15:02.670300 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660533    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 22:15:02.670300 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660535    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 22:15:02.670300 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660538    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 22:15:02.670300 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660542    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 22:15:02.670300 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660544    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 22:15:02.671100 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660547    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 22:15:02.671100 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660550    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 22:15:02.671100 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660552    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 22:15:02.671100 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660555    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 22:15:02.671100 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660557    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 22:15:02.671100 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660560    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 22:15:02.671100 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660563    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 22:15:02.671100 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660565    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 22:15:02.671100 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660569    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 22:15:02.671100 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660572    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 22:15:02.671100 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660575    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 22:15:02.671100 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660578    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 22:15:02.671100 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660581    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 22:15:02.671100 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660583    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 22:15:02.671100 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660587    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 22:15:02.671100 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660590    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 22:15:02.671100 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660592    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 22:15:02.671100 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660595    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 22:15:02.671100 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660597    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 22:15:02.671685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660600    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 22:15:02.671685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660602    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 22:15:02.671685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660605    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 22:15:02.671685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660608    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 22:15:02.671685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660610    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 22:15:02.671685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660613    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 22:15:02.671685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660615    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 22:15:02.671685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660618    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 22:15:02.671685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660620    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 22:15:02.671685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660623    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 22:15:02.671685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660625    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 22:15:02.671685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660629    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 22:15:02.671685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660632    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 22:15:02.671685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660634    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 22:15:02.671685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660637    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 22:15:02.671685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660639    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 22:15:02.671685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660642    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 22:15:02.671685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660644    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 22:15:02.671685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660647    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 22:15:02.672482 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660649    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 22:15:02.672482 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660652    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 22:15:02.672482 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660655    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 22:15:02.672482 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660658    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 22:15:02.672482 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660660    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 22:15:02.672482 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660663    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 22:15:02.672482 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660666    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 22:15:02.672482 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660668    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 22:15:02.672482 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660672    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 22:15:02.672482 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660675    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 22:15:02.672482 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660678    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 22:15:02.672482 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660680    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 22:15:02.672482 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660683    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 22:15:02.672482 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660685    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 22:15:02.672482 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660688    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 22:15:02.672482 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660690    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 22:15:02.672482 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660692    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 22:15:02.672482 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660695    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 22:15:02.672482 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660697    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 22:15:02.672482 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660700    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 22:15:02.673185 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660703    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 22:15:02.673185 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660705    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 22:15:02.673185 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660708    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 22:15:02.673185 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660710    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 22:15:02.673185 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660714    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 22:15:02.673185 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660716    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 22:15:02.673185 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660719    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 22:15:02.673185 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660721    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 22:15:02.673185 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660724    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 22:15:02.673185 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660726    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 22:15:02.673185 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660729    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 22:15:02.673185 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660731    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 22:15:02.673185 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660735    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 22:15:02.673185 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660738    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 22:15:02.673185 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660741    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 22:15:02.673185 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660743    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 22:15:02.673185 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660745    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 22:15:02.673185 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660748    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 22:15:02.673185 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660751    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 22:15:02.673185 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660754    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 22:15:02.673670 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660759    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 22:15:02.673670 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.660763    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 22:15:02.673670 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.661351    2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 22:15:02.673670 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.669404    2555 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 07 22:15:02.673670 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.669427    2555 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 07 22:15:02.673670 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.669456    2555 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 07 22:15:02.673670 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.669465    2555 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 07 22:15:02.673670 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669547    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 22:15:02.673670 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669555    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 22:15:02.673670 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669560    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 22:15:02.673670 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669566    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 22:15:02.673670 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669571    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 22:15:02.673670 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669576    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 22:15:02.673670 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669580    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 22:15:02.674045 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669585    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 22:15:02.674045 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669590    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 22:15:02.674045 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669594    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 22:15:02.674045 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669599    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 22:15:02.674045 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669603    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 22:15:02.674045 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669608    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 22:15:02.674045 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669613    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 22:15:02.674045 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669620    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 22:15:02.674045 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669627    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 22:15:02.674045 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669632    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 22:15:02.674045 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669637    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 22:15:02.674045 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669642    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 22:15:02.674045 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669646    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 22:15:02.674045 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669650    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 22:15:02.674045 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669654    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 22:15:02.674045 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669658    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 22:15:02.674045 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669662    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 22:15:02.674045 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669667    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 22:15:02.674045 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669671    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 22:15:02.674494 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669675    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 22:15:02.674494 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669679    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 22:15:02.674494 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669683    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 22:15:02.674494 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669689    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 22:15:02.674494 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669693    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 22:15:02.674494 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669698    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 22:15:02.674494 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669702    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 22:15:02.674494 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669706    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 22:15:02.674494 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669711    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 22:15:02.674494 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669715    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 22:15:02.674494 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669719    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 22:15:02.674494 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669723    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 22:15:02.674494 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669727    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 22:15:02.674494 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669732    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 22:15:02.674494 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669736    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 22:15:02.674494 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669740    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 22:15:02.674494 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669744    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 22:15:02.674494 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669748    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 22:15:02.674494 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669752    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 22:15:02.674494 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669757    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 22:15:02.675111 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669761    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 22:15:02.675111 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669766    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 22:15:02.675111 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669770    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 22:15:02.675111 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669774    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 22:15:02.675111 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669778    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 22:15:02.675111 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669783    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 22:15:02.675111 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669788    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 22:15:02.675111 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669792    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 22:15:02.675111 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669796    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 22:15:02.675111 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669802    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 22:15:02.675111 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669808    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 22:15:02.675111 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669812    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 22:15:02.675111 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669816    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 22:15:02.675111 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669821    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 22:15:02.675111 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669825    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 22:15:02.675111 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669829    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 22:15:02.675111 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669835    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 22:15:02.675111 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669839    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 22:15:02.675111 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669844    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 22:15:02.675111 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669848    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 22:15:02.675619 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669852    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 22:15:02.675619 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669856    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 22:15:02.675619 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669861    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 22:15:02.675619 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669882    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 22:15:02.675619 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669887    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 22:15:02.675619 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669893    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 22:15:02.675619 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669897    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 22:15:02.675619 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669901    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 22:15:02.675619 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669906    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 22:15:02.675619 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669910    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 22:15:02.675619 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669914    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 22:15:02.675619 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669918    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 22:15:02.675619 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669922    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 22:15:02.675619 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669926    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 22:15:02.675619 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669930    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 22:15:02.675619 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669935    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 22:15:02.675619 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669939    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 22:15:02.675619 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669943    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 22:15:02.675619 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669947    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 22:15:02.675619 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669951    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 22:15:02.676266 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669956    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 22:15:02.676266 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669963    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 22:15:02.676266 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669968    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 22:15:02.676266 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669973    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 22:15:02.676266 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669978    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 22:15:02.676266 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669982    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 22:15:02.676266 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669986    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 22:15:02.676266 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669991    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 22:15:02.676266 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.669995    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 22:15:02.676266 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670000    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 22:15:02.676266 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670004    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 22:15:02.676266 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670008    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 22:15:02.676266 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670012    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 22:15:02.676266 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.670021    2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 22:15:02.676266 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670198    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 22:15:02.676685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670205    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 22:15:02.676685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670210    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 22:15:02.676685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670214    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 22:15:02.676685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670219    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 22:15:02.676685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670223    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 22:15:02.676685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670228    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 22:15:02.676685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670232    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 22:15:02.676685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670236    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 22:15:02.676685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670240    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 22:15:02.676685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670244    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 22:15:02.676685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670249    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 22:15:02.676685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670253    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 22:15:02.676685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670257    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 22:15:02.676685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670261    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 22:15:02.676685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670265    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 22:15:02.676685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670269    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 22:15:02.676685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670273    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 22:15:02.676685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670279    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 22:15:02.676685 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670286    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 22:15:02.677158 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670292    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 22:15:02.677158 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670297    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 22:15:02.677158 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670301    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 22:15:02.677158 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670305    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 22:15:02.677158 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670310    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 22:15:02.677158 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670314    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 22:15:02.677158 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670318    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 22:15:02.677158 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670323    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 22:15:02.677158 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670329    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 22:15:02.677158 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670333    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 22:15:02.677158 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670338    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 22:15:02.677158 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670342    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 22:15:02.677158 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670346    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 22:15:02.677158 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670351    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 22:15:02.677158 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670355    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 22:15:02.677158 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670359    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 22:15:02.677158 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670363    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 22:15:02.677158 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670367    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 22:15:02.677158 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670371    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 22:15:02.677158 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670375    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 22:15:02.677690 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670379    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 22:15:02.677690 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670384    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 22:15:02.677690 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670388    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 22:15:02.677690 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670392    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 22:15:02.677690 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670396    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 22:15:02.677690 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670400    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 22:15:02.677690 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670405    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 22:15:02.677690 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670408    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 22:15:02.677690 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670412    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 22:15:02.677690 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670417    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 22:15:02.677690 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670421    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 22:15:02.677690 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670425    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 22:15:02.677690 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670430    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 22:15:02.677690 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670434    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 22:15:02.677690 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670438    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 22:15:02.677690 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670442    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 22:15:02.677690 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670446    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 22:15:02.677690 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670450    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 22:15:02.677690 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670454    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 22:15:02.677690 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670458    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 22:15:02.678203 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670462    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 22:15:02.678203 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670468    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 22:15:02.678203 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670472    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 22:15:02.678203 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670476    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 22:15:02.678203 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670481    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 22:15:02.678203 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670485    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 22:15:02.678203 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670489    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 22:15:02.678203 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670493    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 22:15:02.678203 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670497    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 22:15:02.678203 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670501    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 22:15:02.678203 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670506    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 22:15:02.678203 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670510    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 22:15:02.678203 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670514    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 22:15:02.678203 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670519    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 22:15:02.678203 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670523    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 22:15:02.678203 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670527    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 22:15:02.678203 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670531    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 22:15:02.678203 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670537    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 22:15:02.678203 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670543    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 22:15:02.678671 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670547    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 22:15:02.678671 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670553    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 22:15:02.678671 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670557    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 22:15:02.678671 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670561    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 22:15:02.678671 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670565    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 22:15:02.678671 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670569    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 22:15:02.678671 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670574    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 22:15:02.678671 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670578    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 22:15:02.678671 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670582    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 22:15:02.678671 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670586    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 22:15:02.678671 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670591    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 22:15:02.678671 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670595    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 22:15:02.678671 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670599    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 22:15:02.678671 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670603    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 22:15:02.678671 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670607    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 22:15:02.678671 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670612    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 22:15:02.678671 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670617    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 22:15:02.678671 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670621    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 22:15:02.678671 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670625    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 22:15:02.678671 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:02.670629    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 22:15:02.679206 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.670636    2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 22:15:02.679206 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.670792    2555 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 07 22:15:02.679206 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.674810    2555 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 07 22:15:02.679206 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.676402    2555 server.go:1019] "Starting client certificate rotation"
Jul 07 22:15:02.679206 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.676510    2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 22:15:02.679206 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.676569    2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 22:15:02.679854 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.679835    2555 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 22:15:02.682408 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.682385    2555 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 22:15:02.695363 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.695345    2555 log.go:25] "Validated CRI v1 runtime API"
Jul 07 22:15:02.697324 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.697309    2555 log.go:25] "Validated CRI v1 image API"
Jul 07 22:15:02.698121 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.698101    2555 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 07 22:15:02.704554 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.704535    2555 fs.go:135] Filesystem UUIDs: map[6dce8b82-a302-4b39-88da-7d7090170bf4:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 eda4cbfe-08f7-4912-898e-b807436824f2:/dev/nvme0n1p3]
Jul 07 22:15:02.704640 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.704553    2555 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 07 22:15:02.707762 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.707712    2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 22:15:02.709575 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.709463    2555 manager.go:216] Machine: {Timestamp:2026-07-07 22:15:02.708185728 +0000 UTC m=+0.380404708 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3117106 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24d975676600f6efc3966dba999701 SystemUUID:ec24d975-6766-00f6-efc3-966dba999701 BootID:a3184902-82b7-4bcc-9211-d99d12a705ad Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6e:d4:4c:4e:5f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6e:d4:4c:4e:5f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:52:a3:ba:53:3a:07 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 07 22:15:02.709575 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.709568    2555 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 07 22:15:02.709676 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.709646    2555 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 07 22:15:02.713214 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.713191    2555 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 07 22:15:02.713348 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.713218    2555 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-159.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 07 22:15:02.713992 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.713970    2555 topology_manager.go:138] "Creating topology manager with none policy"
Jul 07 22:15:02.713992 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.713994    2555 container_manager_linux.go:309] "Creating device plugin manager"
Jul 07 22:15:02.714061 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.714002    2555 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 22:15:02.714061 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.714015    2555 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 22:15:02.715335 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.715325    2555 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 07 22:15:02.720083 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.720071    2555 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 22:15:02.720215 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.720204    2555 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 07 22:15:02.720251 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.720238    2555 kubelet.go:486] "Attempting to sync node with API server"
Jul 07 22:15:02.720279 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.720253    2555 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 07 22:15:02.720279 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.720275    2555 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 07 22:15:02.720331 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.720288    2555 kubelet.go:398] "Adding apiserver pod source"
Jul 07 22:15:02.720331 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.720310    2555 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 07 22:15:02.720737 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.720725    2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 22:15:02.720789 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.720744    2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 22:15:02.723030 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.723014    2555 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 07 22:15:02.725570 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.725555    2555 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 07 22:15:02.725631 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.725587    2555 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 07 22:15:02.725744 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.725710    2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8tlzk"
Jul 07 22:15:02.726802 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.726789    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 07 22:15:02.726843 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.726807    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 07 22:15:02.726843 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.726812    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 07 22:15:02.726843 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.726818    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 07 22:15:02.726843 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.726826    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 07 22:15:02.726843 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.726832    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 07 22:15:02.726843 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.726843    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 07 22:15:02.727022 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.726851    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 07 22:15:02.727022 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.726860    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 07 22:15:02.727022 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.726865    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 07 22:15:02.727022 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.726888    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 07 22:15:02.727022 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.726899    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 07 22:15:02.727664 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.727653    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 07 22:15:02.727664 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.727665    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 07 22:15:02.730362 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.730334    2555 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-159.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 07 22:15:02.730433 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:02.730346    2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-159.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 07 22:15:02.730433 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:02.730398    2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 07 22:15:02.731155 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.731141    2555 server.go:1268] "Started kubelet"
Jul 07 22:15:02.731373 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.731228    2555 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 07 22:15:02.732981 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.731224    2555 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 07 22:15:02.733075 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.733046    2555 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 07 22:15:02.733309 ip-10-0-128-159 systemd[1]: Started Kubernetes Kubelet.
Jul 07 22:15:02.734547 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.734520    2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8tlzk"
Jul 07 22:15:02.736604 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.736587    2555 server.go:310] "Adding debug handlers to kubelet server"
Jul 07 22:15:02.737376 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.737354    2555 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 07 22:15:02.741813 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:02.741790    2555 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 07 22:15:02.744180 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.744165    2555 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 07 22:15:02.744243 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.744177    2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 07 22:15:02.744583 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:02.744307    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-159.ec2.internal\" not found"
Jul 07 22:15:02.744583 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:02.743063    2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-159.ec2.internal.18c021b271846ef9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-159.ec2.internal,UID:ip-10-0-128-159.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-159.ec2.internal,},FirstTimestamp:2026-07-07 22:15:02.731116281 +0000 UTC m=+0.403335262,LastTimestamp:2026-07-07 22:15:02.731116281 +0000 UTC m=+0.403335262,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-159.ec2.internal,}"
Jul 07 22:15:02.744843 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.744826    2555 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 07 22:15:02.744980 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.744964    2555 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 07 22:15:02.745081 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.745032    2555 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 07 22:15:02.745081 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.745044    2555 factory.go:55] Registering systemd factory
Jul 07 22:15:02.745081 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.745053    2555 factory.go:223] Registration of the systemd container factory successfully
Jul 07 22:15:02.745232 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.745088    2555 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 07 22:15:02.745232 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.745093    2555 reconstruct.go:97] "Volume reconstruction finished"
Jul 07 22:15:02.745232 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.745124    2555 reconciler.go:29] "Reconciler: start to sync state"
Jul 07 22:15:02.745362 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.745287    2555 factory.go:153] Registering CRI-O factory
Jul 07 22:15:02.745362 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.745298    2555 factory.go:223] Registration of the crio container factory successfully
Jul 07 22:15:02.745362 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.745317    2555 factory.go:103] Registering Raw factory
Jul 07 22:15:02.745362 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.745326    2555 manager.go:1196] Started watching for new ooms in manager
Jul 07 22:15:02.745930 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.745918    2555 manager.go:349] Starting recovery of all containers
Jul 07 22:15:02.755751 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.755582    2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 22:15:02.756637 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.756621    2555 manager.go:354] Recovery completed
Jul 07 22:15:02.757881 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:02.757847    2555 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 07 22:15:02.758754 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:02.758736    2555 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-159.ec2.internal\" not found" node="ip-10-0-128-159.ec2.internal"
Jul 07 22:15:02.761452 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.761438    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 22:15:02.763351 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.763333    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 22:15:02.763444 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.763366    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 22:15:02.763444 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.763377    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeHasSufficientPID"
Jul 07 22:15:02.763911 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.763899    2555 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 07 22:15:02.763974 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.763911    2555 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 07 22:15:02.763974 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.763930    2555 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 22:15:02.763974 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.763965    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 22:15:02.766111 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.766099    2555 policy_none.go:49] "None policy: Start"
Jul 07 22:15:02.766158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.766114    2555 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 07 22:15:02.766158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.766124    2555 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 07 22:15:02.767018 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.767003    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 22:15:02.767065 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.767034    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 22:15:02.767065 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.767047    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeHasSufficientPID"
Jul 07 22:15:02.768124 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.768096    2555 policy_none.go:47] "Start"
Jul 07 22:15:02.807422 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.807403    2555 manager.go:337] "Starting Device Plugin manager"
Jul 07 22:15:02.807510 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:02.807436    2555 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 07 22:15:02.807510 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.807446    2555 server.go:87] "Starting device plugin registration server"
Jul 07 22:15:02.807702 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.807687    2555 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 07 22:15:02.807817 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.807703    2555 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 07 22:15:02.807817 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.807791    2555 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 07 22:15:02.807920 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.807907    2555 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 07 22:15:02.807920 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.807918    2555 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 07 22:15:02.808535 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:02.808518    2555 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 07 22:15:02.808594 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:02.808559    2555 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-159.ec2.internal\" not found"
Jul 07 22:15:02.908365 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.908325    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 22:15:02.910019 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.909274    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 22:15:02.910019 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.909306    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 22:15:02.910019 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.909317    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeHasSufficientPID"
Jul 07 22:15:02.910019 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.909343    2555 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-159.ec2.internal"
Jul 07 22:15:02.915094 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.915069    2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 07 22:15:02.916344 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.916305    2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 07 22:15:02.916447 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.916348    2555 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 07 22:15:02.916447 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.916372    2555 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 07 22:15:02.916447 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:02.916412    2555 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 07 22:15:02.917353 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.917340    2555 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-159.ec2.internal"
Jul 07 22:15:02.917443 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:02.917363    2555 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-159.ec2.internal\": node \"ip-10-0-128-159.ec2.internal\" not found"
Jul 07 22:15:02.918932 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:02.918914    2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 22:15:02.958909 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:02.958831    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-159.ec2.internal\" not found"
Jul 07 22:15:03.016958 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.016927    2555 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-159.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-159.ec2.internal"]
Jul 07 22:15:03.017083 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.017009    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 22:15:03.018597 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.018582    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 22:15:03.018667 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.018609    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 22:15:03.018667 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.018619    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeHasSufficientPID"
Jul 07 22:15:03.020545 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.020532    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 22:15:03.020685 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.020657    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-159.ec2.internal"
Jul 07 22:15:03.020727 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.020691    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 22:15:03.021256 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.021240    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 22:15:03.021325 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.021246    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 22:15:03.021325 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.021273    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 22:15:03.021325 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.021288    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeHasSufficientPID"
Jul 07 22:15:03.021325 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.021289    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 22:15:03.021325 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.021312    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeHasSufficientPID"
Jul 07 22:15:03.023168 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.023154    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-159.ec2.internal"
Jul 07 22:15:03.023226 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.023178    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 22:15:03.023793 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.023778    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 22:15:03.023884 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.023810    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 22:15:03.023884 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.023823    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeHasSufficientPID"
Jul 07 22:15:03.038395 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:03.038377    2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-159.ec2.internal\" not found" node="ip-10-0-128-159.ec2.internal"
Jul 07 22:15:03.042172 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:03.042157    2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-159.ec2.internal\" not found" node="ip-10-0-128-159.ec2.internal"
Jul 07 22:15:03.047361 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.047346    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e236da3c0fadbab47c6a6accf20cc40f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-159.ec2.internal\" (UID: \"e236da3c0fadbab47c6a6accf20cc40f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-159.ec2.internal"
Jul 07 22:15:03.047409 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.047369    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e236da3c0fadbab47c6a6accf20cc40f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-159.ec2.internal\" (UID: \"e236da3c0fadbab47c6a6accf20cc40f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-159.ec2.internal"
Jul 07 22:15:03.047409 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.047394    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f99cc9ba576df91ea14d32094adce059-config\") pod \"kube-apiserver-proxy-ip-10-0-128-159.ec2.internal\" (UID: \"f99cc9ba576df91ea14d32094adce059\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-159.ec2.internal"
Jul 07 22:15:03.059076 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:03.059061    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-159.ec2.internal\" not found"
Jul 07 22:15:03.147715 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.147689    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e236da3c0fadbab47c6a6accf20cc40f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-159.ec2.internal\" (UID: \"e236da3c0fadbab47c6a6accf20cc40f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-159.ec2.internal"
Jul 07 22:15:03.147818 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.147717    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e236da3c0fadbab47c6a6accf20cc40f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-159.ec2.internal\" (UID: \"e236da3c0fadbab47c6a6accf20cc40f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-159.ec2.internal"
Jul 07 22:15:03.147818 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.147745    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f99cc9ba576df91ea14d32094adce059-config\") pod \"kube-apiserver-proxy-ip-10-0-128-159.ec2.internal\" (UID: \"f99cc9ba576df91ea14d32094adce059\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-159.ec2.internal"
Jul 07 22:15:03.147818 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.147803    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e236da3c0fadbab47c6a6accf20cc40f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-159.ec2.internal\" (UID: \"e236da3c0fadbab47c6a6accf20cc40f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-159.ec2.internal"
Jul 07 22:15:03.147933 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.147854    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f99cc9ba576df91ea14d32094adce059-config\") pod \"kube-apiserver-proxy-ip-10-0-128-159.ec2.internal\" (UID: \"f99cc9ba576df91ea14d32094adce059\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-159.ec2.internal"
Jul 07 22:15:03.147933 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.147898    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e236da3c0fadbab47c6a6accf20cc40f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-159.ec2.internal\" (UID: \"e236da3c0fadbab47c6a6accf20cc40f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-159.ec2.internal"
Jul 07 22:15:03.159428 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:03.159409    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-159.ec2.internal\" not found"
Jul 07 22:15:03.260326 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:03.260260    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-159.ec2.internal\" not found"
Jul 07 22:15:03.343491 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.343456    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-159.ec2.internal"
Jul 07 22:15:03.345452 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.345438    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-159.ec2.internal"
Jul 07 22:15:03.360993 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:03.360971    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-159.ec2.internal\" not found"
Jul 07 22:15:03.461498 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:03.461447    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-159.ec2.internal\" not found"
Jul 07 22:15:03.562038 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:03.561976    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-159.ec2.internal\" not found"
Jul 07 22:15:03.662476 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:03.662442    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-159.ec2.internal\" not found"
Jul 07 22:15:03.675622 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.675596    2555 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 07 22:15:03.675747 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.675729    2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 22:15:03.675802 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.675760    2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 22:15:03.745697 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.745647    2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 22:10:02 +0000 UTC" deadline="2027-12-31 22:48:58.937096459 +0000 UTC"
Jul 07 22:15:03.745697 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.745688    2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13008h33m55.191412119s"
Jul 07 22:15:03.762923 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:03.762891    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-159.ec2.internal\" not found"
Jul 07 22:15:03.832773 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.832711    2555 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 22:15:03.844460 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.844438    2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-159.ec2.internal"
Jul 07 22:15:03.866161 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.866131    2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 22:15:03.868038 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.868021    2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-159.ec2.internal"
Jul 07 22:15:03.886758 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.886731    2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 22:15:03.965514 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:03.965479    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf99cc9ba576df91ea14d32094adce059.slice/crio-2ab0fa123339d18631fcdb96991cc0fa8de41d280300fc633eeefd41e642705f WatchSource:0}: Error finding container 2ab0fa123339d18631fcdb96991cc0fa8de41d280300fc633eeefd41e642705f: Status 404 returned error can't find the container with id 2ab0fa123339d18631fcdb96991cc0fa8de41d280300fc633eeefd41e642705f
Jul 07 22:15:03.966123 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:03.966100    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode236da3c0fadbab47c6a6accf20cc40f.slice/crio-6bd3a24dbe75425176a7c20560eeb87028e589ee85a1f8fdfcf7b5f7d2f51e2a WatchSource:0}: Error finding container 6bd3a24dbe75425176a7c20560eeb87028e589ee85a1f8fdfcf7b5f7d2f51e2a: Status 404 returned error can't find the container with id 6bd3a24dbe75425176a7c20560eeb87028e589ee85a1f8fdfcf7b5f7d2f51e2a
Jul 07 22:15:03.973138 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:03.973119    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 22:15:04.152041 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.151958    2555 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 22:15:04.721620 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.721573    2555 apiserver.go:52] "Watching apiserver"
Jul 07 22:15:04.729308 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.729277    2555 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 07 22:15:04.731040 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.730691    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-hgwst","kube-system/kube-apiserver-proxy-ip-10-0-128-159.ec2.internal","openshift-image-registry/node-ca-vwjdg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-159.ec2.internal","openshift-multus/multus-7x2ml","openshift-multus/network-metrics-daemon-58clx","openshift-network-diagnostics/network-check-target-nbgl7","openshift-network-operator/iptables-alerter-bnnmv","kube-system/global-pull-secret-syncer-hdtt4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf","openshift-cluster-node-tuning-operator/tuned-b5wpj","openshift-dns/node-resolver-t8kwx","openshift-multus/multus-additional-cni-plugins-kz7pz","openshift-ovn-kubernetes/ovnkube-node-mbznv"]
Jul 07 22:15:04.733383 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.733359    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vwjdg"
Jul 07 22:15:04.736232 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.735682    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.736232 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.735910    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 07 22:15:04.736232 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.735909    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 07 22:15:04.736232 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.736071    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qx8hv\""
Jul 07 22:15:04.736482 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.736390    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 07 22:15:04.739767 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.738475    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 07 22:15:04.739767 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.738711    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 07 22:15:04.739767 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.738967    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2zj2b\""
Jul 07 22:15:04.739767 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.739140    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 07 22:15:04.739767 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.739326    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 07 22:15:04.739767 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.739555    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:04.739767 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.739562    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 07 22:15:04.739767 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:04.739616    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbgl7" podUID="c248b400-7255-477d-b130-6daf8344f786"
Jul 07 22:15:04.740440 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.740380    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.742784 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.742744    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6xm7k\""
Jul 07 22:15:04.743009 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.742991    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 07 22:15:04.743204 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.743186    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:04.743364 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:04.743325    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58clx" podUID="28fa7951-7615-4e16-96f8-c918ab062cd3"
Jul 07 22:15:04.745718 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.744861    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hgwst"
Jul 07 22:15:04.745718 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.745115    2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 07 22:15:04.747111 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.747093    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bnnmv"
Jul 07 22:15:04.748056 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.747898    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 07 22:15:04.748056 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.747960    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-f9vtg\""
Jul 07 22:15:04.748312 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.748295    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 07 22:15:04.749665 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.749458    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 07 22:15:04.749665 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.749575    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:04.749665 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:04.749643    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdtt4" podUID="4128cbcd-5ec2-40d9-81fa-9661440e82df"
Jul 07 22:15:04.749883 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.749745    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 07 22:15:04.749883 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.749807    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vg877\""
Jul 07 22:15:04.751032 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.750034    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 07 22:15:04.752289 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.751839    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:04.753827 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.753807    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.754729 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.754252    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 07 22:15:04.754729 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.754328    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 07 22:15:04.754729 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.754477    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7pzzc\""
Jul 07 22:15:04.754729 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.754548    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-multus-daemon-config\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.754729 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.754568    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 07 22:15:04.754729 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.754580    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/111c1daf-0cbf-4a17-bc99-92cb26144a4e-iptables-alerter-script\") pod \"iptables-alerter-bnnmv\" (UID: \"111c1daf-0cbf-4a17-bc99-92cb26144a4e\") " pod="openshift-network-operator/iptables-alerter-bnnmv"
Jul 07 22:15:04.754729 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.754607    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-multus-socket-dir-parent\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.754729 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.754640    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-host-run-k8s-cni-cncf-io\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.754729 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.754661    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-host-var-lib-cni-bin\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.754729 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.754698    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/621e2b82-009d-465d-940a-57c8bff5595d-host\") pod \"node-ca-vwjdg\" (UID: \"621e2b82-009d-465d-940a-57c8bff5595d\") " pod="openshift-image-registry/node-ca-vwjdg"
Jul 07 22:15:04.754729 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.754723    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/621e2b82-009d-465d-940a-57c8bff5595d-serviceca\") pod \"node-ca-vwjdg\" (UID: \"621e2b82-009d-465d-940a-57c8bff5595d\") " pod="openshift-image-registry/node-ca-vwjdg"
Jul 07 22:15:04.755321 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.754746    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-cnibin\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.755321 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.754769    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-system-cni-dir\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.755321 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.754796    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.755321 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.754836    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.755321 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.754860    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-multus-cni-dir\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.755321 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.754902    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-os-release\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.755321 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.754936    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-multus-conf-dir\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.755321 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.754959    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/111c1daf-0cbf-4a17-bc99-92cb26144a4e-host-slash\") pod \"iptables-alerter-bnnmv\" (UID: \"111c1daf-0cbf-4a17-bc99-92cb26144a4e\") " pod="openshift-network-operator/iptables-alerter-bnnmv"
Jul 07 22:15:04.755321 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.754983    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-cni-binary-copy\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.755321 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.755007    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r72xh\" (UniqueName: \"kubernetes.io/projected/621e2b82-009d-465d-940a-57c8bff5595d-kube-api-access-r72xh\") pod \"node-ca-vwjdg\" (UID: \"621e2b82-009d-465d-940a-57c8bff5595d\") " pod="openshift-image-registry/node-ca-vwjdg"
Jul 07 22:15:04.755321 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.755049    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1155169f-bea8-43e4-a927-668ec8c8e77f-agent-certs\") pod \"konnectivity-agent-hgwst\" (UID: \"1155169f-bea8-43e4-a927-668ec8c8e77f\") " pod="kube-system/konnectivity-agent-hgwst"
Jul 07 22:15:04.755321 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.755085    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd995\" (UniqueName: \"kubernetes.io/projected/111c1daf-0cbf-4a17-bc99-92cb26144a4e-kube-api-access-nd995\") pod \"iptables-alerter-bnnmv\" (UID: \"111c1daf-0cbf-4a17-bc99-92cb26144a4e\") " pod="openshift-network-operator/iptables-alerter-bnnmv"
Jul 07 22:15:04.755321 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.755111    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-os-release\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.755321 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.755151    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5bj4\" (UniqueName: \"kubernetes.io/projected/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-kube-api-access-b5bj4\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.755321 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.755175    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-cnibin\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.755321 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.755197    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-hostroot\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.756086 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.755220    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxdbf\" (UniqueName: \"kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf\") pod \"network-check-target-nbgl7\" (UID: \"c248b400-7255-477d-b130-6daf8344f786\") " pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:04.756086 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.755244    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-host-var-lib-cni-multus\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.756086 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.755268    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-cni-binary-copy\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.756086 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.755292    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-host-run-netns\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.756086 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.755314    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-host-run-multus-certs\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.756086 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.755342    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-etc-kubernetes\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.756086 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.755377    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wffg8\" (UniqueName: \"kubernetes.io/projected/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-kube-api-access-wffg8\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.756086 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.755400    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs\") pod \"network-metrics-daemon-58clx\" (UID: \"28fa7951-7615-4e16-96f8-c918ab062cd3\") " pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:04.756086 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.755426    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkrg8\" (UniqueName: \"kubernetes.io/projected/28fa7951-7615-4e16-96f8-c918ab062cd3-kube-api-access-wkrg8\") pod \"network-metrics-daemon-58clx\" (UID: \"28fa7951-7615-4e16-96f8-c918ab062cd3\") " pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:04.756086 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.755451    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1155169f-bea8-43e4-a927-668ec8c8e77f-konnectivity-ca\") pod \"konnectivity-agent-hgwst\" (UID: \"1155169f-bea8-43e4-a927-668ec8c8e77f\") " pod="kube-system/konnectivity-agent-hgwst"
Jul 07 22:15:04.756086 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.755476    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-system-cni-dir\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.756086 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.755514    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.756086 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.755539    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-host-var-lib-kubelet\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.756086 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.755891    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 07 22:15:04.756714 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.756223    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4zltn\""
Jul 07 22:15:04.756714 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.756560    2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 22:15:04.760126 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.758599    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t8kwx"
Jul 07 22:15:04.760126 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.759154    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.760275 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.760167    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 07 22:15:04.763600 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.763559    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 07 22:15:04.763987 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.763932    2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 22:15:04.764257 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.764241    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 07 22:15:04.764327 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.764279    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 07 22:15:04.764464 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.764448    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 07 22:15:04.764578 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.764537    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9rfnl\""
Jul 07 22:15:04.765529 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.765025    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 07 22:15:04.765529 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.765043    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 07 22:15:04.765529 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.765298    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lbq4q\""
Jul 07 22:15:04.765529 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.765300    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 07 22:15:04.765834 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.765588    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 07 22:15:04.779746 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.779576    2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-thxtr"
Jul 07 22:15:04.786416 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.786393    2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-thxtr"
Jul 07 22:15:04.846684 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.846662    2555 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 07 22:15:04.855823 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.855797    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-host-run-netns\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.855965 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.855835    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-host-run-multus-certs\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.855965 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.855881    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0f492bfc-2ea5-4bab-a34b-b6df6736724e-ovnkube-script-lib\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.855965 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.855911    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs\") pod \"network-metrics-daemon-58clx\" (UID: \"28fa7951-7615-4e16-96f8-c918ab062cd3\") " pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:04.855965 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.855935    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-host-run-multus-certs\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.855965 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.855946    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1155169f-bea8-43e4-a927-668ec8c8e77f-konnectivity-ca\") pod \"konnectivity-agent-hgwst\" (UID: \"1155169f-bea8-43e4-a927-668ec8c8e77f\") " pod="kube-system/konnectivity-agent-hgwst"
Jul 07 22:15:04.855965 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.855935    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-host-run-netns\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.856249 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856007    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.856249 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:04.856128    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 22:15:04.856249 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:04.856207    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs podName:28fa7951-7615-4e16-96f8-c918ab062cd3 nodeName:}" failed. No retries permitted until 2026-07-07 22:15:05.356172014 +0000 UTC m=+3.028390986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs") pod "network-metrics-daemon-58clx" (UID: "28fa7951-7615-4e16-96f8-c918ab062cd3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 22:15:04.856401 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856313    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-host-var-lib-kubelet\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.856401 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856345    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-multus-daemon-config\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.856401 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856373    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e12ec4b7-d42c-4cb9-8d31-769cae33e436-etc-selinux\") pod \"aws-ebs-csi-driver-node-sxdgf\" (UID: \"e12ec4b7-d42c-4cb9-8d31-769cae33e436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:04.856401 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856390    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-host-var-lib-kubelet\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.856401 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856397    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/001f8df1-bca6-441f-89d3-01bb8d3741b0-tmp-dir\") pod \"node-resolver-t8kwx\" (UID: \"001f8df1-bca6-441f-89d3-01bb8d3741b0\") " pod="openshift-dns/node-resolver-t8kwx"
Jul 07 22:15:04.856625 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856421    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs68k\" (UniqueName: \"kubernetes.io/projected/001f8df1-bca6-441f-89d3-01bb8d3741b0-kube-api-access-vs68k\") pod \"node-resolver-t8kwx\" (UID: \"001f8df1-bca6-441f-89d3-01bb8d3741b0\") " pod="openshift-dns/node-resolver-t8kwx"
Jul 07 22:15:04.856625 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856462    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/111c1daf-0cbf-4a17-bc99-92cb26144a4e-iptables-alerter-script\") pod \"iptables-alerter-bnnmv\" (UID: \"111c1daf-0cbf-4a17-bc99-92cb26144a4e\") " pod="openshift-network-operator/iptables-alerter-bnnmv"
Jul 07 22:15:04.856625 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856529    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-host-run-k8s-cni-cncf-io\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.856625 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856559    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.856625 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856561    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1155169f-bea8-43e4-a927-668ec8c8e77f-konnectivity-ca\") pod \"konnectivity-agent-hgwst\" (UID: \"1155169f-bea8-43e4-a927-668ec8c8e77f\") " pod="kube-system/konnectivity-agent-hgwst"
Jul 07 22:15:04.856625 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856595    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-host-var-lib-cni-bin\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.856625 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856623    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-host-var-lib-cni-bin\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.856625 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856625    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e12ec4b7-d42c-4cb9-8d31-769cae33e436-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sxdgf\" (UID: \"e12ec4b7-d42c-4cb9-8d31-769cae33e436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:04.857075 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856599    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-host-run-k8s-cni-cncf-io\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.857075 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856654    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-lib-modules\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.857075 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856677    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc0b02e6-393f-4ba4-8708-25daa2922465-tmp\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.857075 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856702    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/621e2b82-009d-465d-940a-57c8bff5595d-host\") pod \"node-ca-vwjdg\" (UID: \"621e2b82-009d-465d-940a-57c8bff5595d\") " pod="openshift-image-registry/node-ca-vwjdg"
Jul 07 22:15:04.857075 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856726    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq975\" (UniqueName: \"kubernetes.io/projected/bc0b02e6-393f-4ba4-8708-25daa2922465-kube-api-access-zq975\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.857075 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856750    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-var-lib-openvswitch\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.857075 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856761    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/621e2b82-009d-465d-940a-57c8bff5595d-host\") pod \"node-ca-vwjdg\" (UID: \"621e2b82-009d-465d-940a-57c8bff5595d\") " pod="openshift-image-registry/node-ca-vwjdg"
Jul 07 22:15:04.857075 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856774    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0f492bfc-2ea5-4bab-a34b-b6df6736724e-ovn-node-metrics-cert\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.857075 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856801    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.857075 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856848    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.857075 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856888    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-os-release\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.857075 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856914    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4128cbcd-5ec2-40d9-81fa-9661440e82df-dbus\") pod \"global-pull-secret-syncer-hdtt4\" (UID: \"4128cbcd-5ec2-40d9-81fa-9661440e82df\") " pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:04.857075 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856918    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-multus-daemon-config\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.857075 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856936    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-etc-sysconfig\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.857075 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856959    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-node-log\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.857075 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.856982    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0f492bfc-2ea5-4bab-a34b-b6df6736724e-ovnkube-config\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.857075 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857003    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-os-release\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.857806 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857007    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.857806 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857025    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-cni-binary-copy\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.857806 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857030    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/111c1daf-0cbf-4a17-bc99-92cb26144a4e-iptables-alerter-script\") pod \"iptables-alerter-bnnmv\" (UID: \"111c1daf-0cbf-4a17-bc99-92cb26144a4e\") " pod="openshift-network-operator/iptables-alerter-bnnmv"
Jul 07 22:15:04.857806 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857053    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-etc-openvswitch\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.857806 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857083    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r72xh\" (UniqueName: \"kubernetes.io/projected/621e2b82-009d-465d-940a-57c8bff5595d-kube-api-access-r72xh\") pod \"node-ca-vwjdg\" (UID: \"621e2b82-009d-465d-940a-57c8bff5595d\") " pod="openshift-image-registry/node-ca-vwjdg"
Jul 07 22:15:04.857806 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857113    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-os-release\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.857806 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857182    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-os-release\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.857806 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857299    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5bj4\" (UniqueName: \"kubernetes.io/projected/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-kube-api-access-b5bj4\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.857806 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857346    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.857806 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857367    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-cnibin\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.857806 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857396    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-hostroot\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.857806 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857436    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e12ec4b7-d42c-4cb9-8d31-769cae33e436-device-dir\") pod \"aws-ebs-csi-driver-node-sxdgf\" (UID: \"e12ec4b7-d42c-4cb9-8d31-769cae33e436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:04.857806 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857489    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-systemd-units\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.857806 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857488    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-cnibin\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.857806 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857526    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-hostroot\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.857806 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857533    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-cni-binary-copy\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.857806 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857564    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-host-cni-netd\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.858556 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857595    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-host-var-lib-cni-multus\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.858556 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857629    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4128cbcd-5ec2-40d9-81fa-9661440e82df-kubelet-config\") pod \"global-pull-secret-syncer-hdtt4\" (UID: \"4128cbcd-5ec2-40d9-81fa-9661440e82df\") " pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:04.858556 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857641    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-host-var-lib-cni-multus\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.858556 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857652    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-var-lib-kubelet\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.858556 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857674    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-host-kubelet\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.858556 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857718    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.858556 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857772    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgp4n\" (UniqueName: \"kubernetes.io/projected/0f492bfc-2ea5-4bab-a34b-b6df6736724e-kube-api-access-jgp4n\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.858556 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857812    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-etc-sysctl-conf\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.858556 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857844    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-cni-binary-copy\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.858556 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857882    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-etc-kubernetes\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.858556 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857908    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wffg8\" (UniqueName: \"kubernetes.io/projected/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-kube-api-access-wffg8\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.858556 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857931    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e12ec4b7-d42c-4cb9-8d31-769cae33e436-socket-dir\") pod \"aws-ebs-csi-driver-node-sxdgf\" (UID: \"e12ec4b7-d42c-4cb9-8d31-769cae33e436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:04.858556 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857954    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e12ec4b7-d42c-4cb9-8d31-769cae33e436-registration-dir\") pod \"aws-ebs-csi-driver-node-sxdgf\" (UID: \"e12ec4b7-d42c-4cb9-8d31-769cae33e436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:04.858556 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857976    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-etc-sysctl-d\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.858556 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.857998    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-host-run-ovn-kubernetes\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.858556 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858027    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-run\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.859315 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858060    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkrg8\" (UniqueName: \"kubernetes.io/projected/28fa7951-7615-4e16-96f8-c918ab062cd3-kube-api-access-wkrg8\") pod \"network-metrics-daemon-58clx\" (UID: \"28fa7951-7615-4e16-96f8-c918ab062cd3\") " pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:04.859315 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858089    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-system-cni-dir\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.859315 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858123    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-run-systemd\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.859315 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858145    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-run-ovn\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.859315 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858162    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-etc-kubernetes\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.859315 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858168    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-multus-socket-dir-parent\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.859315 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858190    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e12ec4b7-d42c-4cb9-8d31-769cae33e436-sys-fs\") pod \"aws-ebs-csi-driver-node-sxdgf\" (UID: \"e12ec4b7-d42c-4cb9-8d31-769cae33e436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:04.859315 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858228    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrpdq\" (UniqueName: \"kubernetes.io/projected/e12ec4b7-d42c-4cb9-8d31-769cae33e436-kube-api-access-xrpdq\") pod \"aws-ebs-csi-driver-node-sxdgf\" (UID: \"e12ec4b7-d42c-4cb9-8d31-769cae33e436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:04.859315 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858261    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/621e2b82-009d-465d-940a-57c8bff5595d-serviceca\") pod \"node-ca-vwjdg\" (UID: \"621e2b82-009d-465d-940a-57c8bff5595d\") " pod="openshift-image-registry/node-ca-vwjdg"
Jul 07 22:15:04.859315 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858361    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-cnibin\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.859315 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858378    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-cni-binary-copy\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.859315 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858395    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-system-cni-dir\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.859315 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858418    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-etc-modprobe-d\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.859315 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858523    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-multus-socket-dir-parent\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.859315 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858573    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-cnibin\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.859315 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858640    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-system-cni-dir\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.859315 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858745    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-system-cni-dir\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.860121 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858793    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/001f8df1-bca6-441f-89d3-01bb8d3741b0-hosts-file\") pod \"node-resolver-t8kwx\" (UID: \"001f8df1-bca6-441f-89d3-01bb8d3741b0\") " pod="openshift-dns/node-resolver-t8kwx"
Jul 07 22:15:04.860121 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858840    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-host-run-netns\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.860121 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858893    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-run-openvswitch\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.860121 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.858977    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-multus-cni-dir\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.860121 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.859016    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-multus-conf-dir\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.860121 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.859018    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/621e2b82-009d-465d-940a-57c8bff5595d-serviceca\") pod \"node-ca-vwjdg\" (UID: \"621e2b82-009d-465d-940a-57c8bff5595d\") " pod="openshift-image-registry/node-ca-vwjdg"
Jul 07 22:15:04.860121 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.859038    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bc0b02e6-393f-4ba4-8708-25daa2922465-etc-tuned\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.860121 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.859060    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0f492bfc-2ea5-4bab-a34b-b6df6736724e-env-overrides\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.860121 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.859077    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-multus-conf-dir\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.860121 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.859087    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/111c1daf-0cbf-4a17-bc99-92cb26144a4e-host-slash\") pod \"iptables-alerter-bnnmv\" (UID: \"111c1daf-0cbf-4a17-bc99-92cb26144a4e\") " pod="openshift-network-operator/iptables-alerter-bnnmv"
Jul 07 22:15:04.860121 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.859098    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-multus-cni-dir\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.860121 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.859145    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/111c1daf-0cbf-4a17-bc99-92cb26144a4e-host-slash\") pod \"iptables-alerter-bnnmv\" (UID: \"111c1daf-0cbf-4a17-bc99-92cb26144a4e\") " pod="openshift-network-operator/iptables-alerter-bnnmv"
Jul 07 22:15:04.860121 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.859180    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret\") pod \"global-pull-secret-syncer-hdtt4\" (UID: \"4128cbcd-5ec2-40d9-81fa-9661440e82df\") " pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:04.860121 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.859212    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-etc-systemd\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.860121 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.859296    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-host-slash\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.860121 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.859325    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-host-cni-bin\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.860121 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.859358    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1155169f-bea8-43e4-a927-668ec8c8e77f-agent-certs\") pod \"konnectivity-agent-hgwst\" (UID: \"1155169f-bea8-43e4-a927-668ec8c8e77f\") " pod="kube-system/konnectivity-agent-hgwst"
Jul 07 22:15:04.860899 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.859383    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd995\" (UniqueName: \"kubernetes.io/projected/111c1daf-0cbf-4a17-bc99-92cb26144a4e-kube-api-access-nd995\") pod \"iptables-alerter-bnnmv\" (UID: \"111c1daf-0cbf-4a17-bc99-92cb26144a4e\") " pod="openshift-network-operator/iptables-alerter-bnnmv"
Jul 07 22:15:04.860899 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.859689    2555 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 07 22:15:04.860899 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.859892    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-sys\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.860899 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.859919    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-log-socket\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.860899 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.859946    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxdbf\" (UniqueName: \"kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf\") pod \"network-check-target-nbgl7\" (UID: \"c248b400-7255-477d-b130-6daf8344f786\") " pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:04.860899 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.860194    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-etc-kubernetes\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.860899 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.860312    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-host\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.869417 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:04.865723    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 22:15:04.869417 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:04.865744    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 22:15:04.869417 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:04.865757    2555 projected.go:196] Error preparing data for projected volume kube-api-access-zxdbf for pod openshift-network-diagnostics/network-check-target-nbgl7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 22:15:04.869417 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:04.865889    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf podName:c248b400-7255-477d-b130-6daf8344f786 nodeName:}" failed. No retries permitted until 2026-07-07 22:15:05.365857807 +0000 UTC m=+3.038076788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zxdbf" (UniqueName: "kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf") pod "network-check-target-nbgl7" (UID: "c248b400-7255-477d-b130-6daf8344f786") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 22:15:04.869417 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.866119    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r72xh\" (UniqueName: \"kubernetes.io/projected/621e2b82-009d-465d-940a-57c8bff5595d-kube-api-access-r72xh\") pod \"node-ca-vwjdg\" (UID: \"621e2b82-009d-465d-940a-57c8bff5595d\") " pod="openshift-image-registry/node-ca-vwjdg"
Jul 07 22:15:04.869417 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.868957    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1155169f-bea8-43e4-a927-668ec8c8e77f-agent-certs\") pod \"konnectivity-agent-hgwst\" (UID: \"1155169f-bea8-43e4-a927-668ec8c8e77f\") " pod="kube-system/konnectivity-agent-hgwst"
Jul 07 22:15:04.870338 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.870046    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wffg8\" (UniqueName: \"kubernetes.io/projected/ad33be48-3fec-4f6d-9d2c-a01dd3e342df-kube-api-access-wffg8\") pod \"multus-7x2ml\" (UID: \"ad33be48-3fec-4f6d-9d2c-a01dd3e342df\") " pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:04.870338 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.870300    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkrg8\" (UniqueName: \"kubernetes.io/projected/28fa7951-7615-4e16-96f8-c918ab062cd3-kube-api-access-wkrg8\") pod \"network-metrics-daemon-58clx\" (UID: \"28fa7951-7615-4e16-96f8-c918ab062cd3\") " pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:04.870535 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.870513    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd995\" (UniqueName: \"kubernetes.io/projected/111c1daf-0cbf-4a17-bc99-92cb26144a4e-kube-api-access-nd995\") pod \"iptables-alerter-bnnmv\" (UID: \"111c1daf-0cbf-4a17-bc99-92cb26144a4e\") " pod="openshift-network-operator/iptables-alerter-bnnmv"
Jul 07 22:15:04.871132 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.871112    2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 22:15:04.872128 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.872102    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5bj4\" (UniqueName: \"kubernetes.io/projected/e32ca0b6-d3f9-4868-9b27-06e20b06a8ae-kube-api-access-b5bj4\") pod \"multus-additional-cni-plugins-kz7pz\" (UID: \"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae\") " pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:04.921421 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.921363    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-159.ec2.internal" event={"ID":"e236da3c0fadbab47c6a6accf20cc40f","Type":"ContainerStarted","Data":"6bd3a24dbe75425176a7c20560eeb87028e589ee85a1f8fdfcf7b5f7d2f51e2a"}
Jul 07 22:15:04.923066 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.923041    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-159.ec2.internal" event={"ID":"f99cc9ba576df91ea14d32094adce059","Type":"ContainerStarted","Data":"2ab0fa123339d18631fcdb96991cc0fa8de41d280300fc633eeefd41e642705f"}
Jul 07 22:15:04.961110 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961071    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-log-socket\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.961270 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961136    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-etc-kubernetes\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.961270 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961159    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-host\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.961270 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961188    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0f492bfc-2ea5-4bab-a34b-b6df6736724e-ovnkube-script-lib\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.961270 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961226    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e12ec4b7-d42c-4cb9-8d31-769cae33e436-etc-selinux\") pod \"aws-ebs-csi-driver-node-sxdgf\" (UID: \"e12ec4b7-d42c-4cb9-8d31-769cae33e436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:04.961270 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961250    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/001f8df1-bca6-441f-89d3-01bb8d3741b0-tmp-dir\") pod \"node-resolver-t8kwx\" (UID: \"001f8df1-bca6-441f-89d3-01bb8d3741b0\") " pod="openshift-dns/node-resolver-t8kwx"
Jul 07 22:15:04.961507 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961272    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs68k\" (UniqueName: \"kubernetes.io/projected/001f8df1-bca6-441f-89d3-01bb8d3741b0-kube-api-access-vs68k\") pod \"node-resolver-t8kwx\" (UID: \"001f8df1-bca6-441f-89d3-01bb8d3741b0\") " pod="openshift-dns/node-resolver-t8kwx"
Jul 07 22:15:04.961507 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961299    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e12ec4b7-d42c-4cb9-8d31-769cae33e436-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sxdgf\" (UID: \"e12ec4b7-d42c-4cb9-8d31-769cae33e436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:04.961507 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961322    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-lib-modules\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.961507 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961343    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc0b02e6-393f-4ba4-8708-25daa2922465-tmp\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.961507 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961368    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq975\" (UniqueName: \"kubernetes.io/projected/bc0b02e6-393f-4ba4-8708-25daa2922465-kube-api-access-zq975\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.961507 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961388    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-host\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.961507 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961390    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-var-lib-openvswitch\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.961507 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961428    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-var-lib-openvswitch\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.961507 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961441    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0f492bfc-2ea5-4bab-a34b-b6df6736724e-ovn-node-metrics-cert\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.961507 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961469    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4128cbcd-5ec2-40d9-81fa-9661440e82df-dbus\") pod \"global-pull-secret-syncer-hdtt4\" (UID: \"4128cbcd-5ec2-40d9-81fa-9661440e82df\") " pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:04.961507 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961490    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-etc-sysconfig\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.961507 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961508    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-node-log\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.962078 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961531    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0f492bfc-2ea5-4bab-a34b-b6df6736724e-ovnkube-config\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.962078 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961538    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e12ec4b7-d42c-4cb9-8d31-769cae33e436-etc-selinux\") pod \"aws-ebs-csi-driver-node-sxdgf\" (UID: \"e12ec4b7-d42c-4cb9-8d31-769cae33e436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:04.962078 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961556    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-etc-openvswitch\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.962078 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961587    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e12ec4b7-d42c-4cb9-8d31-769cae33e436-device-dir\") pod \"aws-ebs-csi-driver-node-sxdgf\" (UID: \"e12ec4b7-d42c-4cb9-8d31-769cae33e436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:04.962078 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961611    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-systemd-units\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.962078 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961631    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-host-cni-netd\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.962078 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961653    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4128cbcd-5ec2-40d9-81fa-9661440e82df-kubelet-config\") pod \"global-pull-secret-syncer-hdtt4\" (UID: \"4128cbcd-5ec2-40d9-81fa-9661440e82df\") " pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:04.962078 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961676    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-var-lib-kubelet\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.962078 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961699    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-host-kubelet\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.962078 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961724    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.962078 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961749    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgp4n\" (UniqueName: \"kubernetes.io/projected/0f492bfc-2ea5-4bab-a34b-b6df6736724e-kube-api-access-jgp4n\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.962078 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961773    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-etc-sysctl-conf\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.962078 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961805    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e12ec4b7-d42c-4cb9-8d31-769cae33e436-socket-dir\") pod \"aws-ebs-csi-driver-node-sxdgf\" (UID: \"e12ec4b7-d42c-4cb9-8d31-769cae33e436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:04.962078 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961826    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/001f8df1-bca6-441f-89d3-01bb8d3741b0-tmp-dir\") pod \"node-resolver-t8kwx\" (UID: \"001f8df1-bca6-441f-89d3-01bb8d3741b0\") " pod="openshift-dns/node-resolver-t8kwx"
Jul 07 22:15:04.962078 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961828    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e12ec4b7-d42c-4cb9-8d31-769cae33e436-registration-dir\") pod \"aws-ebs-csi-driver-node-sxdgf\" (UID: \"e12ec4b7-d42c-4cb9-8d31-769cae33e436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:04.962078 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961895    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e12ec4b7-d42c-4cb9-8d31-769cae33e436-registration-dir\") pod \"aws-ebs-csi-driver-node-sxdgf\" (UID: \"e12ec4b7-d42c-4cb9-8d31-769cae33e436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:04.962078 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961912    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-etc-sysctl-d\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.962835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961939    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-host-run-ovn-kubernetes\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.962835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961938    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-systemd-units\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.962835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961963    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-run\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.962835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961965    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0f492bfc-2ea5-4bab-a34b-b6df6736724e-ovnkube-script-lib\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.962835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961986    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-host-kubelet\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.962835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962008    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-run\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.962835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962025    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.962835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962011    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-run-systemd\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.962835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962064    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-run-ovn\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.962835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962077    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-etc-openvswitch\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.962835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962106    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e12ec4b7-d42c-4cb9-8d31-769cae33e436-sys-fs\") pod \"aws-ebs-csi-driver-node-sxdgf\" (UID: \"e12ec4b7-d42c-4cb9-8d31-769cae33e436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:04.962835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962130    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e12ec4b7-d42c-4cb9-8d31-769cae33e436-device-dir\") pod \"aws-ebs-csi-driver-node-sxdgf\" (UID: \"e12ec4b7-d42c-4cb9-8d31-769cae33e436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:04.962835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962173    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrpdq\" (UniqueName: \"kubernetes.io/projected/e12ec4b7-d42c-4cb9-8d31-769cae33e436-kube-api-access-xrpdq\") pod \"aws-ebs-csi-driver-node-sxdgf\" (UID: \"e12ec4b7-d42c-4cb9-8d31-769cae33e436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:04.962835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962211    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-log-socket\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.962835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962229    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-lib-modules\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.962835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962240    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-etc-modprobe-d\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.962835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962267    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/001f8df1-bca6-441f-89d3-01bb8d3741b0-hosts-file\") pod \"node-resolver-t8kwx\" (UID: \"001f8df1-bca6-441f-89d3-01bb8d3741b0\") " pod="openshift-dns/node-resolver-t8kwx"
Jul 07 22:15:04.963624 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962294    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-etc-sysctl-conf\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.963624 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962306    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-host-run-netns\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.963624 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962318    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e12ec4b7-d42c-4cb9-8d31-769cae33e436-socket-dir\") pod \"aws-ebs-csi-driver-node-sxdgf\" (UID: \"e12ec4b7-d42c-4cb9-8d31-769cae33e436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:04.963624 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962329    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-etc-sysctl-d\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.963624 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962344    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-run-ovn\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.963624 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962333    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-run-openvswitch\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.963624 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962362    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e12ec4b7-d42c-4cb9-8d31-769cae33e436-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sxdgf\" (UID: \"e12ec4b7-d42c-4cb9-8d31-769cae33e436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:04.963624 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962368    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-host-run-ovn-kubernetes\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.963624 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.961941    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-host-cni-netd\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.963624 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962382    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-run-systemd\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.963624 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962386    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bc0b02e6-393f-4ba4-8708-25daa2922465-etc-tuned\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.963624 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962412    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4128cbcd-5ec2-40d9-81fa-9661440e82df-kubelet-config\") pod \"global-pull-secret-syncer-hdtt4\" (UID: \"4128cbcd-5ec2-40d9-81fa-9661440e82df\") " pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:04.963624 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962420    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0f492bfc-2ea5-4bab-a34b-b6df6736724e-env-overrides\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.963624 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962454    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret\") pod \"global-pull-secret-syncer-hdtt4\" (UID: \"4128cbcd-5ec2-40d9-81fa-9661440e82df\") " pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:04.963624 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962478    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-etc-systemd\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.963624 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962487    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0f492bfc-2ea5-4bab-a34b-b6df6736724e-ovnkube-config\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.963624 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962504    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-host-slash\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.964463 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962529    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-host-cni-bin\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.964463 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962553    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-sys\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.964463 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962177    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-var-lib-kubelet\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.964463 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962629    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/001f8df1-bca6-441f-89d3-01bb8d3741b0-hosts-file\") pod \"node-resolver-t8kwx\" (UID: \"001f8df1-bca6-441f-89d3-01bb8d3741b0\") " pod="openshift-dns/node-resolver-t8kwx"
Jul 07 22:15:04.964463 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962671    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-host-cni-bin\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.964463 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962417    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-etc-kubernetes\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.964463 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962672    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-etc-systemd\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.964463 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962686    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-host-slash\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.964463 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962723    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e12ec4b7-d42c-4cb9-8d31-769cae33e436-sys-fs\") pod \"aws-ebs-csi-driver-node-sxdgf\" (UID: \"e12ec4b7-d42c-4cb9-8d31-769cae33e436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:04.964463 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962777    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0f492bfc-2ea5-4bab-a34b-b6df6736724e-env-overrides\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.964463 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962793    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-etc-modprobe-d\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.964463 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962640    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-sys\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.964463 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962833    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-host-run-netns\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.964463 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962840    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bc0b02e6-393f-4ba4-8708-25daa2922465-etc-sysconfig\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.964463 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962846    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-node-log\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.964463 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:04.962835    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 22:15:04.964463 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962893    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0f492bfc-2ea5-4bab-a34b-b6df6736724e-run-openvswitch\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.964463 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:04.962941    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret podName:4128cbcd-5ec2-40d9-81fa-9661440e82df nodeName:}" failed. No retries permitted until 2026-07-07 22:15:05.462921685 +0000 UTC m=+3.135140654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret") pod "global-pull-secret-syncer-hdtt4" (UID: "4128cbcd-5ec2-40d9-81fa-9661440e82df") : object "kube-system"/"original-pull-secret" not registered
Jul 07 22:15:04.965574 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.962959    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4128cbcd-5ec2-40d9-81fa-9661440e82df-dbus\") pod \"global-pull-secret-syncer-hdtt4\" (UID: \"4128cbcd-5ec2-40d9-81fa-9661440e82df\") " pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:04.965574 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.964898    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc0b02e6-393f-4ba4-8708-25daa2922465-tmp\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.965574 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.965211    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bc0b02e6-393f-4ba4-8708-25daa2922465-etc-tuned\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.965574 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.965289    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0f492bfc-2ea5-4bab-a34b-b6df6736724e-ovn-node-metrics-cert\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.982759 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.982653    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs68k\" (UniqueName: \"kubernetes.io/projected/001f8df1-bca6-441f-89d3-01bb8d3741b0-kube-api-access-vs68k\") pod \"node-resolver-t8kwx\" (UID: \"001f8df1-bca6-441f-89d3-01bb8d3741b0\") " pod="openshift-dns/node-resolver-t8kwx"
Jul 07 22:15:04.984632 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.984608    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq975\" (UniqueName: \"kubernetes.io/projected/bc0b02e6-393f-4ba4-8708-25daa2922465-kube-api-access-zq975\") pod \"tuned-b5wpj\" (UID: \"bc0b02e6-393f-4ba4-8708-25daa2922465\") " pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:04.987816 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.987787    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgp4n\" (UniqueName: \"kubernetes.io/projected/0f492bfc-2ea5-4bab-a34b-b6df6736724e-kube-api-access-jgp4n\") pod \"ovnkube-node-mbznv\" (UID: \"0f492bfc-2ea5-4bab-a34b-b6df6736724e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:04.988737 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:04.988714    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrpdq\" (UniqueName: \"kubernetes.io/projected/e12ec4b7-d42c-4cb9-8d31-769cae33e436-kube-api-access-xrpdq\") pod \"aws-ebs-csi-driver-node-sxdgf\" (UID: \"e12ec4b7-d42c-4cb9-8d31-769cae33e436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:05.053687 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:05.053652    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vwjdg"
Jul 07 22:15:05.064730 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:05.064705    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kz7pz"
Jul 07 22:15:05.077445 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:05.077422    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7x2ml"
Jul 07 22:15:05.083552 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:05.083527    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hgwst"
Jul 07 22:15:05.093112 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:05.093092    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bnnmv"
Jul 07 22:15:05.102721 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:05.102699    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf"
Jul 07 22:15:05.118341 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:05.118313    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b5wpj"
Jul 07 22:15:05.124952 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:05.124931    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t8kwx"
Jul 07 22:15:05.131666 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:05.131648    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:05.365847 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:05.365770    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs\") pod \"network-metrics-daemon-58clx\" (UID: \"28fa7951-7615-4e16-96f8-c918ab062cd3\") " pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:05.366002 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:05.365912    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 22:15:05.366002 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:05.365978    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs podName:28fa7951-7615-4e16-96f8-c918ab062cd3 nodeName:}" failed. No retries permitted until 2026-07-07 22:15:06.365960204 +0000 UTC m=+4.038179171 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs") pod "network-metrics-daemon-58clx" (UID: "28fa7951-7615-4e16-96f8-c918ab062cd3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 22:15:05.466698 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:05.466662    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxdbf\" (UniqueName: \"kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf\") pod \"network-check-target-nbgl7\" (UID: \"c248b400-7255-477d-b130-6daf8344f786\") " pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:05.466899 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:05.466756    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret\") pod \"global-pull-secret-syncer-hdtt4\" (UID: \"4128cbcd-5ec2-40d9-81fa-9661440e82df\") " pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:05.466899 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:05.466846    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 22:15:05.466899 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:05.466888    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 22:15:05.466899 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:05.466901    2555 projected.go:196] Error preparing data for projected volume kube-api-access-zxdbf for pod openshift-network-diagnostics/network-check-target-nbgl7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 22:15:05.467112 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:05.466949    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf podName:c248b400-7255-477d-b130-6daf8344f786 nodeName:}" failed. No retries permitted until 2026-07-07 22:15:06.46693476 +0000 UTC m=+4.139153726 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zxdbf" (UniqueName: "kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf") pod "network-check-target-nbgl7" (UID: "c248b400-7255-477d-b130-6daf8344f786") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 22:15:05.467112 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:05.466847    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 22:15:05.467112 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:05.466985    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret podName:4128cbcd-5ec2-40d9-81fa-9661440e82df nodeName:}" failed. No retries permitted until 2026-07-07 22:15:06.466976901 +0000 UTC m=+4.139195867 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret") pod "global-pull-secret-syncer-hdtt4" (UID: "4128cbcd-5ec2-40d9-81fa-9661440e82df") : object "kube-system"/"original-pull-secret" not registered
Jul 07 22:15:05.790512 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:05.790423    2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 22:10:04 +0000 UTC" deadline="2027-12-20 02:57:21.418361068 +0000 UTC"
Jul 07 22:15:05.790512 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:05.790461    2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12724h42m15.627904821s"
Jul 07 22:15:05.917329 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:05.917296    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:05.917510 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:05.917340    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:05.917510 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:05.917449    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58clx" podUID="28fa7951-7615-4e16-96f8-c918ab062cd3"
Jul 07 22:15:05.917601 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:05.917568    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdtt4" podUID="4128cbcd-5ec2-40d9-81fa-9661440e82df"
Jul 07 22:15:06.011913 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:06.011881    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod111c1daf_0cbf_4a17_bc99_92cb26144a4e.slice/crio-15c353f4bf99dcd0169249f63728f606b365871643c31632e659af8b4cb4f7f5 WatchSource:0}: Error finding container 15c353f4bf99dcd0169249f63728f606b365871643c31632e659af8b4cb4f7f5: Status 404 returned error can't find the container with id 15c353f4bf99dcd0169249f63728f606b365871643c31632e659af8b4cb4f7f5
Jul 07 22:15:06.013525 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:06.013403    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc0b02e6_393f_4ba4_8708_25daa2922465.slice/crio-43df84c83dc068ae4509517ceb3fb9a731c7970f60befb04278abfe16716c443 WatchSource:0}: Error finding container 43df84c83dc068ae4509517ceb3fb9a731c7970f60befb04278abfe16716c443: Status 404 returned error can't find the container with id 43df84c83dc068ae4509517ceb3fb9a731c7970f60befb04278abfe16716c443
Jul 07 22:15:06.014311 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:06.014271    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad33be48_3fec_4f6d_9d2c_a01dd3e342df.slice/crio-29ebde095213cac1c9fbf88b63a89b4ae8f840f8b1fbbb3d6aeffc232f1d1942 WatchSource:0}: Error finding container 29ebde095213cac1c9fbf88b63a89b4ae8f840f8b1fbbb3d6aeffc232f1d1942: Status 404 returned error can't find the container with id 29ebde095213cac1c9fbf88b63a89b4ae8f840f8b1fbbb3d6aeffc232f1d1942
Jul 07 22:15:06.017421 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:06.017399    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f492bfc_2ea5_4bab_a34b_b6df6736724e.slice/crio-3d62838b4b49a558e0a402f95fa8ee4c67fadd15216b63b062f6f01e56e0172d WatchSource:0}: Error finding container 3d62838b4b49a558e0a402f95fa8ee4c67fadd15216b63b062f6f01e56e0172d: Status 404 returned error can't find the container with id 3d62838b4b49a558e0a402f95fa8ee4c67fadd15216b63b062f6f01e56e0172d
Jul 07 22:15:06.018432 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:06.018361    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode32ca0b6_d3f9_4868_9b27_06e20b06a8ae.slice/crio-1914b34a101d1262674005096df307af8047c3b2e81d17567ec6a488ef646664 WatchSource:0}: Error finding container 1914b34a101d1262674005096df307af8047c3b2e81d17567ec6a488ef646664: Status 404 returned error can't find the container with id 1914b34a101d1262674005096df307af8047c3b2e81d17567ec6a488ef646664
Jul 07 22:15:06.018969 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:06.018929    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod001f8df1_bca6_441f_89d3_01bb8d3741b0.slice/crio-f4a2e2b45154f0d1b81f0e50b69f58c8cafd8587cd557e9f08bf67e7462aef96 WatchSource:0}: Error finding container f4a2e2b45154f0d1b81f0e50b69f58c8cafd8587cd557e9f08bf67e7462aef96: Status 404 returned error can't find the container with id f4a2e2b45154f0d1b81f0e50b69f58c8cafd8587cd557e9f08bf67e7462aef96
Jul 07 22:15:06.020113 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:06.019907    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode12ec4b7_d42c_4cb9_8d31_769cae33e436.slice/crio-dd9024cb46530ed0240b46c822ed73d4012f8e4e49d91e8d7ef37f1d8c346633 WatchSource:0}: Error finding container dd9024cb46530ed0240b46c822ed73d4012f8e4e49d91e8d7ef37f1d8c346633: Status 404 returned error can't find the container with id dd9024cb46530ed0240b46c822ed73d4012f8e4e49d91e8d7ef37f1d8c346633
Jul 07 22:15:06.021148 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:06.020645    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod621e2b82_009d_465d_940a_57c8bff5595d.slice/crio-37530d3e2b5a223e05a1f66921320bc6a63a9b6b7571b27ba46f5ad76760fea8 WatchSource:0}: Error finding container 37530d3e2b5a223e05a1f66921320bc6a63a9b6b7571b27ba46f5ad76760fea8: Status 404 returned error can't find the container with id 37530d3e2b5a223e05a1f66921320bc6a63a9b6b7571b27ba46f5ad76760fea8
Jul 07 22:15:06.022984 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:06.022146    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1155169f_bea8_43e4_a927_668ec8c8e77f.slice/crio-3a482fbc375ef41da9361cee4e3a1e26947d97946eeaa44a6be2cb24b843df37 WatchSource:0}: Error finding container 3a482fbc375ef41da9361cee4e3a1e26947d97946eeaa44a6be2cb24b843df37: Status 404 returned error can't find the container with id 3a482fbc375ef41da9361cee4e3a1e26947d97946eeaa44a6be2cb24b843df37
Jul 07 22:15:06.372796 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:06.372512    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs\") pod \"network-metrics-daemon-58clx\" (UID: \"28fa7951-7615-4e16-96f8-c918ab062cd3\") " pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:06.372796 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:06.372666    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 22:15:06.373034 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:06.372822    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs podName:28fa7951-7615-4e16-96f8-c918ab062cd3 nodeName:}" failed. No retries permitted until 2026-07-07 22:15:08.372802531 +0000 UTC m=+6.045021500 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs") pod "network-metrics-daemon-58clx" (UID: "28fa7951-7615-4e16-96f8-c918ab062cd3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 22:15:06.473823 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:06.473776    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxdbf\" (UniqueName: \"kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf\") pod \"network-check-target-nbgl7\" (UID: \"c248b400-7255-477d-b130-6daf8344f786\") " pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:06.473991 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:06.473923    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret\") pod \"global-pull-secret-syncer-hdtt4\" (UID: \"4128cbcd-5ec2-40d9-81fa-9661440e82df\") " pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:06.474047 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:06.474000    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 22:15:06.474047 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:06.474035    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 22:15:06.474144 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:06.474052    2555 projected.go:196] Error preparing data for projected volume kube-api-access-zxdbf for pod openshift-network-diagnostics/network-check-target-nbgl7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 22:15:06.474144 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:06.474068    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 22:15:06.474144 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:06.474133    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf podName:c248b400-7255-477d-b130-6daf8344f786 nodeName:}" failed. No retries permitted until 2026-07-07 22:15:08.47411399 +0000 UTC m=+6.146332972 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zxdbf" (UniqueName: "kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf") pod "network-check-target-nbgl7" (UID: "c248b400-7255-477d-b130-6daf8344f786") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 22:15:06.474324 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:06.474155    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret podName:4128cbcd-5ec2-40d9-81fa-9661440e82df nodeName:}" failed. No retries permitted until 2026-07-07 22:15:08.474144438 +0000 UTC m=+6.146363408 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret") pod "global-pull-secret-syncer-hdtt4" (UID: "4128cbcd-5ec2-40d9-81fa-9661440e82df") : object "kube-system"/"original-pull-secret" not registered
Jul 07 22:15:06.791053 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:06.790957    2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 22:10:04 +0000 UTC" deadline="2028-03-16 10:37:07.609975634 +0000 UTC"
Jul 07 22:15:06.791053 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:06.791019    2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14820h22m0.81896035s"
Jul 07 22:15:06.920387 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:06.919859    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:06.920387 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:06.919997    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbgl7" podUID="c248b400-7255-477d-b130-6daf8344f786"
Jul 07 22:15:06.939576 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:06.939424    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-159.ec2.internal" event={"ID":"e236da3c0fadbab47c6a6accf20cc40f","Type":"ContainerDied","Data":"f4f0fd12df5580bce1f689a02f85ab611692a072b78129292dbe8f67038b6c32"}
Jul 07 22:15:06.939576 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:06.939397    2555 generic.go:356] "Generic (PLEG): container finished" podID="e236da3c0fadbab47c6a6accf20cc40f" containerID="f4f0fd12df5580bce1f689a02f85ab611692a072b78129292dbe8f67038b6c32" exitCode=0
Jul 07 22:15:06.956230 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:06.956199    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hgwst" event={"ID":"1155169f-bea8-43e4-a927-668ec8c8e77f","Type":"ContainerStarted","Data":"3a482fbc375ef41da9361cee4e3a1e26947d97946eeaa44a6be2cb24b843df37"}
Jul 07 22:15:06.964829 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:06.964801    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t8kwx" event={"ID":"001f8df1-bca6-441f-89d3-01bb8d3741b0","Type":"ContainerStarted","Data":"f4a2e2b45154f0d1b81f0e50b69f58c8cafd8587cd557e9f08bf67e7462aef96"}
Jul 07 22:15:06.967495 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:06.967469    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kz7pz" event={"ID":"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae","Type":"ContainerStarted","Data":"1914b34a101d1262674005096df307af8047c3b2e81d17567ec6a488ef646664"}
Jul 07 22:15:06.971066 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:06.971041    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mbznv" event={"ID":"0f492bfc-2ea5-4bab-a34b-b6df6736724e","Type":"ContainerStarted","Data":"3d62838b4b49a558e0a402f95fa8ee4c67fadd15216b63b062f6f01e56e0172d"}
Jul 07 22:15:06.996534 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:06.995829    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-159.ec2.internal" event={"ID":"f99cc9ba576df91ea14d32094adce059","Type":"ContainerStarted","Data":"55edf514c9155839f744b4750e64eab5a6816881e1c4350f8d06a535edaa583a"}
Jul 07 22:15:07.007799 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:07.007768    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vwjdg" event={"ID":"621e2b82-009d-465d-940a-57c8bff5595d","Type":"ContainerStarted","Data":"37530d3e2b5a223e05a1f66921320bc6a63a9b6b7571b27ba46f5ad76760fea8"}
Jul 07 22:15:07.010828 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:07.010702    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-159.ec2.internal" podStartSLOduration=4.01067638 podStartE2EDuration="4.01067638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:15:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:15:07.010488181 +0000 UTC m=+4.682707171" watchObservedRunningTime="2026-07-07 22:15:07.01067638 +0000 UTC m=+4.682895368"
Jul 07 22:15:07.027539 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:07.027506    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf" event={"ID":"e12ec4b7-d42c-4cb9-8d31-769cae33e436","Type":"ContainerStarted","Data":"dd9024cb46530ed0240b46c822ed73d4012f8e4e49d91e8d7ef37f1d8c346633"}
Jul 07 22:15:07.032242 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:07.032170    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b5wpj" event={"ID":"bc0b02e6-393f-4ba4-8708-25daa2922465","Type":"ContainerStarted","Data":"43df84c83dc068ae4509517ceb3fb9a731c7970f60befb04278abfe16716c443"}
Jul 07 22:15:07.048297 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:07.048263    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7x2ml" event={"ID":"ad33be48-3fec-4f6d-9d2c-a01dd3e342df","Type":"ContainerStarted","Data":"29ebde095213cac1c9fbf88b63a89b4ae8f840f8b1fbbb3d6aeffc232f1d1942"}
Jul 07 22:15:07.051711 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:07.051660    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bnnmv" event={"ID":"111c1daf-0cbf-4a17-bc99-92cb26144a4e","Type":"ContainerStarted","Data":"15c353f4bf99dcd0169249f63728f606b365871643c31632e659af8b4cb4f7f5"}
Jul 07 22:15:07.917595 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:07.916988    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:07.917595 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:07.917110    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdtt4" podUID="4128cbcd-5ec2-40d9-81fa-9661440e82df"
Jul 07 22:15:07.917595 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:07.917190    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:07.917595 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:07.917274    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58clx" podUID="28fa7951-7615-4e16-96f8-c918ab062cd3"
Jul 07 22:15:08.079628 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:08.079591    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-159.ec2.internal" event={"ID":"e236da3c0fadbab47c6a6accf20cc40f","Type":"ContainerStarted","Data":"edbdb5a80d9a67b9d98a25a0f4f26f6838aecfc4e6d85befa016d536fe21b161"}
Jul 07 22:15:08.391298 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:08.391212    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs\") pod \"network-metrics-daemon-58clx\" (UID: \"28fa7951-7615-4e16-96f8-c918ab062cd3\") " pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:08.391462 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:08.391390    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 22:15:08.391462 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:08.391461    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs podName:28fa7951-7615-4e16-96f8-c918ab062cd3 nodeName:}" failed. No retries permitted until 2026-07-07 22:15:12.391442437 +0000 UTC m=+10.063661420 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs") pod "network-metrics-daemon-58clx" (UID: "28fa7951-7615-4e16-96f8-c918ab062cd3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 22:15:08.491862 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:08.491816    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret\") pod \"global-pull-secret-syncer-hdtt4\" (UID: \"4128cbcd-5ec2-40d9-81fa-9661440e82df\") " pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:08.492073 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:08.491886    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxdbf\" (UniqueName: \"kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf\") pod \"network-check-target-nbgl7\" (UID: \"c248b400-7255-477d-b130-6daf8344f786\") " pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:08.492154 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:08.492072    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 22:15:08.492154 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:08.492092    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 22:15:08.492154 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:08.492105    2555 projected.go:196] Error preparing data for projected volume kube-api-access-zxdbf for pod openshift-network-diagnostics/network-check-target-nbgl7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 22:15:08.492389 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:08.492169    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf podName:c248b400-7255-477d-b130-6daf8344f786 nodeName:}" failed. No retries permitted until 2026-07-07 22:15:12.492143053 +0000 UTC m=+10.164362035 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zxdbf" (UniqueName: "kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf") pod "network-check-target-nbgl7" (UID: "c248b400-7255-477d-b130-6daf8344f786") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 22:15:08.492699 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:08.492552    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 22:15:08.492699 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:08.492608    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret podName:4128cbcd-5ec2-40d9-81fa-9661440e82df nodeName:}" failed. No retries permitted until 2026-07-07 22:15:12.492589801 +0000 UTC m=+10.164808770 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret") pod "global-pull-secret-syncer-hdtt4" (UID: "4128cbcd-5ec2-40d9-81fa-9661440e82df") : object "kube-system"/"original-pull-secret" not registered
Jul 07 22:15:08.919542 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:08.919501    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:08.920000 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:08.919629    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbgl7" podUID="c248b400-7255-477d-b130-6daf8344f786"
Jul 07 22:15:09.916824 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:09.916787    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:09.917012 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:09.916828    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:09.917085 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:09.917019    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdtt4" podUID="4128cbcd-5ec2-40d9-81fa-9661440e82df"
Jul 07 22:15:09.917188 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:09.917148    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58clx" podUID="28fa7951-7615-4e16-96f8-c918ab062cd3"
Jul 07 22:15:10.917693 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:10.917518    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:10.917693 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:10.917652    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbgl7" podUID="c248b400-7255-477d-b130-6daf8344f786"
Jul 07 22:15:11.916997 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:11.916961    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:11.917151 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:11.916970    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:11.917151 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:11.917080    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdtt4" podUID="4128cbcd-5ec2-40d9-81fa-9661440e82df"
Jul 07 22:15:11.917239 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:11.917189    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58clx" podUID="28fa7951-7615-4e16-96f8-c918ab062cd3"
Jul 07 22:15:12.425440 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:12.425385    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs\") pod \"network-metrics-daemon-58clx\" (UID: \"28fa7951-7615-4e16-96f8-c918ab062cd3\") " pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:12.425907 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:12.425517    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 22:15:12.425907 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:12.425576    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs podName:28fa7951-7615-4e16-96f8-c918ab062cd3 nodeName:}" failed. No retries permitted until 2026-07-07 22:15:20.425557017 +0000 UTC m=+18.097775989 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs") pod "network-metrics-daemon-58clx" (UID: "28fa7951-7615-4e16-96f8-c918ab062cd3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 22:15:12.526755 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:12.526717    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret\") pod \"global-pull-secret-syncer-hdtt4\" (UID: \"4128cbcd-5ec2-40d9-81fa-9661440e82df\") " pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:12.526931 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:12.526770    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxdbf\" (UniqueName: \"kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf\") pod \"network-check-target-nbgl7\" (UID: \"c248b400-7255-477d-b130-6daf8344f786\") " pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:12.526931 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:12.526887    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 22:15:12.526931 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:12.526923    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 22:15:12.527101 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:12.526937    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 22:15:12.527101 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:12.526945    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret podName:4128cbcd-5ec2-40d9-81fa-9661440e82df nodeName:}" failed. No retries permitted until 2026-07-07 22:15:20.526926791 +0000 UTC m=+18.199145783 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret") pod "global-pull-secret-syncer-hdtt4" (UID: "4128cbcd-5ec2-40d9-81fa-9661440e82df") : object "kube-system"/"original-pull-secret" not registered
Jul 07 22:15:12.527101 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:12.526949    2555 projected.go:196] Error preparing data for projected volume kube-api-access-zxdbf for pod openshift-network-diagnostics/network-check-target-nbgl7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 22:15:12.527101 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:12.526983    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf podName:c248b400-7255-477d-b130-6daf8344f786 nodeName:}" failed. No retries permitted until 2026-07-07 22:15:20.526972084 +0000 UTC m=+18.199191050 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zxdbf" (UniqueName: "kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf") pod "network-check-target-nbgl7" (UID: "c248b400-7255-477d-b130-6daf8344f786") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 22:15:12.918407 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:12.918334    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:12.918558 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:12.918450    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbgl7" podUID="c248b400-7255-477d-b130-6daf8344f786"
Jul 07 22:15:13.917304 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:13.917264    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:13.917773 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:13.917306    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:13.917773 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:13.917416    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58clx" podUID="28fa7951-7615-4e16-96f8-c918ab062cd3"
Jul 07 22:15:13.917773 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:13.917575    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdtt4" podUID="4128cbcd-5ec2-40d9-81fa-9661440e82df"
Jul 07 22:15:14.917595 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:14.917557    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:14.918064 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:14.917704    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbgl7" podUID="c248b400-7255-477d-b130-6daf8344f786"
Jul 07 22:15:15.917545 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:15.917514    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:15.917719 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:15.917655    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58clx" podUID="28fa7951-7615-4e16-96f8-c918ab062cd3"
Jul 07 22:15:15.917719 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:15.917522    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:15.918136 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:15.917764    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdtt4" podUID="4128cbcd-5ec2-40d9-81fa-9661440e82df"
Jul 07 22:15:16.916812 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:16.916777    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:16.917008 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:16.916931    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbgl7" podUID="c248b400-7255-477d-b130-6daf8344f786"
Jul 07 22:15:17.917419 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:17.917383    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:17.917896 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:17.917506    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdtt4" podUID="4128cbcd-5ec2-40d9-81fa-9661440e82df"
Jul 07 22:15:17.917896 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:17.917546    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:17.917896 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:17.917636    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58clx" podUID="28fa7951-7615-4e16-96f8-c918ab062cd3"
Jul 07 22:15:18.916881 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:18.916830    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:18.917065 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:18.916981    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbgl7" podUID="c248b400-7255-477d-b130-6daf8344f786"
Jul 07 22:15:19.916768 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:19.916729    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:19.917195 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:19.916744    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:19.917195 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:19.916850    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58clx" podUID="28fa7951-7615-4e16-96f8-c918ab062cd3"
Jul 07 22:15:19.917195 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:19.916909    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdtt4" podUID="4128cbcd-5ec2-40d9-81fa-9661440e82df"
Jul 07 22:15:20.491812 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:20.491777    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs\") pod \"network-metrics-daemon-58clx\" (UID: \"28fa7951-7615-4e16-96f8-c918ab062cd3\") " pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:20.492043 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:20.491923    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 22:15:20.492043 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:20.491983    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs podName:28fa7951-7615-4e16-96f8-c918ab062cd3 nodeName:}" failed. No retries permitted until 2026-07-07 22:15:36.491964015 +0000 UTC m=+34.164182982 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs") pod "network-metrics-daemon-58clx" (UID: "28fa7951-7615-4e16-96f8-c918ab062cd3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 22:15:20.592644 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:20.592599    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret\") pod \"global-pull-secret-syncer-hdtt4\" (UID: \"4128cbcd-5ec2-40d9-81fa-9661440e82df\") " pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:20.592644 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:20.592645    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxdbf\" (UniqueName: \"kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf\") pod \"network-check-target-nbgl7\" (UID: \"c248b400-7255-477d-b130-6daf8344f786\") " pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:20.592897 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:20.592770    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 22:15:20.592897 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:20.592794    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 22:15:20.592897 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:20.592811    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 22:15:20.592897 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:20.592823    2555 projected.go:196] Error preparing data for projected volume kube-api-access-zxdbf for pod openshift-network-diagnostics/network-check-target-nbgl7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 22:15:20.592897 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:20.592845    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret podName:4128cbcd-5ec2-40d9-81fa-9661440e82df nodeName:}" failed. No retries permitted until 2026-07-07 22:15:36.592825828 +0000 UTC m=+34.265044796 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret") pod "global-pull-secret-syncer-hdtt4" (UID: "4128cbcd-5ec2-40d9-81fa-9661440e82df") : object "kube-system"/"original-pull-secret" not registered
Jul 07 22:15:20.592897 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:20.592892    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf podName:c248b400-7255-477d-b130-6daf8344f786 nodeName:}" failed. No retries permitted until 2026-07-07 22:15:36.592856649 +0000 UTC m=+34.265075621 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zxdbf" (UniqueName: "kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf") pod "network-check-target-nbgl7" (UID: "c248b400-7255-477d-b130-6daf8344f786") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 22:15:20.917328 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:20.917263    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:20.917729 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:20.917378    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbgl7" podUID="c248b400-7255-477d-b130-6daf8344f786"
Jul 07 22:15:21.917563 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:21.917533    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:21.918033 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:21.917643    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdtt4" podUID="4128cbcd-5ec2-40d9-81fa-9661440e82df"
Jul 07 22:15:21.918033 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:21.917708    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:21.918033 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:21.917808    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58clx" podUID="28fa7951-7615-4e16-96f8-c918ab062cd3"
Jul 07 22:15:22.917724 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:22.917694    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:22.918197 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:22.917774    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbgl7" podUID="c248b400-7255-477d-b130-6daf8344f786"
Jul 07 22:15:23.916704 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:23.916671    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:23.916894 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:23.916793    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdtt4" podUID="4128cbcd-5ec2-40d9-81fa-9661440e82df"
Jul 07 22:15:23.916894 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:23.916859    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:23.917005 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:23.916987    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58clx" podUID="28fa7951-7615-4e16-96f8-c918ab062cd3"
Jul 07 22:15:24.917015 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:24.916826    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:24.917457 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:24.917118    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbgl7" podUID="c248b400-7255-477d-b130-6daf8344f786"
Jul 07 22:15:25.112116 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.111857    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hgwst" event={"ID":"1155169f-bea8-43e4-a927-668ec8c8e77f","Type":"ContainerStarted","Data":"7792d2037bad86e6e7632c2c2851598af6dcac95074c9a275006ca002a8178bc"}
Jul 07 22:15:25.113435 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.113410    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t8kwx" event={"ID":"001f8df1-bca6-441f-89d3-01bb8d3741b0","Type":"ContainerStarted","Data":"d04cd2336e12c238a408a007c67286c1dcbd3f8034549c50934224a0d5fe1667"}
Jul 07 22:15:25.115133 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.115107    2555 generic.go:356] "Generic (PLEG): container finished" podID="e32ca0b6-d3f9-4868-9b27-06e20b06a8ae" containerID="a90201e1a64963cf9a27fb644e7dc81cc0b1c219b8225aa0f63a911f8f3d91ec" exitCode=0
Jul 07 22:15:25.115239 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.115195    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kz7pz" event={"ID":"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae","Type":"ContainerDied","Data":"a90201e1a64963cf9a27fb644e7dc81cc0b1c219b8225aa0f63a911f8f3d91ec"}
Jul 07 22:15:25.117925 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.117904    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:15:25.118255 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.118234    2555 generic.go:356] "Generic (PLEG): container finished" podID="0f492bfc-2ea5-4bab-a34b-b6df6736724e" containerID="72cb86ebf297a1b504e82ac024de5c2abc9b90d468385e47f71e93956321fd87" exitCode=1
Jul 07 22:15:25.118336 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.118258    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mbznv" event={"ID":"0f492bfc-2ea5-4bab-a34b-b6df6736724e","Type":"ContainerStarted","Data":"bdd3c99bca29fad3b8cb51e782f13b4d0d3ccd9f8f43d0932cf9b9c10f45b65e"}
Jul 07 22:15:25.118336 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.118320    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mbznv" event={"ID":"0f492bfc-2ea5-4bab-a34b-b6df6736724e","Type":"ContainerStarted","Data":"e8ada33915d144e07d98e8a4fd5889fdc84ba64de660c322f03c0c0dc7a7d1d4"}
Jul 07 22:15:25.118414 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.118354    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mbznv" event={"ID":"0f492bfc-2ea5-4bab-a34b-b6df6736724e","Type":"ContainerDied","Data":"72cb86ebf297a1b504e82ac024de5c2abc9b90d468385e47f71e93956321fd87"}
Jul 07 22:15:25.118414 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.118383    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mbznv" event={"ID":"0f492bfc-2ea5-4bab-a34b-b6df6736724e","Type":"ContainerStarted","Data":"7d21aae721fe4d6730cf325997a9c28ef5b96585f49a4997981001605f477cf6"}
Jul 07 22:15:25.119854 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.119836    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vwjdg" event={"ID":"621e2b82-009d-465d-940a-57c8bff5595d","Type":"ContainerStarted","Data":"73e335d5111874064fab7c97cb09b5254c6e4deb4e3ae12e702ddb1ec78db8ba"}
Jul 07 22:15:25.121567 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.121521    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf" event={"ID":"e12ec4b7-d42c-4cb9-8d31-769cae33e436","Type":"ContainerStarted","Data":"ae63bc79b4a7c7571d8015925d4e8839943854dd1be64b722287647149f753f4"}
Jul 07 22:15:25.123062 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.123042    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b5wpj" event={"ID":"bc0b02e6-393f-4ba4-8708-25daa2922465","Type":"ContainerStarted","Data":"c7e6f2b3b8469b785648437905538b504150b12cf2b3693c5f103fbae47d47e1"}
Jul 07 22:15:25.124602 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.124569    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7x2ml" event={"ID":"ad33be48-3fec-4f6d-9d2c-a01dd3e342df","Type":"ContainerStarted","Data":"96c4fbdf66383596c3b676f14963ebb9cc261c22cabefc55fea1ea7de2d43c96"}
Jul 07 22:15:25.127440 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.127400    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hgwst" podStartSLOduration=3.798359295 podStartE2EDuration="22.127387736s" totalImagesPullingTime="18.329028441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:15:25.126820381 +0000 UTC m=+22.799039370" watchObservedRunningTime="2026-07-07 22:15:25.127387736 +0000 UTC m=+22.799606723"
Jul 07 22:15:25.127604 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.127569    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-159.ec2.internal" podStartSLOduration=22.127557072 podStartE2EDuration="22.127557072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:15:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:15:08.095220108 +0000 UTC m=+5.767439097" watchObservedRunningTime="2026-07-07 22:15:25.127557072 +0000 UTC m=+22.799776060"
Jul 07 22:15:25.144481 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.144438    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-b5wpj" podStartSLOduration=3.744482209 podStartE2EDuration="22.144424287s" totalImagesPullingTime="18.399942078s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:15:25.143822194 +0000 UTC m=+22.816041192" watchObservedRunningTime="2026-07-07 22:15:25.144424287 +0000 UTC m=+22.816643274"
Jul 07 22:15:25.176703 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.176659    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7x2ml" podStartSLOduration=3.7222643 podStartE2EDuration="22.176641337s" totalImagesPullingTime="18.454377037s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:15:25.176332275 +0000 UTC m=+22.848551264" watchObservedRunningTime="2026-07-07 22:15:25.176641337 +0000 UTC m=+22.848860326"
Jul 07 22:15:25.191420 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.191379    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-t8kwx" podStartSLOduration=3.792359705 podStartE2EDuration="22.191361754s" totalImagesPullingTime="18.399002049s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:15:25.191088222 +0000 UTC m=+22.863307211" watchObservedRunningTime="2026-07-07 22:15:25.191361754 +0000 UTC m=+22.863580744"
Jul 07 22:15:25.203513 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.203482    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vwjdg" podStartSLOduration=4.8346943620000005 podStartE2EDuration="23.203471599s" totalImagesPullingTime="18.368777237s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:15:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:15:25.203181174 +0000 UTC m=+22.875400164" watchObservedRunningTime="2026-07-07 22:15:25.203471599 +0000 UTC m=+22.875690586"
Jul 07 22:15:25.917547 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.917522    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:25.917842 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:25.917555    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:25.917842 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:25.917654    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58clx" podUID="28fa7951-7615-4e16-96f8-c918ab062cd3"
Jul 07 22:15:25.917842 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:25.917749    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdtt4" podUID="4128cbcd-5ec2-40d9-81fa-9661440e82df"
Jul 07 22:15:26.020058 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:26.020038    2555 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 07 22:15:26.130744 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:26.130707    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:15:26.131168 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:26.131132    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mbznv" event={"ID":"0f492bfc-2ea5-4bab-a34b-b6df6736724e","Type":"ContainerStarted","Data":"20a43de3aa8661039e93142d5ca637d2b95d7786ba8dcee71512075832c65100"}
Jul 07 22:15:26.131263 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:26.131218    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mbznv" event={"ID":"0f492bfc-2ea5-4bab-a34b-b6df6736724e","Type":"ContainerStarted","Data":"9a470d5661feef5a4b556ba07ea0d1a11935a182959fea49eb6e9ed652715ad2"}
Jul 07 22:15:26.133083 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:26.133059    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf" event={"ID":"e12ec4b7-d42c-4cb9-8d31-769cae33e436","Type":"ContainerStarted","Data":"e4472a9acba1277c993a9a3181114a5da686e73e6d728e1d765451511bfe7f03"}
Jul 07 22:15:26.134628 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:26.134600    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bnnmv" event={"ID":"111c1daf-0cbf-4a17-bc99-92cb26144a4e","Type":"ContainerStarted","Data":"194b6993eebe77717638edbd8fe41bb299ef2989d101c017704d595fe9942de3"}
Jul 07 22:15:26.148833 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:26.148768    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bnnmv" podStartSLOduration=9.044357432 podStartE2EDuration="23.14875567s" totalImagesPullingTime="14.104398238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:15:26.148542847 +0000 UTC m=+23.820761835" watchObservedRunningTime="2026-07-07 22:15:26.14875567 +0000 UTC m=+23.820974658"
Jul 07 22:15:26.821426 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:26.821088    2555 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T22:15:26.020054724Z","UUID":"8cd21fe5-1cb9-4b71-886b-9cfc5fdf636b","Handler":null,"Name":"","Endpoint":""}
Jul 07 22:15:26.824104 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:26.824076    2555 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 07 22:15:26.824223 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:26.824130    2555 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 07 22:15:26.917438 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:26.917408    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:26.917578 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:26.917509    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbgl7" podUID="c248b400-7255-477d-b130-6daf8344f786"
Jul 07 22:15:27.917277 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:27.917241    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:27.917440 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:27.917376    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdtt4" podUID="4128cbcd-5ec2-40d9-81fa-9661440e82df"
Jul 07 22:15:27.917497 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:27.917433    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:27.917580 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:27.917560    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58clx" podUID="28fa7951-7615-4e16-96f8-c918ab062cd3"
Jul 07 22:15:28.142592 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:28.142520    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf" event={"ID":"e12ec4b7-d42c-4cb9-8d31-769cae33e436","Type":"ContainerStarted","Data":"ea96f952a4989d85273c3f063b89b51f39ee5132771f8514d82a11b879a7dcea"}
Jul 07 22:15:28.149021 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:28.148990    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:15:28.149426 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:28.149396    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mbznv" event={"ID":"0f492bfc-2ea5-4bab-a34b-b6df6736724e","Type":"ContainerStarted","Data":"5e904e1f0fe7620bcca7c9f07002498e803bb5e18da7e8c1c37683df7dce4ea9"}
Jul 07 22:15:28.160212 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:28.160166    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxdgf" podStartSLOduration=4.345849919 podStartE2EDuration="25.160148027s" totalImagesPullingTime="20.814298108s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:15:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:15:28.159527057 +0000 UTC m=+25.831746042" watchObservedRunningTime="2026-07-07 22:15:28.160148027 +0000 UTC m=+25.832367017"
Jul 07 22:15:28.917517 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:28.917484    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:28.917698 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:28.917610    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbgl7" podUID="c248b400-7255-477d-b130-6daf8344f786"
Jul 07 22:15:29.559196 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:29.559171    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hgwst"
Jul 07 22:15:29.559802 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:29.559776    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hgwst"
Jul 07 22:15:29.917211 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:29.916953    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:29.917360 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:29.916952    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:29.917360 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:29.917263    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58clx" podUID="28fa7951-7615-4e16-96f8-c918ab062cd3"
Jul 07 22:15:29.917360 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:29.917332    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdtt4" podUID="4128cbcd-5ec2-40d9-81fa-9661440e82df"
Jul 07 22:15:30.158237 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:30.158214    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:15:30.158729 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:30.158638    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mbznv" event={"ID":"0f492bfc-2ea5-4bab-a34b-b6df6736724e","Type":"ContainerStarted","Data":"4241ab7ba6b48c91c83fe1fbfd1bbc647f2dac578fa3c354c1e74a06b8c6bc32"}
Jul 07 22:15:30.158920 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:30.158857    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hgwst"
Jul 07 22:15:30.159206 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:30.159173    2555 scope.go:117] "RemoveContainer" containerID="72cb86ebf297a1b504e82ac024de5c2abc9b90d468385e47f71e93956321fd87"
Jul 07 22:15:30.159533 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:30.159514    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hgwst"
Jul 07 22:15:30.917425 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:30.917341    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:30.917622 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:30.917487    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbgl7" podUID="c248b400-7255-477d-b130-6daf8344f786"
Jul 07 22:15:31.566288 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:31.566127    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hdtt4"]
Jul 07 22:15:31.566697 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:31.566437    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:31.566697 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:31.566552    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdtt4" podUID="4128cbcd-5ec2-40d9-81fa-9661440e82df"
Jul 07 22:15:31.569510 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:31.569485    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nbgl7"]
Jul 07 22:15:31.569617 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:31.569593    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:31.569727 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:31.569704    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbgl7" podUID="c248b400-7255-477d-b130-6daf8344f786"
Jul 07 22:15:31.570107 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:31.570084    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-58clx"]
Jul 07 22:15:31.570201 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:31.570190    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:31.570297 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:31.570280    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58clx" podUID="28fa7951-7615-4e16-96f8-c918ab062cd3"
Jul 07 22:15:32.164227 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:32.164195    2555 generic.go:356] "Generic (PLEG): container finished" podID="e32ca0b6-d3f9-4868-9b27-06e20b06a8ae" containerID="44275bb9869da5d1df4ae189b523b0921c15c0db1527667bb40bd6dcbcaa7946" exitCode=0
Jul 07 22:15:32.164415 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:32.164286    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kz7pz" event={"ID":"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae","Type":"ContainerDied","Data":"44275bb9869da5d1df4ae189b523b0921c15c0db1527667bb40bd6dcbcaa7946"}
Jul 07 22:15:32.168016 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:32.168001    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:15:32.168407 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:32.168387    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mbznv" event={"ID":"0f492bfc-2ea5-4bab-a34b-b6df6736724e","Type":"ContainerStarted","Data":"3d6e97a5211f02ce3245c43ca0de9620eb6f41dabe82703f0620c9c22608513c"}
Jul 07 22:15:32.168884 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:32.168664    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:32.168884 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:32.168749    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:32.168884 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:32.168802    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:32.183527 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:32.183509    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:32.188322 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:32.188304    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:15:32.207602 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:32.207563    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mbznv" podStartSLOduration=10.573616404 podStartE2EDuration="29.207551491s" totalImagesPullingTime="18.633935087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:15:32.207462721 +0000 UTC m=+29.879681708" watchObservedRunningTime="2026-07-07 22:15:32.207551491 +0000 UTC m=+29.879770479"
Jul 07 22:15:32.918286 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:32.918254    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:32.918852 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:32.918337    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:32.918852 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:32.918351    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbgl7" podUID="c248b400-7255-477d-b130-6daf8344f786"
Jul 07 22:15:32.918852 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:32.918451    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58clx" podUID="28fa7951-7615-4e16-96f8-c918ab062cd3"
Jul 07 22:15:32.918852 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:32.918493    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:32.918852 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:32.918556    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdtt4" podUID="4128cbcd-5ec2-40d9-81fa-9661440e82df"
Jul 07 22:15:34.174756 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:34.174722    2555 generic.go:356] "Generic (PLEG): container finished" podID="e32ca0b6-d3f9-4868-9b27-06e20b06a8ae" containerID="ceb735ca10be40d4fb169b9059dd8a7b0c64540c048b5a19f9a832c277f17073" exitCode=0
Jul 07 22:15:34.175425 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:34.174793    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kz7pz" event={"ID":"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae","Type":"ContainerDied","Data":"ceb735ca10be40d4fb169b9059dd8a7b0c64540c048b5a19f9a832c277f17073"}
Jul 07 22:15:34.188195 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:34.188162    2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-mbznv" podUID="0f492bfc-2ea5-4bab-a34b-b6df6736724e" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 07 22:15:34.917217 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:34.917184    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:34.917410 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:34.917220    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:34.917410 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:34.917185    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:34.917410 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:34.917325    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58clx" podUID="28fa7951-7615-4e16-96f8-c918ab062cd3"
Jul 07 22:15:34.917573 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:34.917419    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbgl7" podUID="c248b400-7255-477d-b130-6daf8344f786"
Jul 07 22:15:34.917573 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:34.917510    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdtt4" podUID="4128cbcd-5ec2-40d9-81fa-9661440e82df"
Jul 07 22:15:36.181920 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:36.181655    2555 generic.go:356] "Generic (PLEG): container finished" podID="e32ca0b6-d3f9-4868-9b27-06e20b06a8ae" containerID="ff77fe2c8ae300ef345f71c86a20c349eb9aac7dc0101e871928a09f74ad94d8" exitCode=0
Jul 07 22:15:36.181920 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:36.181716    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kz7pz" event={"ID":"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae","Type":"ContainerDied","Data":"ff77fe2c8ae300ef345f71c86a20c349eb9aac7dc0101e871928a09f74ad94d8"}
Jul 07 22:15:36.515787 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:36.515757    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs\") pod \"network-metrics-daemon-58clx\" (UID: \"28fa7951-7615-4e16-96f8-c918ab062cd3\") " pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:36.515974 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:36.515930    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 22:15:36.516018 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:36.515981    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs podName:28fa7951-7615-4e16-96f8-c918ab062cd3 nodeName:}" failed. No retries permitted until 2026-07-07 22:16:08.515968041 +0000 UTC m=+66.188187007 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs") pod "network-metrics-daemon-58clx" (UID: "28fa7951-7615-4e16-96f8-c918ab062cd3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 22:15:36.616187 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:36.616157    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret\") pod \"global-pull-secret-syncer-hdtt4\" (UID: \"4128cbcd-5ec2-40d9-81fa-9661440e82df\") " pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:36.616187 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:36.616187    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxdbf\" (UniqueName: \"kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf\") pod \"network-check-target-nbgl7\" (UID: \"c248b400-7255-477d-b130-6daf8344f786\") " pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:36.616379 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:36.616295    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 22:15:36.616379 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:36.616345    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 22:15:36.616379 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:36.616353    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret podName:4128cbcd-5ec2-40d9-81fa-9661440e82df nodeName:}" failed. No retries permitted until 2026-07-07 22:16:08.616336027 +0000 UTC m=+66.288555013 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret") pod "global-pull-secret-syncer-hdtt4" (UID: "4128cbcd-5ec2-40d9-81fa-9661440e82df") : object "kube-system"/"original-pull-secret" not registered
Jul 07 22:15:36.616379 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:36.616358    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 22:15:36.616379 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:36.616367    2555 projected.go:196] Error preparing data for projected volume kube-api-access-zxdbf for pod openshift-network-diagnostics/network-check-target-nbgl7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 22:15:36.616557 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:36.616400    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf podName:c248b400-7255-477d-b130-6daf8344f786 nodeName:}" failed. No retries permitted until 2026-07-07 22:16:08.616388677 +0000 UTC m=+66.288607643 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zxdbf" (UniqueName: "kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf") pod "network-check-target-nbgl7" (UID: "c248b400-7255-477d-b130-6daf8344f786") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 22:15:36.916971 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:36.916889    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:36.917129 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:36.917015    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:36.917129 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:36.917057    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:36.917228 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:36.917019    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdtt4" podUID="4128cbcd-5ec2-40d9-81fa-9661440e82df"
Jul 07 22:15:36.917228 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:36.917127    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58clx" podUID="28fa7951-7615-4e16-96f8-c918ab062cd3"
Jul 07 22:15:36.917350 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:36.917213    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbgl7" podUID="c248b400-7255-477d-b130-6daf8344f786"
Jul 07 22:15:37.508300 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.508267    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-159.ec2.internal" event="NodeReady"
Jul 07 22:15:37.508705 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.508415    2555 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 07 22:15:37.541538 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.541508    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b6484bd4d-q59pd"]
Jul 07 22:15:37.543184 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.543170    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.545524 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.545505    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 07 22:15:37.545691 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.545670    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 07 22:15:37.545828 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.545730    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rqfd9\""
Jul 07 22:15:37.546071 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.546048    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 07 22:15:37.552965 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.552918    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 07 22:15:37.554812 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.554792    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-kmnkq"]
Jul 07 22:15:37.556546 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.556531    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kmnkq"
Jul 07 22:15:37.558028 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.558010    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b6484bd4d-q59pd"]
Jul 07 22:15:37.558917 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.558853    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 07 22:15:37.559013 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.558925    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dxzxn\""
Jul 07 22:15:37.559278 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.559234    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 07 22:15:37.562184 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.562164    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gm7mg"]
Jul 07 22:15:37.564295 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.564279    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gm7mg"
Jul 07 22:15:37.566890 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.566831    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 07 22:15:37.566890 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.566855    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 07 22:15:37.567020 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.566856    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 07 22:15:37.567020 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.566912    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 07 22:15:37.567020 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.566912    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hlhsn\""
Jul 07 22:15:37.569067 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.568983    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kmnkq"]
Jul 07 22:15:37.578574 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.578553    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gm7mg"]
Jul 07 22:15:37.658309 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.658277    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-pplwz"]
Jul 07 22:15:37.660089 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.660065    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pplwz"
Jul 07 22:15:37.662726 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.662705    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 07 22:15:37.662726 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.662718    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 07 22:15:37.662923 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.662805    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-s2dph\""
Jul 07 22:15:37.662923 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.662848    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 07 22:15:37.672970 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.672949    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pplwz"]
Jul 07 22:15:37.724772 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.724735    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-registry-certificates\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.724957 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.724787    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24ck4\" (UniqueName: \"kubernetes.io/projected/301d8bec-bcb2-4a26-8c75-059321459a6b-kube-api-access-24ck4\") pod \"dns-default-kmnkq\" (UID: \"301d8bec-bcb2-4a26-8c75-059321459a6b\") " pod="openshift-dns/dns-default-kmnkq"
Jul 07 22:15:37.724957 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.724809    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d8271ced-dc02-4b00-858a-3c99ef4a9a55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gm7mg\" (UID: \"d8271ced-dc02-4b00-858a-3c99ef4a9a55\") " pod="openshift-insights/insights-runtime-extractor-gm7mg"
Jul 07 22:15:37.724957 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.724831    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/301d8bec-bcb2-4a26-8c75-059321459a6b-config-volume\") pod \"dns-default-kmnkq\" (UID: \"301d8bec-bcb2-4a26-8c75-059321459a6b\") " pod="openshift-dns/dns-default-kmnkq"
Jul 07 22:15:37.724957 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.724846    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d8271ced-dc02-4b00-858a-3c99ef4a9a55-crio-socket\") pod \"insights-runtime-extractor-gm7mg\" (UID: \"d8271ced-dc02-4b00-858a-3c99ef4a9a55\") " pod="openshift-insights/insights-runtime-extractor-gm7mg"
Jul 07 22:15:37.724957 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.724886    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d8271ced-dc02-4b00-858a-3c99ef4a9a55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gm7mg\" (UID: \"d8271ced-dc02-4b00-858a-3c99ef4a9a55\") " pod="openshift-insights/insights-runtime-extractor-gm7mg"
Jul 07 22:15:37.724957 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.724912    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/301d8bec-bcb2-4a26-8c75-059321459a6b-tmp-dir\") pod \"dns-default-kmnkq\" (UID: \"301d8bec-bcb2-4a26-8c75-059321459a6b\") " pod="openshift-dns/dns-default-kmnkq"
Jul 07 22:15:37.724957 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.724935    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmcz4\" (UniqueName: \"kubernetes.io/projected/d8271ced-dc02-4b00-858a-3c99ef4a9a55-kube-api-access-lmcz4\") pod \"insights-runtime-extractor-gm7mg\" (UID: \"d8271ced-dc02-4b00-858a-3c99ef4a9a55\") " pod="openshift-insights/insights-runtime-extractor-gm7mg"
Jul 07 22:15:37.724957 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.724959    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-installation-pull-secrets\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.725348 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.724984    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-bound-sa-token\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.725348 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.725008    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsbr4\" (UniqueName: \"kubernetes.io/projected/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-kube-api-access-zsbr4\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.725348 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.725077    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d8271ced-dc02-4b00-858a-3c99ef4a9a55-data-volume\") pod \"insights-runtime-extractor-gm7mg\" (UID: \"d8271ced-dc02-4b00-858a-3c99ef4a9a55\") " pod="openshift-insights/insights-runtime-extractor-gm7mg"
Jul 07 22:15:37.725348 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.725113    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-registry-tls\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.725348 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.725132    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-trusted-ca\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.725348 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.725168    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-image-registry-private-configuration\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.725348 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.725189    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-ca-trust-extracted\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.725348 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.725220    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/301d8bec-bcb2-4a26-8c75-059321459a6b-metrics-tls\") pod \"dns-default-kmnkq\" (UID: \"301d8bec-bcb2-4a26-8c75-059321459a6b\") " pod="openshift-dns/dns-default-kmnkq"
Jul 07 22:15:37.826475 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.826392    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/301d8bec-bcb2-4a26-8c75-059321459a6b-config-volume\") pod \"dns-default-kmnkq\" (UID: \"301d8bec-bcb2-4a26-8c75-059321459a6b\") " pod="openshift-dns/dns-default-kmnkq"
Jul 07 22:15:37.826475 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.826434    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d8271ced-dc02-4b00-858a-3c99ef4a9a55-crio-socket\") pod \"insights-runtime-extractor-gm7mg\" (UID: \"d8271ced-dc02-4b00-858a-3c99ef4a9a55\") " pod="openshift-insights/insights-runtime-extractor-gm7mg"
Jul 07 22:15:37.826475 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.826464    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d8271ced-dc02-4b00-858a-3c99ef4a9a55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gm7mg\" (UID: \"d8271ced-dc02-4b00-858a-3c99ef4a9a55\") " pod="openshift-insights/insights-runtime-extractor-gm7mg"
Jul 07 22:15:37.826741 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.826489    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/301d8bec-bcb2-4a26-8c75-059321459a6b-tmp-dir\") pod \"dns-default-kmnkq\" (UID: \"301d8bec-bcb2-4a26-8c75-059321459a6b\") " pod="openshift-dns/dns-default-kmnkq"
Jul 07 22:15:37.826741 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.826560    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmcz4\" (UniqueName: \"kubernetes.io/projected/d8271ced-dc02-4b00-858a-3c99ef4a9a55-kube-api-access-lmcz4\") pod \"insights-runtime-extractor-gm7mg\" (UID: \"d8271ced-dc02-4b00-858a-3c99ef4a9a55\") " pod="openshift-insights/insights-runtime-extractor-gm7mg"
Jul 07 22:15:37.826741 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.826610    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-installation-pull-secrets\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.826741 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.826636    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-bound-sa-token\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.826741 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.826662    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsbr4\" (UniqueName: \"kubernetes.io/projected/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-kube-api-access-zsbr4\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.826741 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.826680    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d8271ced-dc02-4b00-858a-3c99ef4a9a55-crio-socket\") pod \"insights-runtime-extractor-gm7mg\" (UID: \"d8271ced-dc02-4b00-858a-3c99ef4a9a55\") " pod="openshift-insights/insights-runtime-extractor-gm7mg"
Jul 07 22:15:37.826741 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.826700    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d8271ced-dc02-4b00-858a-3c99ef4a9a55-data-volume\") pod \"insights-runtime-extractor-gm7mg\" (UID: \"d8271ced-dc02-4b00-858a-3c99ef4a9a55\") " pod="openshift-insights/insights-runtime-extractor-gm7mg"
Jul 07 22:15:37.826741 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.826725    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-registry-tls\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.827140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.826749    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-trusted-ca\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.827140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.826799    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-image-registry-private-configuration\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.827140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.826825    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-ca-trust-extracted\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.827140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.826865    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/301d8bec-bcb2-4a26-8c75-059321459a6b-metrics-tls\") pod \"dns-default-kmnkq\" (UID: \"301d8bec-bcb2-4a26-8c75-059321459a6b\") " pod="openshift-dns/dns-default-kmnkq"
Jul 07 22:15:37.827140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.826865    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/301d8bec-bcb2-4a26-8c75-059321459a6b-tmp-dir\") pod \"dns-default-kmnkq\" (UID: \"301d8bec-bcb2-4a26-8c75-059321459a6b\") " pod="openshift-dns/dns-default-kmnkq"
Jul 07 22:15:37.827140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.826908    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea880af9-eea2-4d99-a37f-63e976e118b0-cert\") pod \"ingress-canary-pplwz\" (UID: \"ea880af9-eea2-4d99-a37f-63e976e118b0\") " pod="openshift-ingress-canary/ingress-canary-pplwz"
Jul 07 22:15:37.827140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.826938    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfqdq\" (UniqueName: \"kubernetes.io/projected/ea880af9-eea2-4d99-a37f-63e976e118b0-kube-api-access-sfqdq\") pod \"ingress-canary-pplwz\" (UID: \"ea880af9-eea2-4d99-a37f-63e976e118b0\") " pod="openshift-ingress-canary/ingress-canary-pplwz"
Jul 07 22:15:37.827140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.827005    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-registry-certificates\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.827140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.827050    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24ck4\" (UniqueName: \"kubernetes.io/projected/301d8bec-bcb2-4a26-8c75-059321459a6b-kube-api-access-24ck4\") pod \"dns-default-kmnkq\" (UID: \"301d8bec-bcb2-4a26-8c75-059321459a6b\") " pod="openshift-dns/dns-default-kmnkq"
Jul 07 22:15:37.827140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.827064    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/301d8bec-bcb2-4a26-8c75-059321459a6b-config-volume\") pod \"dns-default-kmnkq\" (UID: \"301d8bec-bcb2-4a26-8c75-059321459a6b\") " pod="openshift-dns/dns-default-kmnkq"
Jul 07 22:15:37.827140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.827075    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d8271ced-dc02-4b00-858a-3c99ef4a9a55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gm7mg\" (UID: \"d8271ced-dc02-4b00-858a-3c99ef4a9a55\") " pod="openshift-insights/insights-runtime-extractor-gm7mg"
Jul 07 22:15:37.827648 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.827419    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d8271ced-dc02-4b00-858a-3c99ef4a9a55-data-volume\") pod \"insights-runtime-extractor-gm7mg\" (UID: \"d8271ced-dc02-4b00-858a-3c99ef4a9a55\") " pod="openshift-insights/insights-runtime-extractor-gm7mg"
Jul 07 22:15:37.827648 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.827615    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d8271ced-dc02-4b00-858a-3c99ef4a9a55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gm7mg\" (UID: \"d8271ced-dc02-4b00-858a-3c99ef4a9a55\") " pod="openshift-insights/insights-runtime-extractor-gm7mg"
Jul 07 22:15:37.828255 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.828227    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-trusted-ca\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.828355 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.828289    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-registry-certificates\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.828530 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.828505    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-ca-trust-extracted\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.831587 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.831562    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-image-registry-private-configuration\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.831587 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.831581    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/301d8bec-bcb2-4a26-8c75-059321459a6b-metrics-tls\") pod \"dns-default-kmnkq\" (UID: \"301d8bec-bcb2-4a26-8c75-059321459a6b\") " pod="openshift-dns/dns-default-kmnkq"
Jul 07 22:15:37.831748 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.831638    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d8271ced-dc02-4b00-858a-3c99ef4a9a55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gm7mg\" (UID: \"d8271ced-dc02-4b00-858a-3c99ef4a9a55\") " pod="openshift-insights/insights-runtime-extractor-gm7mg"
Jul 07 22:15:37.831748 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.831667    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-installation-pull-secrets\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.831843 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.831828    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-registry-tls\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.835003 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.834964    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-bound-sa-token\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.835232 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.835209    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24ck4\" (UniqueName: \"kubernetes.io/projected/301d8bec-bcb2-4a26-8c75-059321459a6b-kube-api-access-24ck4\") pod \"dns-default-kmnkq\" (UID: \"301d8bec-bcb2-4a26-8c75-059321459a6b\") " pod="openshift-dns/dns-default-kmnkq"
Jul 07 22:15:37.835539 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.835517    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsbr4\" (UniqueName: \"kubernetes.io/projected/9ff6ff5b-138c-4f29-ae44-ad7b6e692318-kube-api-access-zsbr4\") pod \"image-registry-6b6484bd4d-q59pd\" (UID: \"9ff6ff5b-138c-4f29-ae44-ad7b6e692318\") " pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.835539 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.835527    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmcz4\" (UniqueName: \"kubernetes.io/projected/d8271ced-dc02-4b00-858a-3c99ef4a9a55-kube-api-access-lmcz4\") pod \"insights-runtime-extractor-gm7mg\" (UID: \"d8271ced-dc02-4b00-858a-3c99ef4a9a55\") " pod="openshift-insights/insights-runtime-extractor-gm7mg"
Jul 07 22:15:37.856609 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.856582    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:37.867178 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.867156    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kmnkq"
Jul 07 22:15:37.876389 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.876366    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gm7mg"
Jul 07 22:15:37.935943 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.935214    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea880af9-eea2-4d99-a37f-63e976e118b0-cert\") pod \"ingress-canary-pplwz\" (UID: \"ea880af9-eea2-4d99-a37f-63e976e118b0\") " pod="openshift-ingress-canary/ingress-canary-pplwz"
Jul 07 22:15:37.935943 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.935259    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfqdq\" (UniqueName: \"kubernetes.io/projected/ea880af9-eea2-4d99-a37f-63e976e118b0-kube-api-access-sfqdq\") pod \"ingress-canary-pplwz\" (UID: \"ea880af9-eea2-4d99-a37f-63e976e118b0\") " pod="openshift-ingress-canary/ingress-canary-pplwz"
Jul 07 22:15:37.943556 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.943460    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea880af9-eea2-4d99-a37f-63e976e118b0-cert\") pod \"ingress-canary-pplwz\" (UID: \"ea880af9-eea2-4d99-a37f-63e976e118b0\") " pod="openshift-ingress-canary/ingress-canary-pplwz"
Jul 07 22:15:37.947743 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.947678    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfqdq\" (UniqueName: \"kubernetes.io/projected/ea880af9-eea2-4d99-a37f-63e976e118b0-kube-api-access-sfqdq\") pod \"ingress-canary-pplwz\" (UID: \"ea880af9-eea2-4d99-a37f-63e976e118b0\") " pod="openshift-ingress-canary/ingress-canary-pplwz"
Jul 07 22:15:37.970381 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:37.970165    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pplwz"
Jul 07 22:15:38.038559 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:38.038526    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b6484bd4d-q59pd"]
Jul 07 22:15:38.046625 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:38.046591    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ff6ff5b_138c_4f29_ae44_ad7b6e692318.slice/crio-41a11b7d81b5ad81224699a79f8a619480a132c88f56c0eae80288c90d8c9f3c WatchSource:0}: Error finding container 41a11b7d81b5ad81224699a79f8a619480a132c88f56c0eae80288c90d8c9f3c: Status 404 returned error can't find the container with id 41a11b7d81b5ad81224699a79f8a619480a132c88f56c0eae80288c90d8c9f3c
Jul 07 22:15:38.066018 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:38.065989    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kmnkq"]
Jul 07 22:15:38.069593 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:38.069568    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod301d8bec_bcb2_4a26_8c75_059321459a6b.slice/crio-bfcf4aa036624f67e93ddd854d37381143627c20f2757cdf55278964b0880e33 WatchSource:0}: Error finding container bfcf4aa036624f67e93ddd854d37381143627c20f2757cdf55278964b0880e33: Status 404 returned error can't find the container with id bfcf4aa036624f67e93ddd854d37381143627c20f2757cdf55278964b0880e33
Jul 07 22:15:38.077601 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:38.077441    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gm7mg"]
Jul 07 22:15:38.083401 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:38.083376    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8271ced_dc02_4b00_858a_3c99ef4a9a55.slice/crio-3a82df1b5bb43bba31bfa37eb0acc8be7f5fdac06188c63cf750d0dd34fb15b2 WatchSource:0}: Error finding container 3a82df1b5bb43bba31bfa37eb0acc8be7f5fdac06188c63cf750d0dd34fb15b2: Status 404 returned error can't find the container with id 3a82df1b5bb43bba31bfa37eb0acc8be7f5fdac06188c63cf750d0dd34fb15b2
Jul 07 22:15:38.125976 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:38.125774    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pplwz"]
Jul 07 22:15:38.128816 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:38.128793    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea880af9_eea2_4d99_a37f_63e976e118b0.slice/crio-766ea26603aa1af9a59f78f4c01264013bfa13ed63a6339ab1a0843f885f1e63 WatchSource:0}: Error finding container 766ea26603aa1af9a59f78f4c01264013bfa13ed63a6339ab1a0843f885f1e63: Status 404 returned error can't find the container with id 766ea26603aa1af9a59f78f4c01264013bfa13ed63a6339ab1a0843f885f1e63
Jul 07 22:15:38.187908 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:38.187864    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pplwz" event={"ID":"ea880af9-eea2-4d99-a37f-63e976e118b0","Type":"ContainerStarted","Data":"766ea26603aa1af9a59f78f4c01264013bfa13ed63a6339ab1a0843f885f1e63"}
Jul 07 22:15:38.188842 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:38.188819    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gm7mg" event={"ID":"d8271ced-dc02-4b00-858a-3c99ef4a9a55","Type":"ContainerStarted","Data":"3a82df1b5bb43bba31bfa37eb0acc8be7f5fdac06188c63cf750d0dd34fb15b2"}
Jul 07 22:15:38.189934 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:38.189905    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kmnkq" event={"ID":"301d8bec-bcb2-4a26-8c75-059321459a6b","Type":"ContainerStarted","Data":"bfcf4aa036624f67e93ddd854d37381143627c20f2757cdf55278964b0880e33"}
Jul 07 22:15:38.196131 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:38.196104    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd" event={"ID":"9ff6ff5b-138c-4f29-ae44-ad7b6e692318","Type":"ContainerStarted","Data":"6c338397fdc7b566aa6a35a00381281ec860f65c8ff6a4cae3dfdbec425b1439"}
Jul 07 22:15:38.196235 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:38.196171    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd" event={"ID":"9ff6ff5b-138c-4f29-ae44-ad7b6e692318","Type":"ContainerStarted","Data":"41a11b7d81b5ad81224699a79f8a619480a132c88f56c0eae80288c90d8c9f3c"}
Jul 07 22:15:38.196294 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:38.196257    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:38.219010 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:38.218962    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd" podStartSLOduration=9.218945875 podStartE2EDuration="9.218945875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:15:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:15:38.218121975 +0000 UTC m=+35.890340995" watchObservedRunningTime="2026-07-07 22:15:38.218945875 +0000 UTC m=+35.891164863"
Jul 07 22:15:38.919742 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:38.917993    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:15:38.919742 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:38.919216    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:15:38.923501 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:38.920546    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:15:38.923501 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:38.921079    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j7fbf\""
Jul 07 22:15:38.923501 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:38.922250    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 22:15:38.923501 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:38.922345    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lkskh\""
Jul 07 22:15:38.923501 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:38.922482    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 22:15:38.923501 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:38.922568    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 22:15:38.923501 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:38.922891    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 22:15:47.221718 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:47.221681    2555 generic.go:356] "Generic (PLEG): container finished" podID="e32ca0b6-d3f9-4868-9b27-06e20b06a8ae" containerID="206fd8b6b0357bf93cd3f96c845539cd78e2ca3ac2bfa4f78b82e44dff723f0a" exitCode=0
Jul 07 22:15:47.222319 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:47.221749    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kz7pz" event={"ID":"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae","Type":"ContainerDied","Data":"206fd8b6b0357bf93cd3f96c845539cd78e2ca3ac2bfa4f78b82e44dff723f0a"}
Jul 07 22:15:47.223341 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:47.223315    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pplwz" event={"ID":"ea880af9-eea2-4d99-a37f-63e976e118b0","Type":"ContainerStarted","Data":"e7081eb58eb294fa71e355eea41de0eea1780b15eec96bce5ef51a179303c5eb"}
Jul 07 22:15:47.224708 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:47.224679    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gm7mg" event={"ID":"d8271ced-dc02-4b00-858a-3c99ef4a9a55","Type":"ContainerStarted","Data":"10bdd2b958d13d9995a19e5c5a88ea4882751e3a68af06038d1cd1146004e22c"}
Jul 07 22:15:47.226254 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:47.226231    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kmnkq" event={"ID":"301d8bec-bcb2-4a26-8c75-059321459a6b","Type":"ContainerStarted","Data":"14019ed6fa23b2f77b1bfc72afc3dac789ae2d2b91a4d7851b48e30847d18dee"}
Jul 07 22:15:47.226333 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:47.226277    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kmnkq" event={"ID":"301d8bec-bcb2-4a26-8c75-059321459a6b","Type":"ContainerStarted","Data":"d9a90a0b13bf53a8f00827f57d3275199281a41f9c0c0f620b0a84f1ca06124e"}
Jul 07 22:15:47.226388 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:47.226373    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-kmnkq"
Jul 07 22:15:47.257320 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:47.257280    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-kmnkq" podStartSLOduration=2.074324081 podStartE2EDuration="10.257268004s" totalImagesPullingTime="8.182943923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:15:47.256367652 +0000 UTC m=+44.928586640" watchObservedRunningTime="2026-07-07 22:15:47.257268004 +0000 UTC m=+44.929486983"
Jul 07 22:15:47.270903 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:47.270840    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-pplwz" podStartSLOduration=2.146262787 podStartE2EDuration="10.270830875s" totalImagesPullingTime="8.124568088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:15:47.270293893 +0000 UTC m=+44.942512881" watchObservedRunningTime="2026-07-07 22:15:47.270830875 +0000 UTC m=+44.943049862"
Jul 07 22:15:48.233638 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:48.233604    2555 generic.go:356] "Generic (PLEG): container finished" podID="e32ca0b6-d3f9-4868-9b27-06e20b06a8ae" containerID="e0a0a3c55fdad9e66745e030567d3342284f87db015c9c532d7837e4a290f5be" exitCode=0
Jul 07 22:15:48.234158 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:48.233685    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kz7pz" event={"ID":"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae","Type":"ContainerDied","Data":"e0a0a3c55fdad9e66745e030567d3342284f87db015c9c532d7837e4a290f5be"}
Jul 07 22:15:48.235778 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:48.235749    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gm7mg" event={"ID":"d8271ced-dc02-4b00-858a-3c99ef4a9a55","Type":"ContainerStarted","Data":"28bb3b842529f771439f7d02c48e0f3fa800c9710a796f1d375ca9c877564fef"}
Jul 07 22:15:49.243291 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:49.243260    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kz7pz" event={"ID":"e32ca0b6-d3f9-4868-9b27-06e20b06a8ae","Type":"ContainerStarted","Data":"d0abb0ba4cea6d01e8dad85d8bc91a5b54d115b594aa1d784b58b72d715739d2"}
Jul 07 22:15:49.265981 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:49.265933    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kz7pz" podStartSLOduration=10.20020966 podStartE2EDuration="47.265918755s" totalImagesPullingTime="37.065709095s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:15:02 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:15:49.265126595 +0000 UTC m=+46.937345586" watchObservedRunningTime="2026-07-07 22:15:49.265918755 +0000 UTC m=+46.938137754"
Jul 07 22:15:51.251724 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:51.251683    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gm7mg" event={"ID":"d8271ced-dc02-4b00-858a-3c99ef4a9a55","Type":"ContainerStarted","Data":"89dd092c1505dc3537188f5b8926c01e5a14e7bdcce469366b3e28ad6fb12708"}
Jul 07 22:15:51.269889 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:51.269829    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gm7mg" podStartSLOduration=1.878135941 podStartE2EDuration="14.269811702s" totalImagesPullingTime="12.391675761s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:15:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:15:51.268137054 +0000 UTC m=+48.940356053" watchObservedRunningTime="2026-07-07 22:15:51.269811702 +0000 UTC m=+48.942030689"
Jul 07 22:15:51.876592 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:51.875109    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4"]
Jul 07 22:15:51.892961 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:51.892935    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4"]
Jul 07 22:15:51.893087 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:51.893050    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4"
Jul 07 22:15:51.895501 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:51.895476    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 07 22:15:51.895638 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:51.895528    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 07 22:15:51.895638 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:51.895573    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 07 22:15:51.895826 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:51.895810    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-lf72b\""
Jul 07 22:15:51.896782 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:51.896768    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 07 22:15:51.896825 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:51.896794    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 07 22:15:51.950447 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:51.950416    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a14cc856-5969-4f2d-866d-96c609b5e2cc-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-7gzb4\" (UID: \"a14cc856-5969-4f2d-866d-96c609b5e2cc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4"
Jul 07 22:15:51.950571 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:51.950460    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a14cc856-5969-4f2d-866d-96c609b5e2cc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-7gzb4\" (UID: \"a14cc856-5969-4f2d-866d-96c609b5e2cc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4"
Jul 07 22:15:51.950571 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:51.950480    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a14cc856-5969-4f2d-866d-96c609b5e2cc-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-7gzb4\" (UID: \"a14cc856-5969-4f2d-866d-96c609b5e2cc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4"
Jul 07 22:15:51.950640 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:51.950572    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m2qf\" (UniqueName: \"kubernetes.io/projected/a14cc856-5969-4f2d-866d-96c609b5e2cc-kube-api-access-4m2qf\") pod \"prometheus-operator-567f7b8b7c-7gzb4\" (UID: \"a14cc856-5969-4f2d-866d-96c609b5e2cc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4"
Jul 07 22:15:52.051696 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:52.051663    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a14cc856-5969-4f2d-866d-96c609b5e2cc-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-7gzb4\" (UID: \"a14cc856-5969-4f2d-866d-96c609b5e2cc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4"
Jul 07 22:15:52.051822 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:52.051717    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a14cc856-5969-4f2d-866d-96c609b5e2cc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-7gzb4\" (UID: \"a14cc856-5969-4f2d-866d-96c609b5e2cc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4"
Jul 07 22:15:52.051822 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:52.051803    2555 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 07 22:15:52.051924 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:52.051821    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a14cc856-5969-4f2d-866d-96c609b5e2cc-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-7gzb4\" (UID: \"a14cc856-5969-4f2d-866d-96c609b5e2cc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4"
Jul 07 22:15:52.051924 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:52.051862    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a14cc856-5969-4f2d-866d-96c609b5e2cc-prometheus-operator-tls podName:a14cc856-5969-4f2d-866d-96c609b5e2cc nodeName:}" failed. No retries permitted until 2026-07-07 22:15:52.551846465 +0000 UTC m=+50.224065432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/a14cc856-5969-4f2d-866d-96c609b5e2cc-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-7gzb4" (UID: "a14cc856-5969-4f2d-866d-96c609b5e2cc") : secret "prometheus-operator-tls" not found
Jul 07 22:15:52.051999 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:52.051954    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m2qf\" (UniqueName: \"kubernetes.io/projected/a14cc856-5969-4f2d-866d-96c609b5e2cc-kube-api-access-4m2qf\") pod \"prometheus-operator-567f7b8b7c-7gzb4\" (UID: \"a14cc856-5969-4f2d-866d-96c609b5e2cc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4"
Jul 07 22:15:52.052373 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:52.052357    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a14cc856-5969-4f2d-866d-96c609b5e2cc-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-7gzb4\" (UID: \"a14cc856-5969-4f2d-866d-96c609b5e2cc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4"
Jul 07 22:15:52.054782 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:52.054753    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a14cc856-5969-4f2d-866d-96c609b5e2cc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-7gzb4\" (UID: \"a14cc856-5969-4f2d-866d-96c609b5e2cc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4"
Jul 07 22:15:52.062669 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:52.062650    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m2qf\" (UniqueName: \"kubernetes.io/projected/a14cc856-5969-4f2d-866d-96c609b5e2cc-kube-api-access-4m2qf\") pod \"prometheus-operator-567f7b8b7c-7gzb4\" (UID: \"a14cc856-5969-4f2d-866d-96c609b5e2cc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4"
Jul 07 22:15:52.555264 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:52.555230    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a14cc856-5969-4f2d-866d-96c609b5e2cc-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-7gzb4\" (UID: \"a14cc856-5969-4f2d-866d-96c609b5e2cc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4"
Jul 07 22:15:52.557510 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:52.557489    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a14cc856-5969-4f2d-866d-96c609b5e2cc-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-7gzb4\" (UID: \"a14cc856-5969-4f2d-866d-96c609b5e2cc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4"
Jul 07 22:15:52.801485 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:52.801453    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4"
Jul 07 22:15:52.933751 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:52.933725    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4"]
Jul 07 22:15:52.937008 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:52.936984    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda14cc856_5969_4f2d_866d_96c609b5e2cc.slice/crio-89de389d5e4dfc6d2d41b5972970553d5e04ed312f604e84491de50088669199 WatchSource:0}: Error finding container 89de389d5e4dfc6d2d41b5972970553d5e04ed312f604e84491de50088669199: Status 404 returned error can't find the container with id 89de389d5e4dfc6d2d41b5972970553d5e04ed312f604e84491de50088669199
Jul 07 22:15:53.258705 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:53.258674    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4" event={"ID":"a14cc856-5969-4f2d-866d-96c609b5e2cc","Type":"ContainerStarted","Data":"89de389d5e4dfc6d2d41b5972970553d5e04ed312f604e84491de50088669199"}
Jul 07 22:15:55.267580 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:55.267548    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4" event={"ID":"a14cc856-5969-4f2d-866d-96c609b5e2cc","Type":"ContainerStarted","Data":"fb7ede9f273d5ba28f9157cd5f07a3badabeb5e37dafcfeafa6dbd79d78df0d5"}
Jul 07 22:15:56.271830 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:56.271789    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4" event={"ID":"a14cc856-5969-4f2d-866d-96c609b5e2cc","Type":"ContainerStarted","Data":"361a0b021b67380da342d35f226710d4eba6b818b7716d88d8319c24456f3f4e"}
Jul 07 22:15:56.288568 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:56.288517    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7gzb4" podStartSLOduration=3.146345491 podStartE2EDuration="5.288502758s" totalImagesPullingTime="2.142157267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:15:56.286828081 +0000 UTC m=+53.959047069" watchObservedRunningTime="2026-07-07 22:15:56.288502758 +0000 UTC m=+53.960721746"
Jul 07 22:15:57.238163 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:57.238130    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-kmnkq"
Jul 07 22:15:57.861489 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:57.861447    2555 patch_prober.go:28] interesting pod/image-registry-6b6484bd4d-q59pd container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 07 22:15:57.861960 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:57.861506    2555 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd" podUID="9ff6ff5b-138c-4f29-ae44-ad7b6e692318" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 22:15:58.275789 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.275760    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mjfl4"]
Jul 07 22:15:58.278342 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.278326    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.280534 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.280513    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 07 22:15:58.280637 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.280556    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 07 22:15:58.280637 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.280564    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-w7tnb\""
Jul 07 22:15:58.280835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.280818    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 07 22:15:58.400672 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.400631    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/21ea1eda-39bd-4514-b0b2-46e947c338f3-node-exporter-wtmp\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.400978 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.400693    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21ea1eda-39bd-4514-b0b2-46e947c338f3-metrics-client-ca\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.400978 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.400824    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnh6h\" (UniqueName: \"kubernetes.io/projected/21ea1eda-39bd-4514-b0b2-46e947c338f3-kube-api-access-pnh6h\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.400978 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.400918    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/21ea1eda-39bd-4514-b0b2-46e947c338f3-node-exporter-tls\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.400978 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.400964    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21ea1eda-39bd-4514-b0b2-46e947c338f3-sys\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.401344 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.400994    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/21ea1eda-39bd-4514-b0b2-46e947c338f3-node-exporter-textfile\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.401344 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.401042    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/21ea1eda-39bd-4514-b0b2-46e947c338f3-node-exporter-accelerators-collector-config\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.401344 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.401114    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/21ea1eda-39bd-4514-b0b2-46e947c338f3-root\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.401344 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.401146    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/21ea1eda-39bd-4514-b0b2-46e947c338f3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.501972 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.501932    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/21ea1eda-39bd-4514-b0b2-46e947c338f3-node-exporter-wtmp\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.501972 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.501974    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21ea1eda-39bd-4514-b0b2-46e947c338f3-metrics-client-ca\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.502208 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.502024    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnh6h\" (UniqueName: \"kubernetes.io/projected/21ea1eda-39bd-4514-b0b2-46e947c338f3-kube-api-access-pnh6h\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.502208 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.502063    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/21ea1eda-39bd-4514-b0b2-46e947c338f3-node-exporter-tls\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.502208 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.502093    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21ea1eda-39bd-4514-b0b2-46e947c338f3-sys\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.502208 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.502115    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/21ea1eda-39bd-4514-b0b2-46e947c338f3-node-exporter-textfile\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.502208 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.502131    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/21ea1eda-39bd-4514-b0b2-46e947c338f3-node-exporter-wtmp\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.502208 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.502151    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/21ea1eda-39bd-4514-b0b2-46e947c338f3-node-exporter-accelerators-collector-config\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.502496 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.502230    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21ea1eda-39bd-4514-b0b2-46e947c338f3-sys\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.502496 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.502249    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/21ea1eda-39bd-4514-b0b2-46e947c338f3-root\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.502496 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.502277    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/21ea1eda-39bd-4514-b0b2-46e947c338f3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.502496 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.502322    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/21ea1eda-39bd-4514-b0b2-46e947c338f3-root\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.502693 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.502575    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/21ea1eda-39bd-4514-b0b2-46e947c338f3-node-exporter-textfile\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.502750 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.502734    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21ea1eda-39bd-4514-b0b2-46e947c338f3-metrics-client-ca\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.502803 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.502764    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/21ea1eda-39bd-4514-b0b2-46e947c338f3-node-exporter-accelerators-collector-config\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.504385 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.504365    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/21ea1eda-39bd-4514-b0b2-46e947c338f3-node-exporter-tls\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.504484 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.504442    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/21ea1eda-39bd-4514-b0b2-46e947c338f3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.509488 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.509468    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnh6h\" (UniqueName: \"kubernetes.io/projected/21ea1eda-39bd-4514-b0b2-46e947c338f3-kube-api-access-pnh6h\") pod \"node-exporter-mjfl4\" (UID: \"21ea1eda-39bd-4514-b0b2-46e947c338f3\") " pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.586686 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:58.586606    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mjfl4"
Jul 07 22:15:58.595952 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:15:58.595929    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21ea1eda_39bd_4514_b0b2_46e947c338f3.slice/crio-1b5a2379221581753809792a78a6aea7ace1b17150237f194f16e3f7e1ec3f99 WatchSource:0}: Error finding container 1b5a2379221581753809792a78a6aea7ace1b17150237f194f16e3f7e1ec3f99: Status 404 returned error can't find the container with id 1b5a2379221581753809792a78a6aea7ace1b17150237f194f16e3f7e1ec3f99
Jul 07 22:15:59.206692 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.206661    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6b6484bd4d-q59pd"
Jul 07 22:15:59.287132 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.287078    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mjfl4" event={"ID":"21ea1eda-39bd-4514-b0b2-46e947c338f3","Type":"ContainerStarted","Data":"1b5a2379221581753809792a78a6aea7ace1b17150237f194f16e3f7e1ec3f99"}
Jul 07 22:15:59.323952 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.323918    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 22:15:59.326850 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.326827    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.329176 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.329154    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 07 22:15:59.329343 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.329154    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 07 22:15:59.329461 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.329447    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 07 22:15:59.329557 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.329453    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 07 22:15:59.329557 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.329512    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 07 22:15:59.329660 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.329568    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 07 22:15:59.329840 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.329808    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mftm9\""
Jul 07 22:15:59.329967 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.329894    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 07 22:15:59.329967 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.329918    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 07 22:15:59.330107 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.330094    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 07 22:15:59.345636 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.345048    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 22:15:59.410478 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.410447    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/358c3c21-d142-490a-a10d-9ef15de87b62-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.410478 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.410482    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/358c3c21-d142-490a-a10d-9ef15de87b62-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.410707 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.410508    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.410707 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.410533    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkfg6\" (UniqueName: \"kubernetes.io/projected/358c3c21-d142-490a-a10d-9ef15de87b62-kube-api-access-jkfg6\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.410707 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.410633    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-web-config\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.410707 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.410668    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/358c3c21-d142-490a-a10d-9ef15de87b62-tls-assets\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.410939 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.410734    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.410939 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.410788    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.410939 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.410814    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/358c3c21-d142-490a-a10d-9ef15de87b62-config-out\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.410939 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.410843    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-config-volume\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.411080 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.410934    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.411080 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.410976    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.411080 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.410994    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/358c3c21-d142-490a-a10d-9ef15de87b62-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.511464 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.511385    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkfg6\" (UniqueName: \"kubernetes.io/projected/358c3c21-d142-490a-a10d-9ef15de87b62-kube-api-access-jkfg6\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.511464 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.511433    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-web-config\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.511464 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.511461    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/358c3c21-d142-490a-a10d-9ef15de87b62-tls-assets\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.511722 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.511507    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.511722 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.511538    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.511722 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.511565    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/358c3c21-d142-490a-a10d-9ef15de87b62-config-out\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.513140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.512066    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-config-volume\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.513140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.512154    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.513140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.512207    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.513140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.512236    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/358c3c21-d142-490a-a10d-9ef15de87b62-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.513140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.512271    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/358c3c21-d142-490a-a10d-9ef15de87b62-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.513140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.512307    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/358c3c21-d142-490a-a10d-9ef15de87b62-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.513140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.512339    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.513140 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:59.512454    2555 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 07 22:15:59.513140 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:59.512511    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-secret-alertmanager-main-tls podName:358c3c21-d142-490a-a10d-9ef15de87b62 nodeName:}" failed. No retries permitted until 2026-07-07 22:16:00.012492103 +0000 UTC m=+57.684711070 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "358c3c21-d142-490a-a10d-9ef15de87b62") : secret "alertmanager-main-tls" not found
Jul 07 22:15:59.513140 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:15:59.512777    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/358c3c21-d142-490a-a10d-9ef15de87b62-alertmanager-trusted-ca-bundle podName:358c3c21-d142-490a-a10d-9ef15de87b62 nodeName:}" failed. No retries permitted until 2026-07-07 22:16:00.012760585 +0000 UTC m=+57.684979552 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/358c3c21-d142-490a-a10d-9ef15de87b62-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "358c3c21-d142-490a-a10d-9ef15de87b62") : configmap references non-existent config key: ca-bundle.crt
Jul 07 22:15:59.513140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.513051    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/358c3c21-d142-490a-a10d-9ef15de87b62-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.513140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.513068    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/358c3c21-d142-490a-a10d-9ef15de87b62-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.514487 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.514426    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/358c3c21-d142-490a-a10d-9ef15de87b62-config-out\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.514602 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.514517    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.514602 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.514585    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/358c3c21-d142-490a-a10d-9ef15de87b62-tls-assets\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.514602 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.514594    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-web-config\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.514763 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.514742    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.516028 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.515985    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.516828 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.516770    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-config-volume\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.516961 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.516942    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:15:59.518862 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:15:59.518842    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkfg6\" (UniqueName: \"kubernetes.io/projected/358c3c21-d142-490a-a10d-9ef15de87b62-kube-api-access-jkfg6\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:16:00.016077 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:00.015995    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/358c3c21-d142-490a-a10d-9ef15de87b62-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:16:00.016077 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:00.016034    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:16:00.017255 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:00.017230    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/358c3c21-d142-490a-a10d-9ef15de87b62-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:16:00.018554 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:00.018524    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/358c3c21-d142-490a-a10d-9ef15de87b62-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"358c3c21-d142-490a-a10d-9ef15de87b62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:16:00.237177 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:00.237145    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 07 22:16:00.292900 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:00.292842    2555 generic.go:356] "Generic (PLEG): container finished" podID="21ea1eda-39bd-4514-b0b2-46e947c338f3" containerID="d60b97990e4e3fb344ee75f11bd30874c81dbf883877a2ad852c5f41929b024c" exitCode=0
Jul 07 22:16:00.293036 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:00.292914    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mjfl4" event={"ID":"21ea1eda-39bd-4514-b0b2-46e947c338f3","Type":"ContainerDied","Data":"d60b97990e4e3fb344ee75f11bd30874c81dbf883877a2ad852c5f41929b024c"}
Jul 07 22:16:00.374586 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:00.374562    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 22:16:00.377841 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:16:00.377820    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod358c3c21_d142_490a_a10d_9ef15de87b62.slice/crio-fac3fd8f6f5c5fc7224a3bcd2086af5cfbd17559e12ce4c08509611c84b9f54c WatchSource:0}: Error finding container fac3fd8f6f5c5fc7224a3bcd2086af5cfbd17559e12ce4c08509611c84b9f54c: Status 404 returned error can't find the container with id fac3fd8f6f5c5fc7224a3bcd2086af5cfbd17559e12ce4c08509611c84b9f54c
Jul 07 22:16:01.298388 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:01.298296    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"358c3c21-d142-490a-a10d-9ef15de87b62","Type":"ContainerStarted","Data":"fac3fd8f6f5c5fc7224a3bcd2086af5cfbd17559e12ce4c08509611c84b9f54c"}
Jul 07 22:16:01.301075 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:01.301046    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mjfl4" event={"ID":"21ea1eda-39bd-4514-b0b2-46e947c338f3","Type":"ContainerStarted","Data":"8ddabc227012d70c42c04089b425f23cca66364eafc7fc70a9a9c18d3aee0b1c"}
Jul 07 22:16:01.301206 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:01.301100    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mjfl4" event={"ID":"21ea1eda-39bd-4514-b0b2-46e947c338f3","Type":"ContainerStarted","Data":"1723186da769999d50739c258a7eb2a4c28c443b3279a8e9d1a495ee6cf70c64"}
Jul 07 22:16:01.321481 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:01.321435    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mjfl4" podStartSLOduration=2.217694468 podStartE2EDuration="3.321419496s" totalImagesPullingTime="1.039906819s" totalInitContainerRuntime="63.818209ms" isStatefulPod=true podCreationTimestamp="2026-07-07 22:15:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:16:01.320438534 +0000 UTC m=+58.992657522" watchObservedRunningTime="2026-07-07 22:16:01.321419496 +0000 UTC m=+58.993638484"
Jul 07 22:16:02.306095 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.306063    2555 generic.go:356] "Generic (PLEG): container finished" podID="358c3c21-d142-490a-a10d-9ef15de87b62" containerID="1ba2525e758497748469694ea83d940403f20be7c9fa87cdc97eefa96d395b21" exitCode=0
Jul 07 22:16:02.306540 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.306150    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"358c3c21-d142-490a-a10d-9ef15de87b62","Type":"ContainerDied","Data":"1ba2525e758497748469694ea83d940403f20be7c9fa87cdc97eefa96d395b21"}
Jul 07 22:16:02.541981 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.541944    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5fcd4db656-kdf9t"]
Jul 07 22:16:02.544877 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.544853    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.547633 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.547602    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 07 22:16:02.547746 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.547635    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-m77kt\""
Jul 07 22:16:02.547746 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.547647    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 07 22:16:02.547886 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.547748    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-aikkldkg6l752\""
Jul 07 22:16:02.547886 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.547861    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 07 22:16:02.548459 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.548444    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 07 22:16:02.555617 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.555592    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5fcd4db656-kdf9t"]
Jul 07 22:16:02.637947 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.637857    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkm42\" (UniqueName: \"kubernetes.io/projected/9291038d-df5f-4bac-8167-1449e10584aa-kube-api-access-rkm42\") pod \"metrics-server-5fcd4db656-kdf9t\" (UID: \"9291038d-df5f-4bac-8167-1449e10584aa\") " pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.638077 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.637975    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9291038d-df5f-4bac-8167-1449e10584aa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5fcd4db656-kdf9t\" (UID: \"9291038d-df5f-4bac-8167-1449e10584aa\") " pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.638077 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.638015    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9291038d-df5f-4bac-8167-1449e10584aa-secret-metrics-server-tls\") pod \"metrics-server-5fcd4db656-kdf9t\" (UID: \"9291038d-df5f-4bac-8167-1449e10584aa\") " pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.638077 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.638042    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9291038d-df5f-4bac-8167-1449e10584aa-client-ca-bundle\") pod \"metrics-server-5fcd4db656-kdf9t\" (UID: \"9291038d-df5f-4bac-8167-1449e10584aa\") " pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.638077 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.638074    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9291038d-df5f-4bac-8167-1449e10584aa-audit-log\") pod \"metrics-server-5fcd4db656-kdf9t\" (UID: \"9291038d-df5f-4bac-8167-1449e10584aa\") " pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.638251 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.638130    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9291038d-df5f-4bac-8167-1449e10584aa-metrics-server-audit-profiles\") pod \"metrics-server-5fcd4db656-kdf9t\" (UID: \"9291038d-df5f-4bac-8167-1449e10584aa\") " pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.638251 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.638150    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9291038d-df5f-4bac-8167-1449e10584aa-secret-metrics-server-client-certs\") pod \"metrics-server-5fcd4db656-kdf9t\" (UID: \"9291038d-df5f-4bac-8167-1449e10584aa\") " pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.739512 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.739471    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9291038d-df5f-4bac-8167-1449e10584aa-metrics-server-audit-profiles\") pod \"metrics-server-5fcd4db656-kdf9t\" (UID: \"9291038d-df5f-4bac-8167-1449e10584aa\") " pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.739692 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.739533    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9291038d-df5f-4bac-8167-1449e10584aa-secret-metrics-server-client-certs\") pod \"metrics-server-5fcd4db656-kdf9t\" (UID: \"9291038d-df5f-4bac-8167-1449e10584aa\") " pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.739692 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.739622    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkm42\" (UniqueName: \"kubernetes.io/projected/9291038d-df5f-4bac-8167-1449e10584aa-kube-api-access-rkm42\") pod \"metrics-server-5fcd4db656-kdf9t\" (UID: \"9291038d-df5f-4bac-8167-1449e10584aa\") " pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.739692 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.739686    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9291038d-df5f-4bac-8167-1449e10584aa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5fcd4db656-kdf9t\" (UID: \"9291038d-df5f-4bac-8167-1449e10584aa\") " pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.739861 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.739742    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9291038d-df5f-4bac-8167-1449e10584aa-secret-metrics-server-tls\") pod \"metrics-server-5fcd4db656-kdf9t\" (UID: \"9291038d-df5f-4bac-8167-1449e10584aa\") " pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.739861 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.739787    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9291038d-df5f-4bac-8167-1449e10584aa-client-ca-bundle\") pod \"metrics-server-5fcd4db656-kdf9t\" (UID: \"9291038d-df5f-4bac-8167-1449e10584aa\") " pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.739861 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.739831    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9291038d-df5f-4bac-8167-1449e10584aa-audit-log\") pod \"metrics-server-5fcd4db656-kdf9t\" (UID: \"9291038d-df5f-4bac-8167-1449e10584aa\") " pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.742474 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.740278    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9291038d-df5f-4bac-8167-1449e10584aa-audit-log\") pod \"metrics-server-5fcd4db656-kdf9t\" (UID: \"9291038d-df5f-4bac-8167-1449e10584aa\") " pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.743268 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.743239    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 07 22:16:02.743406 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.743269    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 07 22:16:02.743406 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.743360    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-aikkldkg6l752\""
Jul 07 22:16:02.743522 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.743507    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 07 22:16:02.743564 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.743557    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 07 22:16:02.749019 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.749000    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkm42\" (UniqueName: \"kubernetes.io/projected/9291038d-df5f-4bac-8167-1449e10584aa-kube-api-access-rkm42\") pod \"metrics-server-5fcd4db656-kdf9t\" (UID: \"9291038d-df5f-4bac-8167-1449e10584aa\") " pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.751517 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.751492    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9291038d-df5f-4bac-8167-1449e10584aa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5fcd4db656-kdf9t\" (UID: \"9291038d-df5f-4bac-8167-1449e10584aa\") " pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.751766 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.751727    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9291038d-df5f-4bac-8167-1449e10584aa-metrics-server-audit-profiles\") pod \"metrics-server-5fcd4db656-kdf9t\" (UID: \"9291038d-df5f-4bac-8167-1449e10584aa\") " pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.752996 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.752973    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9291038d-df5f-4bac-8167-1449e10584aa-client-ca-bundle\") pod \"metrics-server-5fcd4db656-kdf9t\" (UID: \"9291038d-df5f-4bac-8167-1449e10584aa\") " pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.753210 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.753121    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9291038d-df5f-4bac-8167-1449e10584aa-secret-metrics-server-client-certs\") pod \"metrics-server-5fcd4db656-kdf9t\" (UID: \"9291038d-df5f-4bac-8167-1449e10584aa\") " pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.753210 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.753141    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9291038d-df5f-4bac-8167-1449e10584aa-secret-metrics-server-tls\") pod \"metrics-server-5fcd4db656-kdf9t\" (UID: \"9291038d-df5f-4bac-8167-1449e10584aa\") " pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:02.856116 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.856082    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-m77kt\""
Jul 07 22:16:02.864105 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:02.864083    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:03.029604 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:03.029520    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-lrgk5"]
Jul 07 22:16:03.036669 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:16:03.036643    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9291038d_df5f_4bac_8167_1449e10584aa.slice/crio-d006d3fe7ff6b3c7ee3ec2c6bcbe787bee88192c6455892e85784b6cfe5c1bd8 WatchSource:0}: Error finding container d006d3fe7ff6b3c7ee3ec2c6bcbe787bee88192c6455892e85784b6cfe5c1bd8: Status 404 returned error can't find the container with id d006d3fe7ff6b3c7ee3ec2c6bcbe787bee88192c6455892e85784b6cfe5c1bd8
Jul 07 22:16:03.056041 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:03.056019    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5fcd4db656-kdf9t"]
Jul 07 22:16:03.056041 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:03.056053    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-lrgk5"]
Jul 07 22:16:03.056200 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:03.056145    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lrgk5"
Jul 07 22:16:03.058754 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:03.058729    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 07 22:16:03.058754 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:03.058748    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-95qsx\""
Jul 07 22:16:03.144181 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:03.144148    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9fs6\" (UniqueName: \"kubernetes.io/projected/177bad92-0c32-4217-8632-80e203931851-kube-api-access-p9fs6\") pod \"monitoring-plugin-d5dcff7cf-lrgk5\" (UID: \"177bad92-0c32-4217-8632-80e203931851\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lrgk5"
Jul 07 22:16:03.144348 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:03.144220    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/177bad92-0c32-4217-8632-80e203931851-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-lrgk5\" (UID: \"177bad92-0c32-4217-8632-80e203931851\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lrgk5"
Jul 07 22:16:03.245564 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:03.245534    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/177bad92-0c32-4217-8632-80e203931851-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-lrgk5\" (UID: \"177bad92-0c32-4217-8632-80e203931851\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lrgk5"
Jul 07 22:16:03.245718 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:03.245626    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9fs6\" (UniqueName: \"kubernetes.io/projected/177bad92-0c32-4217-8632-80e203931851-kube-api-access-p9fs6\") pod \"monitoring-plugin-d5dcff7cf-lrgk5\" (UID: \"177bad92-0c32-4217-8632-80e203931851\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lrgk5"
Jul 07 22:16:03.245718 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:16:03.245688    2555 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 07 22:16:03.245789 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:16:03.245778    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/177bad92-0c32-4217-8632-80e203931851-monitoring-plugin-cert podName:177bad92-0c32-4217-8632-80e203931851 nodeName:}" failed. No retries permitted until 2026-07-07 22:16:03.745738179 +0000 UTC m=+61.417957147 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/177bad92-0c32-4217-8632-80e203931851-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-lrgk5" (UID: "177bad92-0c32-4217-8632-80e203931851") : secret "monitoring-plugin-cert" not found
Jul 07 22:16:03.254242 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:03.254216    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9fs6\" (UniqueName: \"kubernetes.io/projected/177bad92-0c32-4217-8632-80e203931851-kube-api-access-p9fs6\") pod \"monitoring-plugin-d5dcff7cf-lrgk5\" (UID: \"177bad92-0c32-4217-8632-80e203931851\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lrgk5"
Jul 07 22:16:03.313386 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:03.313351    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t" event={"ID":"9291038d-df5f-4bac-8167-1449e10584aa","Type":"ContainerStarted","Data":"d006d3fe7ff6b3c7ee3ec2c6bcbe787bee88192c6455892e85784b6cfe5c1bd8"}
Jul 07 22:16:03.750304 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:03.750269    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/177bad92-0c32-4217-8632-80e203931851-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-lrgk5\" (UID: \"177bad92-0c32-4217-8632-80e203931851\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lrgk5"
Jul 07 22:16:03.752627 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:03.752607    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/177bad92-0c32-4217-8632-80e203931851-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-lrgk5\" (UID: \"177bad92-0c32-4217-8632-80e203931851\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lrgk5"
Jul 07 22:16:03.965469 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:03.965432    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lrgk5"
Jul 07 22:16:04.127436 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.127379    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-lrgk5"]
Jul 07 22:16:04.130752 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:16:04.130719    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod177bad92_0c32_4217_8632_80e203931851.slice/crio-83aa275f621cff9ba761f84f597b99ede13bf8a28bc0e0ee238efa3b48d620dd WatchSource:0}: Error finding container 83aa275f621cff9ba761f84f597b99ede13bf8a28bc0e0ee238efa3b48d620dd: Status 404 returned error can't find the container with id 83aa275f621cff9ba761f84f597b99ede13bf8a28bc0e0ee238efa3b48d620dd
Jul 07 22:16:04.185815 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.185786    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mbznv"
Jul 07 22:16:04.318500 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.318426    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lrgk5" event={"ID":"177bad92-0c32-4217-8632-80e203931851","Type":"ContainerStarted","Data":"83aa275f621cff9ba761f84f597b99ede13bf8a28bc0e0ee238efa3b48d620dd"}
Jul 07 22:16:04.538299 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.538267    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 22:16:04.540920 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.540891    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.544207 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.543792    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 07 22:16:04.544842 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.544823    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 07 22:16:04.545033 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.545013    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 07 22:16:04.545129 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.545013    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 07 22:16:04.545362 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.545330    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 07 22:16:04.545494 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.545467    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 07 22:16:04.545607 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.545528    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 07 22:16:04.545661 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.545617    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 07 22:16:04.546108 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.545943    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 07 22:16:04.546108 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.546036    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 07 22:16:04.546509 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.546480    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cnn2aucutkldp\""
Jul 07 22:16:04.546780 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.546754    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-d7xdg\""
Jul 07 22:16:04.547157 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.547086    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 07 22:16:04.547549 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.547382    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 07 22:16:04.547685 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.547662    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 07 22:16:04.552297 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.550895    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 07 22:16:04.563109 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.563086    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 22:16:04.659059 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.658978    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2b157f2f-470a-4f09-a74f-6cc238145567-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.659059 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.659027    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b157f2f-470a-4f09-a74f-6cc238145567-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.659059 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.659055    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-web-config\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.659368 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.659129    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.659368 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.659155    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.659368 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.659184    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2b157f2f-470a-4f09-a74f-6cc238145567-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.659368 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.659211    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-config\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.659368 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.659248    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2b157f2f-470a-4f09-a74f-6cc238145567-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.659368 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.659280    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2b157f2f-470a-4f09-a74f-6cc238145567-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.659368 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.659328    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.659773 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.659371    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2b157f2f-470a-4f09-a74f-6cc238145567-config-out\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.659773 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.659445    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b157f2f-470a-4f09-a74f-6cc238145567-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.659773 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.659475    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.659773 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.659507    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brnrz\" (UniqueName: \"kubernetes.io/projected/2b157f2f-470a-4f09-a74f-6cc238145567-kube-api-access-brnrz\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.659773 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.659559    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2b157f2f-470a-4f09-a74f-6cc238145567-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.659773 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.659586    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.659773 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.659608    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2b157f2f-470a-4f09-a74f-6cc238145567-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.659773 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.659631    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.659773 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.659657    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b157f2f-470a-4f09-a74f-6cc238145567-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.659773 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.659687    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.760421 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.760383    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2b157f2f-470a-4f09-a74f-6cc238145567-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.760611 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.760435    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.760611 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.760461    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2b157f2f-470a-4f09-a74f-6cc238145567-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.760611 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.760485    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.760611 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.760512    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b157f2f-470a-4f09-a74f-6cc238145567-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.760611 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.760544    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.760611 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.760585    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2b157f2f-470a-4f09-a74f-6cc238145567-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.760933 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.760619    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b157f2f-470a-4f09-a74f-6cc238145567-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.760933 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.760644    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-web-config\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.760933 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.760702    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.760933 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.760721    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.760933 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.760752    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2b157f2f-470a-4f09-a74f-6cc238145567-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.760933 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.760778    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-config\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.760933 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.760793    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2b157f2f-470a-4f09-a74f-6cc238145567-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.760933 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.760812    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2b157f2f-470a-4f09-a74f-6cc238145567-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.760933 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.760827    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.760933 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.760849    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2b157f2f-470a-4f09-a74f-6cc238145567-config-out\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.760933 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.760895    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b157f2f-470a-4f09-a74f-6cc238145567-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.760933 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.760925    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.762340 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.760966    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brnrz\" (UniqueName: \"kubernetes.io/projected/2b157f2f-470a-4f09-a74f-6cc238145567-kube-api-access-brnrz\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.762340 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.761895    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2b157f2f-470a-4f09-a74f-6cc238145567-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.762340 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.762118    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b157f2f-470a-4f09-a74f-6cc238145567-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.762340 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.762190    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b157f2f-470a-4f09-a74f-6cc238145567-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.763653 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.763334    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2b157f2f-470a-4f09-a74f-6cc238145567-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.764211 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.764155    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-web-config\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.764323 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.764232    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2b157f2f-470a-4f09-a74f-6cc238145567-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.765015 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.764988    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2b157f2f-470a-4f09-a74f-6cc238145567-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.766083 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.766048    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2b157f2f-470a-4f09-a74f-6cc238145567-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.768294 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.767711    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.768294 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.767814    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.768294 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.768219    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.768294 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.768243    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.768532 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.768354    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2b157f2f-470a-4f09-a74f-6cc238145567-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.768723 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.768657    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.769169 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.769129    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-config\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.769706 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.769672    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2b157f2f-470a-4f09-a74f-6cc238145567-config-out\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.770340 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.770310    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b157f2f-470a-4f09-a74f-6cc238145567-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.770968 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.770927    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.771369 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.771346    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2b157f2f-470a-4f09-a74f-6cc238145567-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.772487 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.772462    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brnrz\" (UniqueName: \"kubernetes.io/projected/2b157f2f-470a-4f09-a74f-6cc238145567-kube-api-access-brnrz\") pod \"prometheus-k8s-0\" (UID: \"2b157f2f-470a-4f09-a74f-6cc238145567\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:04.856119 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:04.856079    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:05.564769 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:05.564651    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 22:16:05.570959 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:16:05.570927    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b157f2f_470a_4f09_a74f_6cc238145567.slice/crio-efd381a02fbb29b6b4adc48cf289f5b789880c6ed9712b0fed9b06cebedac324 WatchSource:0}: Error finding container efd381a02fbb29b6b4adc48cf289f5b789880c6ed9712b0fed9b06cebedac324: Status 404 returned error can't find the container with id efd381a02fbb29b6b4adc48cf289f5b789880c6ed9712b0fed9b06cebedac324
Jul 07 22:16:06.336148 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:06.336110    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"358c3c21-d142-490a-a10d-9ef15de87b62","Type":"ContainerStarted","Data":"c981b20394e418a16b8cba208df5592e260be34027826b60e8b9f9591acee6c2"}
Jul 07 22:16:06.336311 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:06.336191    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"358c3c21-d142-490a-a10d-9ef15de87b62","Type":"ContainerStarted","Data":"80a4ff4ba431a4182194a082426be281a06c817035c3ccb885e9fa61834854f3"}
Jul 07 22:16:06.336311 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:06.336230    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"358c3c21-d142-490a-a10d-9ef15de87b62","Type":"ContainerStarted","Data":"4c795d8101d48ca9bfa0654e5ec4413bec265bb5dde4cd8ec73ab2767a75b2a2"}
Jul 07 22:16:06.336311 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:06.336265    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"358c3c21-d142-490a-a10d-9ef15de87b62","Type":"ContainerStarted","Data":"742c28390ac03c6f26083a9a4c187b44a7620ebc344334ef65037046b047652b"}
Jul 07 22:16:06.336311 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:06.336301    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"358c3c21-d142-490a-a10d-9ef15de87b62","Type":"ContainerStarted","Data":"08278ac32c8cbaa74add291a6336a73ac7b905ccaa7adff1584f7ca11a43130b"}
Jul 07 22:16:06.337935 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:06.337902    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lrgk5" event={"ID":"177bad92-0c32-4217-8632-80e203931851","Type":"ContainerStarted","Data":"bcb0e3ee9abf6491cf4e1a9570fb631361e563eb9a33b4306fcbbc49c2d33fcf"}
Jul 07 22:16:06.338337 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:06.338292    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lrgk5"
Jul 07 22:16:06.339998 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:06.339973    2555 generic.go:356] "Generic (PLEG): container finished" podID="2b157f2f-470a-4f09-a74f-6cc238145567" containerID="e099ac3f7c27c74050abfa690191296b07a7a2c0d10ecfec41b6a2a741e6d443" exitCode=0
Jul 07 22:16:06.340159 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:06.340047    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2b157f2f-470a-4f09-a74f-6cc238145567","Type":"ContainerDied","Data":"e099ac3f7c27c74050abfa690191296b07a7a2c0d10ecfec41b6a2a741e6d443"}
Jul 07 22:16:06.340159 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:06.340093    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2b157f2f-470a-4f09-a74f-6cc238145567","Type":"ContainerStarted","Data":"efd381a02fbb29b6b4adc48cf289f5b789880c6ed9712b0fed9b06cebedac324"}
Jul 07 22:16:06.342116 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:06.342005    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t" event={"ID":"9291038d-df5f-4bac-8167-1449e10584aa","Type":"ContainerStarted","Data":"34da1201733be8a68211579e2ec1e63e1557b75e73c0a8739c0392166b335dce"}
Jul 07 22:16:06.343543 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:06.343525    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lrgk5"
Jul 07 22:16:06.355042 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:06.354991    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lrgk5" podStartSLOduration=1.320704551 podStartE2EDuration="3.354973784s" totalImagesPullingTime="2.034269233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:16:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:16:06.354660933 +0000 UTC m=+64.026879922" watchObservedRunningTime="2026-07-07 22:16:06.354973784 +0000 UTC m=+64.027192776"
Jul 07 22:16:06.386894 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:06.386817    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t" podStartSLOduration=2.054380301 podStartE2EDuration="4.386795931s" totalImagesPullingTime="2.33241563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:16:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:16:06.385938871 +0000 UTC m=+64.058157868" watchObservedRunningTime="2026-07-07 22:16:06.386795931 +0000 UTC m=+64.059014923"
Jul 07 22:16:07.353142 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:07.353097    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"358c3c21-d142-490a-a10d-9ef15de87b62","Type":"ContainerStarted","Data":"580741413c94497afd46ba969b690f7f160b94e8314b161acfe9a2db5f903de7"}
Jul 07 22:16:07.379711 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:07.379657    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.183486422 podStartE2EDuration="8.379618752s" totalImagesPullingTime="5.180575844s" totalInitContainerRuntime="15.556486ms" isStatefulPod=false podCreationTimestamp="2026-07-07 22:15:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:16:07.378309897 +0000 UTC m=+65.050528886" watchObservedRunningTime="2026-07-07 22:16:07.379618752 +0000 UTC m=+65.051837741"
Jul 07 22:16:08.598834 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:08.598801    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs\") pod \"network-metrics-daemon-58clx\" (UID: \"28fa7951-7615-4e16-96f8-c918ab062cd3\") " pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:16:08.601577 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:08.601555    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 22:16:08.612340 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:08.612317    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28fa7951-7615-4e16-96f8-c918ab062cd3-metrics-certs\") pod \"network-metrics-daemon-58clx\" (UID: \"28fa7951-7615-4e16-96f8-c918ab062cd3\") " pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:16:08.648862 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:08.648836    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lkskh\""
Jul 07 22:16:08.657011 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:08.656988    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58clx"
Jul 07 22:16:08.700015 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:08.699977    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret\") pod \"global-pull-secret-syncer-hdtt4\" (UID: \"4128cbcd-5ec2-40d9-81fa-9661440e82df\") " pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:16:08.700251 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:08.700027    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxdbf\" (UniqueName: \"kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf\") pod \"network-check-target-nbgl7\" (UID: \"c248b400-7255-477d-b130-6daf8344f786\") " pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:16:08.702787 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:08.702763    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 22:16:08.702904 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:08.702810    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 22:16:08.712595 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:08.712557    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 22:16:08.712792 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:08.712773    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4128cbcd-5ec2-40d9-81fa-9661440e82df-original-pull-secret\") pod \"global-pull-secret-syncer-hdtt4\" (UID: \"4128cbcd-5ec2-40d9-81fa-9661440e82df\") " pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:16:08.723065 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:08.723044    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxdbf\" (UniqueName: \"kubernetes.io/projected/c248b400-7255-477d-b130-6daf8344f786-kube-api-access-zxdbf\") pod \"network-check-target-nbgl7\" (UID: \"c248b400-7255-477d-b130-6daf8344f786\") " pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:16:08.939437 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:08.939361    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j7fbf\""
Jul 07 22:16:08.947459 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:08.947434    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:16:08.954618 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:08.954597    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdtt4"
Jul 07 22:16:09.873108 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:09.873080    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nbgl7"]
Jul 07 22:16:09.876171 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:16:09.876143    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc248b400_7255_477d_b130_6daf8344f786.slice/crio-54e3ba350b0e413274aeff029e0f266e844893b2dee096fba8b1b1860cf83ac3 WatchSource:0}: Error finding container 54e3ba350b0e413274aeff029e0f266e844893b2dee096fba8b1b1860cf83ac3: Status 404 returned error can't find the container with id 54e3ba350b0e413274aeff029e0f266e844893b2dee096fba8b1b1860cf83ac3
Jul 07 22:16:10.100725 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:10.100654    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-58clx"]
Jul 07 22:16:10.103442 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:10.103375    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hdtt4"]
Jul 07 22:16:10.103648 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:16:10.103582    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28fa7951_7615_4e16_96f8_c918ab062cd3.slice/crio-74b5d1797cf66fe3d24fa98e38c174a0e7552b5236b0cc397e01a516ce12ca3c WatchSource:0}: Error finding container 74b5d1797cf66fe3d24fa98e38c174a0e7552b5236b0cc397e01a516ce12ca3c: Status 404 returned error can't find the container with id 74b5d1797cf66fe3d24fa98e38c174a0e7552b5236b0cc397e01a516ce12ca3c
Jul 07 22:16:10.106016 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:16:10.105986    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4128cbcd_5ec2_40d9_81fa_9661440e82df.slice/crio-c42b4ee42a2c4a2f2cd629aacd7403ac4200100aef80b9085fa2366e1327441a WatchSource:0}: Error finding container c42b4ee42a2c4a2f2cd629aacd7403ac4200100aef80b9085fa2366e1327441a: Status 404 returned error can't find the container with id c42b4ee42a2c4a2f2cd629aacd7403ac4200100aef80b9085fa2366e1327441a
Jul 07 22:16:10.370909 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:10.369822    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2b157f2f-470a-4f09-a74f-6cc238145567","Type":"ContainerStarted","Data":"e7000945082d7ccb2357cc82493eac916c0be89f90028eac62e36520bc9c9651"}
Jul 07 22:16:10.370909 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:10.369936    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2b157f2f-470a-4f09-a74f-6cc238145567","Type":"ContainerStarted","Data":"d0c2b0f7e0ebafa58172dc94ef8dc1a10fbaaa5666f23b9531f2cc201d01ef66"}
Jul 07 22:16:10.371656 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:10.371625    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nbgl7" event={"ID":"c248b400-7255-477d-b130-6daf8344f786","Type":"ContainerStarted","Data":"54e3ba350b0e413274aeff029e0f266e844893b2dee096fba8b1b1860cf83ac3"}
Jul 07 22:16:10.373180 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:10.373159    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hdtt4" event={"ID":"4128cbcd-5ec2-40d9-81fa-9661440e82df","Type":"ContainerStarted","Data":"c42b4ee42a2c4a2f2cd629aacd7403ac4200100aef80b9085fa2366e1327441a"}
Jul 07 22:16:10.374318 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:10.374294    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-58clx" event={"ID":"28fa7951-7615-4e16-96f8-c918ab062cd3","Type":"ContainerStarted","Data":"74b5d1797cf66fe3d24fa98e38c174a0e7552b5236b0cc397e01a516ce12ca3c"}
Jul 07 22:16:12.389717 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:12.389402    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-58clx" event={"ID":"28fa7951-7615-4e16-96f8-c918ab062cd3","Type":"ContainerStarted","Data":"ef016ae990127424d56ea2490f5b763a17cb5579a583a3115b0403ed22f65185"}
Jul 07 22:16:12.389717 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:12.389466    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-58clx" event={"ID":"28fa7951-7615-4e16-96f8-c918ab062cd3","Type":"ContainerStarted","Data":"4cb20983c21cc8bfd8beb356090ddd71ef9c614f5eb861e745679c635fa4de3e"}
Jul 07 22:16:12.407447 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:12.407387    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-58clx" podStartSLOduration=68.259666605 podStartE2EDuration="1m9.407366049s" totalImagesPullingTime="1.147699444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:16:12.403520674 +0000 UTC m=+70.075739676" watchObservedRunningTime="2026-07-07 22:16:12.407366049 +0000 UTC m=+70.079585040"
Jul 07 22:16:15.402808 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:15.402773    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nbgl7" event={"ID":"c248b400-7255-477d-b130-6daf8344f786","Type":"ContainerStarted","Data":"f5834347a50a95d5eaac2654b6faa13d733260b979a9f34d8fc8edb93ecbcccf"}
Jul 07 22:16:16.409574 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:16.409537    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2b157f2f-470a-4f09-a74f-6cc238145567","Type":"ContainerStarted","Data":"ca59c0a2dca2e890db4a11685a7d6e21f5d393655ce04c18bc04eff003bdb09f"}
Jul 07 22:16:16.410067 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:16.409618    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2b157f2f-470a-4f09-a74f-6cc238145567","Type":"ContainerStarted","Data":"2417377de21378df6d69dbf9c77faa936914357bee84f3c90e45fa160240e9cf"}
Jul 07 22:16:16.410067 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:16.409661    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2b157f2f-470a-4f09-a74f-6cc238145567","Type":"ContainerStarted","Data":"03e7bc4b0f15c2ec8b3e762c91a085410b7e2e63599448c823120c71d61d2a15"}
Jul 07 22:16:16.410067 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:16.409696    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2b157f2f-470a-4f09-a74f-6cc238145567","Type":"ContainerStarted","Data":"6966646ae996ac07f3230e59a7021acebbb4e62095d4563441a8eb6bbc4dc5a2"}
Jul 07 22:16:16.411039 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:16.411013    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hdtt4" event={"ID":"4128cbcd-5ec2-40d9-81fa-9661440e82df","Type":"ContainerStarted","Data":"e075519608eabb724a5ea24a1791a4b4d730b888bb04e15299e87ec632dfda24"}
Jul 07 22:16:16.438663 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:16.438621    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.6123572900000003 podStartE2EDuration="12.43860586s" totalImagesPullingTime="8.803292465s" totalInitContainerRuntime="22.956105ms" isStatefulPod=false podCreationTimestamp="2026-07-07 22:16:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:16:16.436615404 +0000 UTC m=+74.108834392" watchObservedRunningTime="2026-07-07 22:16:16.43860586 +0000 UTC m=+74.110824851"
Jul 07 22:16:16.451384 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:16.451345    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hdtt4" podStartSLOduration=68.251281094 podStartE2EDuration="1m13.451330658s" totalImagesPullingTime="5.200049564s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:16:16.450928163 +0000 UTC m=+74.123147151" watchObservedRunningTime="2026-07-07 22:16:16.451330658 +0000 UTC m=+74.123549647"
Jul 07 22:16:16.465654 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:16.465613    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nbgl7" podStartSLOduration=68.047363415 podStartE2EDuration="1m13.465600671s" totalImagesPullingTime="5.418237256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:16:16.464749218 +0000 UTC m=+74.136968220" watchObservedRunningTime="2026-07-07 22:16:16.465600671 +0000 UTC m=+74.137819660"
Jul 07 22:16:19.856853 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:19.856816    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:16:22.625010 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:22.624950    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pplwz_ea880af9-eea2-4d99-a37f-63e976e118b0/serve-healthcheck-canary/0.log"
Jul 07 22:16:22.864656 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:22.864618    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:22.864851 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:22.864715    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:26.411577 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:26.411422    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:16:42.869903 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:42.869859    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:42.873762 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:42.873740    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5fcd4db656-kdf9t"
Jul 07 22:16:46.414792 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:16:46.414763    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nbgl7"
Jul 07 22:17:04.857057 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:17:04.857015    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:17:04.875638 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:17:04.875606    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:17:05.624075 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:17:05.624049    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 22:19:54.421633 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.421596    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"]
Jul 07 22:19:54.423686 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.423669    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"
Jul 07 22:19:54.429730 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.429706    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 07 22:19:54.430970 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.430948    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 07 22:19:54.431091 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.431075    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-clx4d\""
Jul 07 22:19:54.431163 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.431082    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 07 22:19:54.431490 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.431471    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 07 22:19:54.449360 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.449334    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"]
Jul 07 22:19:54.512317 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.512282    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f58441f9-4810-4b47-b7c7-5b700dc11727-manifests\") pod \"opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg\" (UID: \"f58441f9-4810-4b47-b7c7-5b700dc11727\") " pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"
Jul 07 22:19:54.512317 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.512318    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f58441f9-4810-4b47-b7c7-5b700dc11727-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg\" (UID: \"f58441f9-4810-4b47-b7c7-5b700dc11727\") " pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"
Jul 07 22:19:54.512524 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.512351    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f58441f9-4810-4b47-b7c7-5b700dc11727-tmp\") pod \"opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg\" (UID: \"f58441f9-4810-4b47-b7c7-5b700dc11727\") " pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"
Jul 07 22:19:54.512524 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.512391    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f58441f9-4810-4b47-b7c7-5b700dc11727-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg\" (UID: \"f58441f9-4810-4b47-b7c7-5b700dc11727\") " pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"
Jul 07 22:19:54.512524 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.512451    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwfvj\" (UniqueName: \"kubernetes.io/projected/f58441f9-4810-4b47-b7c7-5b700dc11727-kube-api-access-mwfvj\") pod \"opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg\" (UID: \"f58441f9-4810-4b47-b7c7-5b700dc11727\") " pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"
Jul 07 22:19:54.613599 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.613562    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f58441f9-4810-4b47-b7c7-5b700dc11727-manifests\") pod \"opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg\" (UID: \"f58441f9-4810-4b47-b7c7-5b700dc11727\") " pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"
Jul 07 22:19:54.613599 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.613604    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f58441f9-4810-4b47-b7c7-5b700dc11727-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg\" (UID: \"f58441f9-4810-4b47-b7c7-5b700dc11727\") " pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"
Jul 07 22:19:54.613842 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.613644    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f58441f9-4810-4b47-b7c7-5b700dc11727-tmp\") pod \"opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg\" (UID: \"f58441f9-4810-4b47-b7c7-5b700dc11727\") " pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"
Jul 07 22:19:54.613842 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.613667    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f58441f9-4810-4b47-b7c7-5b700dc11727-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg\" (UID: \"f58441f9-4810-4b47-b7c7-5b700dc11727\") " pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"
Jul 07 22:19:54.613842 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.613740    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwfvj\" (UniqueName: \"kubernetes.io/projected/f58441f9-4810-4b47-b7c7-5b700dc11727-kube-api-access-mwfvj\") pod \"opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg\" (UID: \"f58441f9-4810-4b47-b7c7-5b700dc11727\") " pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"
Jul 07 22:19:54.614017 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.613965    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f58441f9-4810-4b47-b7c7-5b700dc11727-manifests\") pod \"opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg\" (UID: \"f58441f9-4810-4b47-b7c7-5b700dc11727\") " pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"
Jul 07 22:19:54.615936 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.615907    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f58441f9-4810-4b47-b7c7-5b700dc11727-tmp\") pod \"opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg\" (UID: \"f58441f9-4810-4b47-b7c7-5b700dc11727\") " pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"
Jul 07 22:19:54.616165 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.616146    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f58441f9-4810-4b47-b7c7-5b700dc11727-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg\" (UID: \"f58441f9-4810-4b47-b7c7-5b700dc11727\") " pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"
Jul 07 22:19:54.616229 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.616166    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f58441f9-4810-4b47-b7c7-5b700dc11727-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg\" (UID: \"f58441f9-4810-4b47-b7c7-5b700dc11727\") " pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"
Jul 07 22:19:54.621437 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.621416    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwfvj\" (UniqueName: \"kubernetes.io/projected/f58441f9-4810-4b47-b7c7-5b700dc11727-kube-api-access-mwfvj\") pod \"opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg\" (UID: \"f58441f9-4810-4b47-b7c7-5b700dc11727\") " pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"
Jul 07 22:19:54.734277 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.734239    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"
Jul 07 22:19:54.864047 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:54.863831    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"]
Jul 07 22:19:54.866689 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:19:54.866647    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf58441f9_4810_4b47_b7c7_5b700dc11727.slice/crio-e368861c0277527ea616f5fd4617506c956133780b9944d5b47e36c598774cbc WatchSource:0}: Error finding container e368861c0277527ea616f5fd4617506c956133780b9944d5b47e36c598774cbc: Status 404 returned error can't find the container with id e368861c0277527ea616f5fd4617506c956133780b9944d5b47e36c598774cbc
Jul 07 22:19:55.196348 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:55.196269    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg" event={"ID":"f58441f9-4810-4b47-b7c7-5b700dc11727","Type":"ContainerStarted","Data":"e368861c0277527ea616f5fd4617506c956133780b9944d5b47e36c598774cbc"}
Jul 07 22:19:58.209911 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:58.209863    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg" event={"ID":"f58441f9-4810-4b47-b7c7-5b700dc11727","Type":"ContainerStarted","Data":"c7f52c49549524306851f0ad918d7c14be75abf86c1e90eb228dab7de7fc17e8"}
Jul 07 22:19:59.215022 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:59.214990    2555 generic.go:356] "Generic (PLEG): container finished" podID="f58441f9-4810-4b47-b7c7-5b700dc11727" containerID="c7f52c49549524306851f0ad918d7c14be75abf86c1e90eb228dab7de7fc17e8" exitCode=0
Jul 07 22:19:59.215396 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:19:59.215070    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg" event={"ID":"f58441f9-4810-4b47-b7c7-5b700dc11727","Type":"ContainerDied","Data":"c7f52c49549524306851f0ad918d7c14be75abf86c1e90eb228dab7de7fc17e8"}
Jul 07 22:20:00.220728 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:00.220685    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg" event={"ID":"f58441f9-4810-4b47-b7c7-5b700dc11727","Type":"ContainerStarted","Data":"14b4ba7fe31c4cd7231c210999459f54b3f5b531613c56ec4e45bf5bd895fa1c"}
Jul 07 22:20:00.221142 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:00.220779    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"
Jul 07 22:20:00.242603 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:00.242557    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg" podStartSLOduration=2.048430056 podStartE2EDuration="6.242542727s" totalImagesPullingTime="3.192170675s" totalInitContainerRuntime="1.001941996s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:19:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:20:00.241101914 +0000 UTC m=+297.913320902" watchObservedRunningTime="2026-07-07 22:20:00.242542727 +0000 UTC m=+297.914761713"
Jul 07 22:20:02.766960 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:02.766935    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:20:02.768255 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:02.768224    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:20:02.770724 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:02.770704    2555 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 22:20:11.227050 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:11.227022    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg"
Jul 07 22:20:13.760615 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:13.760583    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-776659d659-w7qkt"]
Jul 07 22:20:13.762627 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:13.762607    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-776659d659-w7qkt"
Jul 07 22:20:13.765534 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:13.765512    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 07 22:20:13.765639 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:13.765524    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 07 22:20:13.765778 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:13.765757    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 07 22:20:13.766524 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:13.766508    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 07 22:20:13.778765 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:13.778739    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-776659d659-w7qkt"]
Jul 07 22:20:13.877229 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:13.877193    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f615c2a4-314a-413d-a879-6c8c38d9b332-tls-certs\") pod \"kube-auth-proxy-776659d659-w7qkt\" (UID: \"f615c2a4-314a-413d-a879-6c8c38d9b332\") " pod="openshift-ingress/kube-auth-proxy-776659d659-w7qkt"
Jul 07 22:20:13.877397 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:13.877253    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnmxq\" (UniqueName: \"kubernetes.io/projected/f615c2a4-314a-413d-a879-6c8c38d9b332-kube-api-access-gnmxq\") pod \"kube-auth-proxy-776659d659-w7qkt\" (UID: \"f615c2a4-314a-413d-a879-6c8c38d9b332\") " pod="openshift-ingress/kube-auth-proxy-776659d659-w7qkt"
Jul 07 22:20:13.877397 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:13.877281    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f615c2a4-314a-413d-a879-6c8c38d9b332-tmp\") pod \"kube-auth-proxy-776659d659-w7qkt\" (UID: \"f615c2a4-314a-413d-a879-6c8c38d9b332\") " pod="openshift-ingress/kube-auth-proxy-776659d659-w7qkt"
Jul 07 22:20:13.978226 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:13.978194    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnmxq\" (UniqueName: \"kubernetes.io/projected/f615c2a4-314a-413d-a879-6c8c38d9b332-kube-api-access-gnmxq\") pod \"kube-auth-proxy-776659d659-w7qkt\" (UID: \"f615c2a4-314a-413d-a879-6c8c38d9b332\") " pod="openshift-ingress/kube-auth-proxy-776659d659-w7qkt"
Jul 07 22:20:13.978226 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:13.978232    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f615c2a4-314a-413d-a879-6c8c38d9b332-tmp\") pod \"kube-auth-proxy-776659d659-w7qkt\" (UID: \"f615c2a4-314a-413d-a879-6c8c38d9b332\") " pod="openshift-ingress/kube-auth-proxy-776659d659-w7qkt"
Jul 07 22:20:13.978486 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:13.978271    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f615c2a4-314a-413d-a879-6c8c38d9b332-tls-certs\") pod \"kube-auth-proxy-776659d659-w7qkt\" (UID: \"f615c2a4-314a-413d-a879-6c8c38d9b332\") " pod="openshift-ingress/kube-auth-proxy-776659d659-w7qkt"
Jul 07 22:20:13.978486 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:20:13.978373    2555 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 07 22:20:13.978486 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:20:13.978423    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f615c2a4-314a-413d-a879-6c8c38d9b332-tls-certs podName:f615c2a4-314a-413d-a879-6c8c38d9b332 nodeName:}" failed. No retries permitted until 2026-07-07 22:20:14.478408647 +0000 UTC m=+312.150627613 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/f615c2a4-314a-413d-a879-6c8c38d9b332-tls-certs") pod "kube-auth-proxy-776659d659-w7qkt" (UID: "f615c2a4-314a-413d-a879-6c8c38d9b332") : secret "kube-auth-proxy-tls" not found
Jul 07 22:20:13.980498 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:13.980473    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f615c2a4-314a-413d-a879-6c8c38d9b332-tmp\") pod \"kube-auth-proxy-776659d659-w7qkt\" (UID: \"f615c2a4-314a-413d-a879-6c8c38d9b332\") " pod="openshift-ingress/kube-auth-proxy-776659d659-w7qkt"
Jul 07 22:20:13.993439 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:13.993415    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnmxq\" (UniqueName: \"kubernetes.io/projected/f615c2a4-314a-413d-a879-6c8c38d9b332-kube-api-access-gnmxq\") pod \"kube-auth-proxy-776659d659-w7qkt\" (UID: \"f615c2a4-314a-413d-a879-6c8c38d9b332\") " pod="openshift-ingress/kube-auth-proxy-776659d659-w7qkt"
Jul 07 22:20:14.482623 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:14.482592    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f615c2a4-314a-413d-a879-6c8c38d9b332-tls-certs\") pod \"kube-auth-proxy-776659d659-w7qkt\" (UID: \"f615c2a4-314a-413d-a879-6c8c38d9b332\") " pod="openshift-ingress/kube-auth-proxy-776659d659-w7qkt"
Jul 07 22:20:14.484981 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:14.484962    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f615c2a4-314a-413d-a879-6c8c38d9b332-tls-certs\") pod \"kube-auth-proxy-776659d659-w7qkt\" (UID: \"f615c2a4-314a-413d-a879-6c8c38d9b332\") " pod="openshift-ingress/kube-auth-proxy-776659d659-w7qkt"
Jul 07 22:20:14.672222 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:14.672183    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-776659d659-w7qkt"
Jul 07 22:20:14.799477 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:14.799456    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-776659d659-w7qkt"]
Jul 07 22:20:14.801445 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:20:14.801424    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf615c2a4_314a_413d_a879_6c8c38d9b332.slice/crio-e45b4625d35faff79bb71170399ebe59f2694e53c6644c373aad607c71f37cc7 WatchSource:0}: Error finding container e45b4625d35faff79bb71170399ebe59f2694e53c6644c373aad607c71f37cc7: Status 404 returned error can't find the container with id e45b4625d35faff79bb71170399ebe59f2694e53c6644c373aad607c71f37cc7
Jul 07 22:20:14.803013 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:14.802998    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 22:20:15.102298 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:15.102216    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-82b4p"]
Jul 07 22:20:15.104994 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:15.104975    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-82b4p"
Jul 07 22:20:15.108018 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:15.108000    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-fj68w\""
Jul 07 22:20:15.108123 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:15.108053    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 07 22:20:15.126592 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:15.125697    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-82b4p"]
Jul 07 22:20:15.186914 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:15.186861    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8lb5\" (UniqueName: \"kubernetes.io/projected/ff7041e3-a577-42d2-ba13-27baf97a5b02-kube-api-access-s8lb5\") pod \"model-serving-api-54d89bd5fc-82b4p\" (UID: \"ff7041e3-a577-42d2-ba13-27baf97a5b02\") " pod="opendatahub/model-serving-api-54d89bd5fc-82b4p"
Jul 07 22:20:15.187077 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:15.186957    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff7041e3-a577-42d2-ba13-27baf97a5b02-tls-certs\") pod \"model-serving-api-54d89bd5fc-82b4p\" (UID: \"ff7041e3-a577-42d2-ba13-27baf97a5b02\") " pod="opendatahub/model-serving-api-54d89bd5fc-82b4p"
Jul 07 22:20:15.277741 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:15.277703    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-776659d659-w7qkt" event={"ID":"f615c2a4-314a-413d-a879-6c8c38d9b332","Type":"ContainerStarted","Data":"e45b4625d35faff79bb71170399ebe59f2694e53c6644c373aad607c71f37cc7"}
Jul 07 22:20:15.288193 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:15.288174    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8lb5\" (UniqueName: \"kubernetes.io/projected/ff7041e3-a577-42d2-ba13-27baf97a5b02-kube-api-access-s8lb5\") pod \"model-serving-api-54d89bd5fc-82b4p\" (UID: \"ff7041e3-a577-42d2-ba13-27baf97a5b02\") " pod="opendatahub/model-serving-api-54d89bd5fc-82b4p"
Jul 07 22:20:15.288285 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:15.288227    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff7041e3-a577-42d2-ba13-27baf97a5b02-tls-certs\") pod \"model-serving-api-54d89bd5fc-82b4p\" (UID: \"ff7041e3-a577-42d2-ba13-27baf97a5b02\") " pod="opendatahub/model-serving-api-54d89bd5fc-82b4p"
Jul 07 22:20:15.290598 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:15.290578    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff7041e3-a577-42d2-ba13-27baf97a5b02-tls-certs\") pod \"model-serving-api-54d89bd5fc-82b4p\" (UID: \"ff7041e3-a577-42d2-ba13-27baf97a5b02\") " pod="opendatahub/model-serving-api-54d89bd5fc-82b4p"
Jul 07 22:20:15.296514 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:15.296491    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8lb5\" (UniqueName: \"kubernetes.io/projected/ff7041e3-a577-42d2-ba13-27baf97a5b02-kube-api-access-s8lb5\") pod \"model-serving-api-54d89bd5fc-82b4p\" (UID: \"ff7041e3-a577-42d2-ba13-27baf97a5b02\") " pod="opendatahub/model-serving-api-54d89bd5fc-82b4p"
Jul 07 22:20:15.415689 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:15.415590    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-82b4p"
Jul 07 22:20:15.543429 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:15.543404    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-82b4p"]
Jul 07 22:20:15.545233 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:20:15.545209    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff7041e3_a577_42d2_ba13_27baf97a5b02.slice/crio-8096cf2e5d7e24878ce3bf5a5247497801958a73597c8ad3ac47defd0c1d60d2 WatchSource:0}: Error finding container 8096cf2e5d7e24878ce3bf5a5247497801958a73597c8ad3ac47defd0c1d60d2: Status 404 returned error can't find the container with id 8096cf2e5d7e24878ce3bf5a5247497801958a73597c8ad3ac47defd0c1d60d2
Jul 07 22:20:16.282798 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:16.282757    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-82b4p" event={"ID":"ff7041e3-a577-42d2-ba13-27baf97a5b02","Type":"ContainerStarted","Data":"8096cf2e5d7e24878ce3bf5a5247497801958a73597c8ad3ac47defd0c1d60d2"}
Jul 07 22:20:18.292844 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.292764    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-82b4p" event={"ID":"ff7041e3-a577-42d2-ba13-27baf97a5b02","Type":"ContainerStarted","Data":"897cb957fa4006b9ce0fc101d3da229fdff885d658253279764d1fc444b312df"}
Jul 07 22:20:18.293276 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.292854    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-82b4p"
Jul 07 22:20:18.294251 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.294229    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-776659d659-w7qkt" event={"ID":"f615c2a4-314a-413d-a879-6c8c38d9b332","Type":"ContainerStarted","Data":"f75a259ef6b5164200a0173de019fb9c4dc007d26b321781848c66659ffdcf72"}
Jul 07 22:20:18.311055 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.311007    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-82b4p" podStartSLOduration=0.836449829 podStartE2EDuration="3.310993892s" totalImagesPullingTime="2.474544063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:20:18.30946046 +0000 UTC m=+315.981679449" watchObservedRunningTime="2026-07-07 22:20:18.310993892 +0000 UTC m=+315.983212881"
Jul 07 22:20:18.326492 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.326445    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-776659d659-w7qkt" podStartSLOduration=2.10091013 podStartE2EDuration="5.326433701s" totalImagesPullingTime="3.225523571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:20:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:20:18.326128529 +0000 UTC m=+315.998347518" watchObservedRunningTime="2026-07-07 22:20:18.326433701 +0000 UTC m=+315.998652693"
Jul 07 22:20:18.521184 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.521155    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw"]
Jul 07 22:20:18.524284 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.524268    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw"
Jul 07 22:20:18.526891 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.526855    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 07 22:20:18.527020 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.526962    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 07 22:20:18.527020 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.526978    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 07 22:20:18.527141 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.527045    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-frkrl\""
Jul 07 22:20:18.527194 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.527152    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 07 22:20:18.527252 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.527231    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 07 22:20:18.532736 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.532714    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw"]
Jul 07 22:20:18.613677 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.613596    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1fcc44f-3c50-4ca1-aa24-c5cc957015dc-metrics-cert\") pod \"lws-controller-manager-6f8cffc658-xzwjw\" (UID: \"e1fcc44f-3c50-4ca1-aa24-c5cc957015dc\") " pod="openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw"
Jul 07 22:20:18.613677 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.613644    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1fcc44f-3c50-4ca1-aa24-c5cc957015dc-cert\") pod \"lws-controller-manager-6f8cffc658-xzwjw\" (UID: \"e1fcc44f-3c50-4ca1-aa24-c5cc957015dc\") " pod="openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw"
Jul 07 22:20:18.613677 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.613674    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e1fcc44f-3c50-4ca1-aa24-c5cc957015dc-manager-config\") pod \"lws-controller-manager-6f8cffc658-xzwjw\" (UID: \"e1fcc44f-3c50-4ca1-aa24-c5cc957015dc\") " pod="openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw"
Jul 07 22:20:18.613911 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.613693    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkzts\" (UniqueName: \"kubernetes.io/projected/e1fcc44f-3c50-4ca1-aa24-c5cc957015dc-kube-api-access-jkzts\") pod \"lws-controller-manager-6f8cffc658-xzwjw\" (UID: \"e1fcc44f-3c50-4ca1-aa24-c5cc957015dc\") " pod="openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw"
Jul 07 22:20:18.715113 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.715080    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1fcc44f-3c50-4ca1-aa24-c5cc957015dc-metrics-cert\") pod \"lws-controller-manager-6f8cffc658-xzwjw\" (UID: \"e1fcc44f-3c50-4ca1-aa24-c5cc957015dc\") " pod="openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw"
Jul 07 22:20:18.715265 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.715128    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1fcc44f-3c50-4ca1-aa24-c5cc957015dc-cert\") pod \"lws-controller-manager-6f8cffc658-xzwjw\" (UID: \"e1fcc44f-3c50-4ca1-aa24-c5cc957015dc\") " pod="openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw"
Jul 07 22:20:18.715265 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.715242    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e1fcc44f-3c50-4ca1-aa24-c5cc957015dc-manager-config\") pod \"lws-controller-manager-6f8cffc658-xzwjw\" (UID: \"e1fcc44f-3c50-4ca1-aa24-c5cc957015dc\") " pod="openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw"
Jul 07 22:20:18.715344 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.715270    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkzts\" (UniqueName: \"kubernetes.io/projected/e1fcc44f-3c50-4ca1-aa24-c5cc957015dc-kube-api-access-jkzts\") pod \"lws-controller-manager-6f8cffc658-xzwjw\" (UID: \"e1fcc44f-3c50-4ca1-aa24-c5cc957015dc\") " pod="openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw"
Jul 07 22:20:18.715920 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.715900    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e1fcc44f-3c50-4ca1-aa24-c5cc957015dc-manager-config\") pod \"lws-controller-manager-6f8cffc658-xzwjw\" (UID: \"e1fcc44f-3c50-4ca1-aa24-c5cc957015dc\") " pod="openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw"
Jul 07 22:20:18.717485 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.717459    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1fcc44f-3c50-4ca1-aa24-c5cc957015dc-cert\") pod \"lws-controller-manager-6f8cffc658-xzwjw\" (UID: \"e1fcc44f-3c50-4ca1-aa24-c5cc957015dc\") " pod="openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw"
Jul 07 22:20:18.717569 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.717550    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1fcc44f-3c50-4ca1-aa24-c5cc957015dc-metrics-cert\") pod \"lws-controller-manager-6f8cffc658-xzwjw\" (UID: \"e1fcc44f-3c50-4ca1-aa24-c5cc957015dc\") " pod="openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw"
Jul 07 22:20:18.722421 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.722399    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkzts\" (UniqueName: \"kubernetes.io/projected/e1fcc44f-3c50-4ca1-aa24-c5cc957015dc-kube-api-access-jkzts\") pod \"lws-controller-manager-6f8cffc658-xzwjw\" (UID: \"e1fcc44f-3c50-4ca1-aa24-c5cc957015dc\") " pod="openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw"
Jul 07 22:20:18.833725 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.833695    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw"
Jul 07 22:20:18.969247 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:18.969224    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw"]
Jul 07 22:20:18.971388 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:20:18.971364    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1fcc44f_3c50_4ca1_aa24_c5cc957015dc.slice/crio-42190bd989beb64574d7520ec847d9be59d9eac73bbcafa0b9a33fa76bdfcd89 WatchSource:0}: Error finding container 42190bd989beb64574d7520ec847d9be59d9eac73bbcafa0b9a33fa76bdfcd89: Status 404 returned error can't find the container with id 42190bd989beb64574d7520ec847d9be59d9eac73bbcafa0b9a33fa76bdfcd89
Jul 07 22:20:19.299379 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:19.299337    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw" event={"ID":"e1fcc44f-3c50-4ca1-aa24-c5cc957015dc","Type":"ContainerStarted","Data":"42190bd989beb64574d7520ec847d9be59d9eac73bbcafa0b9a33fa76bdfcd89"}
Jul 07 22:20:21.311451 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:21.311416    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw" event={"ID":"e1fcc44f-3c50-4ca1-aa24-c5cc957015dc","Type":"ContainerStarted","Data":"ccebe58d93528359dc7d2332afaf66005eaf9e0bab6a467e6dfc4a9c47953c79"}
Jul 07 22:20:21.311852 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:21.311527    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw"
Jul 07 22:20:21.330137 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:21.330087    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw" podStartSLOduration=1.071928536 podStartE2EDuration="3.330069704s" totalImagesPullingTime="2.258141168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:20:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:20:21.329247079 +0000 UTC m=+319.001466079" watchObservedRunningTime="2026-07-07 22:20:21.330069704 +0000 UTC m=+319.002288695"
Jul 07 22:20:29.303048 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:29.303022    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-82b4p"
Jul 07 22:20:32.318555 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:20:32.318525    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6f8cffc658-xzwjw"
Jul 07 22:21:51.246805 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:21:51.246774    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"]
Jul 07 22:21:51.250071 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:21:51.250055    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"
Jul 07 22:21:51.254073 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:21:51.254047    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 22:21:51.254191 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:21:51.254099    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 22:21:51.254191 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:21:51.254127    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xs4cq\""
Jul 07 22:21:51.264917 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:21:51.264856    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"]
Jul 07 22:21:51.405445 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:21:51.405409    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/17ab7b75-c04c-44c3-a87a-1cf82712ba22-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xdjdk\" (UID: \"17ab7b75-c04c-44c3-a87a-1cf82712ba22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"
Jul 07 22:21:51.405622 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:21:51.405451    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsqxs\" (UniqueName: \"kubernetes.io/projected/17ab7b75-c04c-44c3-a87a-1cf82712ba22-kube-api-access-lsqxs\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xdjdk\" (UID: \"17ab7b75-c04c-44c3-a87a-1cf82712ba22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"
Jul 07 22:21:51.506307 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:21:51.506226    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/17ab7b75-c04c-44c3-a87a-1cf82712ba22-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xdjdk\" (UID: \"17ab7b75-c04c-44c3-a87a-1cf82712ba22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"
Jul 07 22:21:51.506307 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:21:51.506261    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsqxs\" (UniqueName: \"kubernetes.io/projected/17ab7b75-c04c-44c3-a87a-1cf82712ba22-kube-api-access-lsqxs\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xdjdk\" (UID: \"17ab7b75-c04c-44c3-a87a-1cf82712ba22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"
Jul 07 22:21:51.506604 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:21:51.506585    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/17ab7b75-c04c-44c3-a87a-1cf82712ba22-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xdjdk\" (UID: \"17ab7b75-c04c-44c3-a87a-1cf82712ba22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"
Jul 07 22:21:51.517939 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:21:51.517916    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsqxs\" (UniqueName: \"kubernetes.io/projected/17ab7b75-c04c-44c3-a87a-1cf82712ba22-kube-api-access-lsqxs\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xdjdk\" (UID: \"17ab7b75-c04c-44c3-a87a-1cf82712ba22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"
Jul 07 22:21:51.559829 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:21:51.559793    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"
Jul 07 22:21:51.697486 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:21:51.697453    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"]
Jul 07 22:21:51.700130 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:21:51.700095    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17ab7b75_c04c_44c3_a87a_1cf82712ba22.slice/crio-119de58079be25c79214bb32c0162d77297254020245ce8da24fbe7ae5f34579 WatchSource:0}: Error finding container 119de58079be25c79214bb32c0162d77297254020245ce8da24fbe7ae5f34579: Status 404 returned error can't find the container with id 119de58079be25c79214bb32c0162d77297254020245ce8da24fbe7ae5f34579
Jul 07 22:21:52.673354 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:21:52.673315    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk" event={"ID":"17ab7b75-c04c-44c3-a87a-1cf82712ba22","Type":"ContainerStarted","Data":"119de58079be25c79214bb32c0162d77297254020245ce8da24fbe7ae5f34579"}
Jul 07 22:21:58.699827 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:21:58.699793    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk" event={"ID":"17ab7b75-c04c-44c3-a87a-1cf82712ba22","Type":"ContainerStarted","Data":"cf3eab847a15ad23139a723e2da77e41f3a303fe8cd01595ff383a84b6d38d6e"}
Jul 07 22:21:58.700197 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:21:58.699903    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"
Jul 07 22:21:58.719829 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:21:58.719784    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk" podStartSLOduration=1.72268729 podStartE2EDuration="7.719769022s" totalImagesPullingTime="5.997081732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:21:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:21:58.717737955 +0000 UTC m=+416.389956940" watchObservedRunningTime="2026-07-07 22:21:58.719769022 +0000 UTC m=+416.391988010"
Jul 07 22:22:00.474840 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:00.474806    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-skpm9"]
Jul 07 22:22:00.478128 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:00.478108    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-skpm9"
Jul 07 22:22:00.481056 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:00.481033    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 07 22:22:00.481184 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:00.481043    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-f55xt\""
Jul 07 22:22:00.481184 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:00.481043    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 07 22:22:00.486712 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:00.486688    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-skpm9"]
Jul 07 22:22:00.584420 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:00.584387    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv5jq\" (UniqueName: \"kubernetes.io/projected/7aa4edef-63b9-49ee-ba93-9637cbde4b50-kube-api-access-sv5jq\") pod \"kuadrant-console-plugin-66b5774985-skpm9\" (UID: \"7aa4edef-63b9-49ee-ba93-9637cbde4b50\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-skpm9"
Jul 07 22:22:00.584420 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:00.584430    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7aa4edef-63b9-49ee-ba93-9637cbde4b50-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-skpm9\" (UID: \"7aa4edef-63b9-49ee-ba93-9637cbde4b50\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-skpm9"
Jul 07 22:22:00.584644 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:00.584512    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7aa4edef-63b9-49ee-ba93-9637cbde4b50-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-skpm9\" (UID: \"7aa4edef-63b9-49ee-ba93-9637cbde4b50\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-skpm9"
Jul 07 22:22:00.685590 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:00.685557    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7aa4edef-63b9-49ee-ba93-9637cbde4b50-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-skpm9\" (UID: \"7aa4edef-63b9-49ee-ba93-9637cbde4b50\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-skpm9"
Jul 07 22:22:00.685775 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:00.685610    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv5jq\" (UniqueName: \"kubernetes.io/projected/7aa4edef-63b9-49ee-ba93-9637cbde4b50-kube-api-access-sv5jq\") pod \"kuadrant-console-plugin-66b5774985-skpm9\" (UID: \"7aa4edef-63b9-49ee-ba93-9637cbde4b50\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-skpm9"
Jul 07 22:22:00.685775 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:00.685643    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7aa4edef-63b9-49ee-ba93-9637cbde4b50-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-skpm9\" (UID: \"7aa4edef-63b9-49ee-ba93-9637cbde4b50\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-skpm9"
Jul 07 22:22:00.686284 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:00.686261    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7aa4edef-63b9-49ee-ba93-9637cbde4b50-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-skpm9\" (UID: \"7aa4edef-63b9-49ee-ba93-9637cbde4b50\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-skpm9"
Jul 07 22:22:00.687932 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:00.687913    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7aa4edef-63b9-49ee-ba93-9637cbde4b50-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-skpm9\" (UID: \"7aa4edef-63b9-49ee-ba93-9637cbde4b50\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-skpm9"
Jul 07 22:22:00.696113 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:00.696093    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv5jq\" (UniqueName: \"kubernetes.io/projected/7aa4edef-63b9-49ee-ba93-9637cbde4b50-kube-api-access-sv5jq\") pod \"kuadrant-console-plugin-66b5774985-skpm9\" (UID: \"7aa4edef-63b9-49ee-ba93-9637cbde4b50\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-skpm9"
Jul 07 22:22:00.809921 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:00.809803    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-skpm9"
Jul 07 22:22:00.943534 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:00.943509    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-skpm9"]
Jul 07 22:22:00.945846 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:22:00.945814    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7aa4edef_63b9_49ee_ba93_9637cbde4b50.slice/crio-6b21cd48fc0c769aa2d49638ed74ff7ecdf0b36c64d964091b79662ebe7f6495 WatchSource:0}: Error finding container 6b21cd48fc0c769aa2d49638ed74ff7ecdf0b36c64d964091b79662ebe7f6495: Status 404 returned error can't find the container with id 6b21cd48fc0c769aa2d49638ed74ff7ecdf0b36c64d964091b79662ebe7f6495
Jul 07 22:22:01.715522 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:01.715486    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-skpm9" event={"ID":"7aa4edef-63b9-49ee-ba93-9637cbde4b50","Type":"ContainerStarted","Data":"6b21cd48fc0c769aa2d49638ed74ff7ecdf0b36c64d964091b79662ebe7f6495"}
Jul 07 22:22:09.706769 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:09.706739    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"
Jul 07 22:22:11.307934 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.307900    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"]
Jul 07 22:22:11.308418 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.308155    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk" podUID="17ab7b75-c04c-44c3-a87a-1cf82712ba22" containerName="manager" containerID="cri-o://cf3eab847a15ad23139a723e2da77e41f3a303fe8cd01595ff383a84b6d38d6e" gracePeriod=2
Jul 07 22:22:11.314240 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.314214    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4"]
Jul 07 22:22:11.330560 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.330530    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"]
Jul 07 22:22:11.330700 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.330644    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4"
Jul 07 22:22:11.334859 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:22:11.334822    2555 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"17ab7b75-c04c-44c3-a87a-1cf82712ba22\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-xdjdk\": pods \"kuadrant-operator-controller-manager-8684c9ff49-xdjdk\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"
Jul 07 22:22:11.345286 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.345228    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4"]
Jul 07 22:22:11.347003 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.346977    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb"]
Jul 07 22:22:11.347394 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.347369    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17ab7b75-c04c-44c3-a87a-1cf82712ba22" containerName="manager"
Jul 07 22:22:11.347394 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.347390    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="17ab7b75-c04c-44c3-a87a-1cf82712ba22" containerName="manager"
Jul 07 22:22:11.347547 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.347469    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="17ab7b75-c04c-44c3-a87a-1cf82712ba22" containerName="manager"
Jul 07 22:22:11.350637 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.350615    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb"
Jul 07 22:22:11.384612 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.384582    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klb6l\" (UniqueName: \"kubernetes.io/projected/4aefb39e-3576-4e0d-9af0-a91f2e249893-kube-api-access-klb6l\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lkbf4\" (UID: \"4aefb39e-3576-4e0d-9af0-a91f2e249893\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4"
Jul 07 22:22:11.384763 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.384647    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4aefb39e-3576-4e0d-9af0-a91f2e249893-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lkbf4\" (UID: \"4aefb39e-3576-4e0d-9af0-a91f2e249893\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4"
Jul 07 22:22:11.388300 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.388268    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb"]
Jul 07 22:22:11.394923 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:22:11.394862    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xdjdk\" is forbidden: User \"system:node:ip-10-0-128-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-159.ec2.internal' and this object" podUID="17ab7b75-c04c-44c3-a87a-1cf82712ba22" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"
Jul 07 22:22:11.397157 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:22:11.397113    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xdjdk\" is forbidden: User \"system:node:ip-10-0-128-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-159.ec2.internal' and this object" podUID="17ab7b75-c04c-44c3-a87a-1cf82712ba22" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"
Jul 07 22:22:11.485102 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.485070    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klb6l\" (UniqueName: \"kubernetes.io/projected/4aefb39e-3576-4e0d-9af0-a91f2e249893-kube-api-access-klb6l\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lkbf4\" (UID: \"4aefb39e-3576-4e0d-9af0-a91f2e249893\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4"
Jul 07 22:22:11.485288 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.485124    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4aefb39e-3576-4e0d-9af0-a91f2e249893-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lkbf4\" (UID: \"4aefb39e-3576-4e0d-9af0-a91f2e249893\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4"
Jul 07 22:22:11.485288 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.485170    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9586\" (UniqueName: \"kubernetes.io/projected/daa848f3-2535-4b90-9738-205902373a61-kube-api-access-z9586\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gs5fb\" (UID: \"daa848f3-2535-4b90-9738-205902373a61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb"
Jul 07 22:22:11.485288 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.485236    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/daa848f3-2535-4b90-9738-205902373a61-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gs5fb\" (UID: \"daa848f3-2535-4b90-9738-205902373a61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb"
Jul 07 22:22:11.485540 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.485521    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4aefb39e-3576-4e0d-9af0-a91f2e249893-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lkbf4\" (UID: \"4aefb39e-3576-4e0d-9af0-a91f2e249893\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4"
Jul 07 22:22:11.503613 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.503581    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klb6l\" (UniqueName: \"kubernetes.io/projected/4aefb39e-3576-4e0d-9af0-a91f2e249893-kube-api-access-klb6l\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lkbf4\" (UID: \"4aefb39e-3576-4e0d-9af0-a91f2e249893\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4"
Jul 07 22:22:11.586239 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.586148    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9586\" (UniqueName: \"kubernetes.io/projected/daa848f3-2535-4b90-9738-205902373a61-kube-api-access-z9586\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gs5fb\" (UID: \"daa848f3-2535-4b90-9738-205902373a61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb"
Jul 07 22:22:11.586403 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.586248    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/daa848f3-2535-4b90-9738-205902373a61-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gs5fb\" (UID: \"daa848f3-2535-4b90-9738-205902373a61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb"
Jul 07 22:22:11.586672 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.586653    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/daa848f3-2535-4b90-9738-205902373a61-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gs5fb\" (UID: \"daa848f3-2535-4b90-9738-205902373a61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb"
Jul 07 22:22:11.606721 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.606692    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9586\" (UniqueName: \"kubernetes.io/projected/daa848f3-2535-4b90-9738-205902373a61-kube-api-access-z9586\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gs5fb\" (UID: \"daa848f3-2535-4b90-9738-205902373a61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb"
Jul 07 22:22:11.685831 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.685794    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4"
Jul 07 22:22:11.695795 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:11.695775    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb"
Jul 07 22:22:12.921260 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:22:12.921216    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xdjdk\" is forbidden: User \"system:node:ip-10-0-128-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-159.ec2.internal' and this object" podUID="17ab7b75-c04c-44c3-a87a-1cf82712ba22" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"
Jul 07 22:22:23.284162 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.284134    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"
Jul 07 22:22:23.286352 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:22:23.286326    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xdjdk\" is forbidden: User \"system:node:ip-10-0-128-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-159.ec2.internal' and this object" podUID="17ab7b75-c04c-44c3-a87a-1cf82712ba22" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"
Jul 07 22:22:23.397000 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.396973    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsqxs\" (UniqueName: \"kubernetes.io/projected/17ab7b75-c04c-44c3-a87a-1cf82712ba22-kube-api-access-lsqxs\") pod \"17ab7b75-c04c-44c3-a87a-1cf82712ba22\" (UID: \"17ab7b75-c04c-44c3-a87a-1cf82712ba22\") "
Jul 07 22:22:23.397131 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.397025    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/17ab7b75-c04c-44c3-a87a-1cf82712ba22-extensions-socket-volume\") pod \"17ab7b75-c04c-44c3-a87a-1cf82712ba22\" (UID: \"17ab7b75-c04c-44c3-a87a-1cf82712ba22\") "
Jul 07 22:22:23.397391 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.397367    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17ab7b75-c04c-44c3-a87a-1cf82712ba22-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "17ab7b75-c04c-44c3-a87a-1cf82712ba22" (UID: "17ab7b75-c04c-44c3-a87a-1cf82712ba22"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:22:23.398927 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.398907    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17ab7b75-c04c-44c3-a87a-1cf82712ba22-kube-api-access-lsqxs" (OuterVolumeSpecName: "kube-api-access-lsqxs") pod "17ab7b75-c04c-44c3-a87a-1cf82712ba22" (UID: "17ab7b75-c04c-44c3-a87a-1cf82712ba22"). InnerVolumeSpecName "kube-api-access-lsqxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:22:23.432759 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.432723    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb"]
Jul 07 22:22:23.433725 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:22:23.433695    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaa848f3_2535_4b90_9738_205902373a61.slice/crio-ba6cab25e756646fb59fdab221b5eeffdb9ed63e0f5998ac59b9e7a484865567 WatchSource:0}: Error finding container ba6cab25e756646fb59fdab221b5eeffdb9ed63e0f5998ac59b9e7a484865567: Status 404 returned error can't find the container with id ba6cab25e756646fb59fdab221b5eeffdb9ed63e0f5998ac59b9e7a484865567
Jul 07 22:22:23.434393 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.434370    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4"]
Jul 07 22:22:23.434482 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:22:23.434427    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aefb39e_3576_4e0d_9af0_a91f2e249893.slice/crio-729af4412b2cbd4588977cb9653607c92f1038d45c6df5385ea1a3fb350cf80f WatchSource:0}: Error finding container 729af4412b2cbd4588977cb9653607c92f1038d45c6df5385ea1a3fb350cf80f: Status 404 returned error can't find the container with id 729af4412b2cbd4588977cb9653607c92f1038d45c6df5385ea1a3fb350cf80f
Jul 07 22:22:23.498481 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.498462    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsqxs\" (UniqueName: \"kubernetes.io/projected/17ab7b75-c04c-44c3-a87a-1cf82712ba22-kube-api-access-lsqxs\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:22:23.498481 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.498482    2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/17ab7b75-c04c-44c3-a87a-1cf82712ba22-extensions-socket-volume\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:22:23.825627 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.825586    2555 generic.go:356] "Generic (PLEG): container finished" podID="17ab7b75-c04c-44c3-a87a-1cf82712ba22" containerID="cf3eab847a15ad23139a723e2da77e41f3a303fe8cd01595ff383a84b6d38d6e" exitCode=0
Jul 07 22:22:23.825785 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.825636    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"
Jul 07 22:22:23.825785 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.825684    2555 scope.go:117] "RemoveContainer" containerID="cf3eab847a15ad23139a723e2da77e41f3a303fe8cd01595ff383a84b6d38d6e"
Jul 07 22:22:23.827336 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.827305    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-skpm9" event={"ID":"7aa4edef-63b9-49ee-ba93-9637cbde4b50","Type":"ContainerStarted","Data":"79cc274e648e5b20f6b496899eaa4da66acc5b79c28640f4daf294965042c48c"}
Jul 07 22:22:23.828117 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:22:23.828087    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xdjdk\" is forbidden: User \"system:node:ip-10-0-128-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-159.ec2.internal' and this object" podUID="17ab7b75-c04c-44c3-a87a-1cf82712ba22" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"
Jul 07 22:22:23.829180 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.829154    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb" event={"ID":"daa848f3-2535-4b90-9738-205902373a61","Type":"ContainerStarted","Data":"9438c98a954fedfbb93173ff69adb9703f52977dec61dbc50b1415da15261e61"}
Jul 07 22:22:23.829267 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.829201    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb" event={"ID":"daa848f3-2535-4b90-9738-205902373a61","Type":"ContainerStarted","Data":"ba6cab25e756646fb59fdab221b5eeffdb9ed63e0f5998ac59b9e7a484865567"}
Jul 07 22:22:23.829267 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.829225    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb"
Jul 07 22:22:23.829899 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:22:23.829860    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xdjdk\" is forbidden: User \"system:node:ip-10-0-128-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-159.ec2.internal' and this object" podUID="17ab7b75-c04c-44c3-a87a-1cf82712ba22" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"
Jul 07 22:22:23.830959 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.830918    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4" event={"ID":"4aefb39e-3576-4e0d-9af0-a91f2e249893","Type":"ContainerStarted","Data":"1d0596956d3137fc36140256e12c1737b12348f66848de839a2f0732ea4f3944"}
Jul 07 22:22:23.831182 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.830960    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4" event={"ID":"4aefb39e-3576-4e0d-9af0-a91f2e249893","Type":"ContainerStarted","Data":"729af4412b2cbd4588977cb9653607c92f1038d45c6df5385ea1a3fb350cf80f"}
Jul 07 22:22:23.831182 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.831075    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4"
Jul 07 22:22:23.835701 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.835679    2555 scope.go:117] "RemoveContainer" containerID="cf3eab847a15ad23139a723e2da77e41f3a303fe8cd01595ff383a84b6d38d6e"
Jul 07 22:22:23.835960 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:22:23.835931    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf3eab847a15ad23139a723e2da77e41f3a303fe8cd01595ff383a84b6d38d6e\": container with ID starting with cf3eab847a15ad23139a723e2da77e41f3a303fe8cd01595ff383a84b6d38d6e not found: ID does not exist" containerID="cf3eab847a15ad23139a723e2da77e41f3a303fe8cd01595ff383a84b6d38d6e"
Jul 07 22:22:23.836014 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.835966    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf3eab847a15ad23139a723e2da77e41f3a303fe8cd01595ff383a84b6d38d6e"} err="failed to get container status \"cf3eab847a15ad23139a723e2da77e41f3a303fe8cd01595ff383a84b6d38d6e\": rpc error: code = NotFound desc = could not find container \"cf3eab847a15ad23139a723e2da77e41f3a303fe8cd01595ff383a84b6d38d6e\": container with ID starting with cf3eab847a15ad23139a723e2da77e41f3a303fe8cd01595ff383a84b6d38d6e not found: ID does not exist"
Jul 07 22:22:23.855433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.855391    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-skpm9" podStartSLOduration=1.503402673 podStartE2EDuration="23.855379216s" totalImagesPullingTime="22.351976543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:22:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:22:23.853719495 +0000 UTC m=+441.525938484" watchObservedRunningTime="2026-07-07 22:22:23.855379216 +0000 UTC m=+441.527598204"
Jul 07 22:22:23.882300 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.881778    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4" podStartSLOduration=12.881761224 podStartE2EDuration="12.881761224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:22:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:22:23.881189564 +0000 UTC m=+441.553408579" watchObservedRunningTime="2026-07-07 22:22:23.881761224 +0000 UTC m=+441.553980212"
Jul 07 22:22:23.883075 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:22:23.883040    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xdjdk\" is forbidden: User \"system:node:ip-10-0-128-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-159.ec2.internal' and this object" podUID="17ab7b75-c04c-44c3-a87a-1cf82712ba22" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdjdk"
Jul 07 22:22:23.902250 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:23.902208    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb" podStartSLOduration=12.902194846 podStartE2EDuration="12.902194846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:22:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:22:23.901688709 +0000 UTC m=+441.573907699" watchObservedRunningTime="2026-07-07 22:22:23.902194846 +0000 UTC m=+441.574413833"
Jul 07 22:22:24.923246 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:24.923218    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17ab7b75-c04c-44c3-a87a-1cf82712ba22" path="/var/lib/kubelet/pods/17ab7b75-c04c-44c3-a87a-1cf82712ba22/volumes"
Jul 07 22:22:34.838651 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:34.838624    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4"
Jul 07 22:22:34.839025 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:34.838690    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb"
Jul 07 22:22:34.914281 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:34.914235    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb"]
Jul 07 22:22:34.914500 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:34.914457    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb" podUID="daa848f3-2535-4b90-9738-205902373a61" containerName="manager" containerID="cri-o://9438c98a954fedfbb93173ff69adb9703f52977dec61dbc50b1415da15261e61" gracePeriod=10
Jul 07 22:22:35.132254 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.132228    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77"]
Jul 07 22:22:35.134504 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.134482    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77"
Jul 07 22:22:35.145460 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.145441    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77"]
Jul 07 22:22:35.161114 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.161091    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb"
Jul 07 22:22:35.298436 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.298401    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9586\" (UniqueName: \"kubernetes.io/projected/daa848f3-2535-4b90-9738-205902373a61-kube-api-access-z9586\") pod \"daa848f3-2535-4b90-9738-205902373a61\" (UID: \"daa848f3-2535-4b90-9738-205902373a61\") "
Jul 07 22:22:35.298614 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.298486    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/daa848f3-2535-4b90-9738-205902373a61-extensions-socket-volume\") pod \"daa848f3-2535-4b90-9738-205902373a61\" (UID: \"daa848f3-2535-4b90-9738-205902373a61\") "
Jul 07 22:22:35.298686 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.298637    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2b5a22d-6987-4746-b33b-7b35c97d2220-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-42f77\" (UID: \"f2b5a22d-6987-4746-b33b-7b35c97d2220\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77"
Jul 07 22:22:35.298758 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.298742    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc2sv\" (UniqueName: \"kubernetes.io/projected/f2b5a22d-6987-4746-b33b-7b35c97d2220-kube-api-access-pc2sv\") pod \"kuadrant-operator-controller-manager-86878c769f-42f77\" (UID: \"f2b5a22d-6987-4746-b33b-7b35c97d2220\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77"
Jul 07 22:22:35.298893 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.298846    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daa848f3-2535-4b90-9738-205902373a61-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "daa848f3-2535-4b90-9738-205902373a61" (UID: "daa848f3-2535-4b90-9738-205902373a61"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:22:35.300439 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.300417    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daa848f3-2535-4b90-9738-205902373a61-kube-api-access-z9586" (OuterVolumeSpecName: "kube-api-access-z9586") pod "daa848f3-2535-4b90-9738-205902373a61" (UID: "daa848f3-2535-4b90-9738-205902373a61"). InnerVolumeSpecName "kube-api-access-z9586". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:22:35.399935 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.399839    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2b5a22d-6987-4746-b33b-7b35c97d2220-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-42f77\" (UID: \"f2b5a22d-6987-4746-b33b-7b35c97d2220\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77"
Jul 07 22:22:35.399935 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.399924    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc2sv\" (UniqueName: \"kubernetes.io/projected/f2b5a22d-6987-4746-b33b-7b35c97d2220-kube-api-access-pc2sv\") pod \"kuadrant-operator-controller-manager-86878c769f-42f77\" (UID: \"f2b5a22d-6987-4746-b33b-7b35c97d2220\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77"
Jul 07 22:22:35.400106 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.399976    2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/daa848f3-2535-4b90-9738-205902373a61-extensions-socket-volume\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:22:35.400106 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.399986    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9586\" (UniqueName: \"kubernetes.io/projected/daa848f3-2535-4b90-9738-205902373a61-kube-api-access-z9586\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:22:35.400212 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.400192    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2b5a22d-6987-4746-b33b-7b35c97d2220-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-42f77\" (UID: \"f2b5a22d-6987-4746-b33b-7b35c97d2220\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77"
Jul 07 22:22:35.409552 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.409525    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc2sv\" (UniqueName: \"kubernetes.io/projected/f2b5a22d-6987-4746-b33b-7b35c97d2220-kube-api-access-pc2sv\") pod \"kuadrant-operator-controller-manager-86878c769f-42f77\" (UID: \"f2b5a22d-6987-4746-b33b-7b35c97d2220\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77"
Jul 07 22:22:35.458064 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.458039    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77"
Jul 07 22:22:35.585992 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.585971    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77"]
Jul 07 22:22:35.588037 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:22:35.588007    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2b5a22d_6987_4746_b33b_7b35c97d2220.slice/crio-abbbf3f335ee7491270ea863082b643afd4df926d38086364d9512e0ef1857c7 WatchSource:0}: Error finding container abbbf3f335ee7491270ea863082b643afd4df926d38086364d9512e0ef1857c7: Status 404 returned error can't find the container with id abbbf3f335ee7491270ea863082b643afd4df926d38086364d9512e0ef1857c7
Jul 07 22:22:35.886942 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.886907    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77" event={"ID":"f2b5a22d-6987-4746-b33b-7b35c97d2220","Type":"ContainerStarted","Data":"47728590bc8b588a61c432cdefff70a50d007c382ff25e8b018713f0eafa8998"}
Jul 07 22:22:35.887373 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.886961    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77" event={"ID":"f2b5a22d-6987-4746-b33b-7b35c97d2220","Type":"ContainerStarted","Data":"abbbf3f335ee7491270ea863082b643afd4df926d38086364d9512e0ef1857c7"}
Jul 07 22:22:35.887373 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.886995    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77"
Jul 07 22:22:35.888258 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.888236    2555 generic.go:356] "Generic (PLEG): container finished" podID="daa848f3-2535-4b90-9738-205902373a61" containerID="9438c98a954fedfbb93173ff69adb9703f52977dec61dbc50b1415da15261e61" exitCode=0
Jul 07 22:22:35.888374 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.888282    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb"
Jul 07 22:22:35.888374 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.888322    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb" event={"ID":"daa848f3-2535-4b90-9738-205902373a61","Type":"ContainerDied","Data":"9438c98a954fedfbb93173ff69adb9703f52977dec61dbc50b1415da15261e61"}
Jul 07 22:22:35.888487 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.888375    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb" event={"ID":"daa848f3-2535-4b90-9738-205902373a61","Type":"ContainerDied","Data":"ba6cab25e756646fb59fdab221b5eeffdb9ed63e0f5998ac59b9e7a484865567"}
Jul 07 22:22:35.888487 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.888406    2555 scope.go:117] "RemoveContainer" containerID="9438c98a954fedfbb93173ff69adb9703f52977dec61dbc50b1415da15261e61"
Jul 07 22:22:35.897781 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.897766    2555 scope.go:117] "RemoveContainer" containerID="9438c98a954fedfbb93173ff69adb9703f52977dec61dbc50b1415da15261e61"
Jul 07 22:22:35.898080 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:22:35.898063    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9438c98a954fedfbb93173ff69adb9703f52977dec61dbc50b1415da15261e61\": container with ID starting with 9438c98a954fedfbb93173ff69adb9703f52977dec61dbc50b1415da15261e61 not found: ID does not exist" containerID="9438c98a954fedfbb93173ff69adb9703f52977dec61dbc50b1415da15261e61"
Jul 07 22:22:35.898142 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.898086    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9438c98a954fedfbb93173ff69adb9703f52977dec61dbc50b1415da15261e61"} err="failed to get container status \"9438c98a954fedfbb93173ff69adb9703f52977dec61dbc50b1415da15261e61\": rpc error: code = NotFound desc = could not find container \"9438c98a954fedfbb93173ff69adb9703f52977dec61dbc50b1415da15261e61\": container with ID starting with 9438c98a954fedfbb93173ff69adb9703f52977dec61dbc50b1415da15261e61 not found: ID does not exist"
Jul 07 22:22:35.915117 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.915076    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77" podStartSLOduration=0.915060103 podStartE2EDuration="915.060103ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:22:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:22:35.913673958 +0000 UTC m=+453.585892956" watchObservedRunningTime="2026-07-07 22:22:35.915060103 +0000 UTC m=+453.587279090"
Jul 07 22:22:35.940406 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.940387    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb"]
Jul 07 22:22:35.964847 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:35.964826    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gs5fb"]
Jul 07 22:22:36.922259 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:36.922224    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="daa848f3-2535-4b90-9738-205902373a61" path="/var/lib/kubelet/pods/daa848f3-2535-4b90-9738-205902373a61/volumes"
Jul 07 22:22:46.896241 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:46.896208    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77"
Jul 07 22:22:46.945778 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:46.945750    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4"]
Jul 07 22:22:46.945979 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:46.945951    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4" podUID="4aefb39e-3576-4e0d-9af0-a91f2e249893" containerName="manager" containerID="cri-o://1d0596956d3137fc36140256e12c1737b12348f66848de839a2f0732ea4f3944" gracePeriod=10
Jul 07 22:22:47.187156 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:47.187136    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4"
Jul 07 22:22:47.189103 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:47.189087    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4aefb39e-3576-4e0d-9af0-a91f2e249893-extensions-socket-volume\") pod \"4aefb39e-3576-4e0d-9af0-a91f2e249893\" (UID: \"4aefb39e-3576-4e0d-9af0-a91f2e249893\") "
Jul 07 22:22:47.189197 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:47.189145    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klb6l\" (UniqueName: \"kubernetes.io/projected/4aefb39e-3576-4e0d-9af0-a91f2e249893-kube-api-access-klb6l\") pod \"4aefb39e-3576-4e0d-9af0-a91f2e249893\" (UID: \"4aefb39e-3576-4e0d-9af0-a91f2e249893\") "
Jul 07 22:22:47.189462 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:47.189441    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4aefb39e-3576-4e0d-9af0-a91f2e249893-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4aefb39e-3576-4e0d-9af0-a91f2e249893" (UID: "4aefb39e-3576-4e0d-9af0-a91f2e249893"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:22:47.191054 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:47.191021    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aefb39e-3576-4e0d-9af0-a91f2e249893-kube-api-access-klb6l" (OuterVolumeSpecName: "kube-api-access-klb6l") pod "4aefb39e-3576-4e0d-9af0-a91f2e249893" (UID: "4aefb39e-3576-4e0d-9af0-a91f2e249893"). InnerVolumeSpecName "kube-api-access-klb6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:22:47.289608 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:47.289574    2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4aefb39e-3576-4e0d-9af0-a91f2e249893-extensions-socket-volume\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:22:47.289608 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:47.289601    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klb6l\" (UniqueName: \"kubernetes.io/projected/4aefb39e-3576-4e0d-9af0-a91f2e249893-kube-api-access-klb6l\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:22:47.941236 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:47.941206    2555 generic.go:356] "Generic (PLEG): container finished" podID="4aefb39e-3576-4e0d-9af0-a91f2e249893" containerID="1d0596956d3137fc36140256e12c1737b12348f66848de839a2f0732ea4f3944" exitCode=0
Jul 07 22:22:47.941648 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:47.941243    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4"
Jul 07 22:22:47.941648 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:47.941289    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4" event={"ID":"4aefb39e-3576-4e0d-9af0-a91f2e249893","Type":"ContainerDied","Data":"1d0596956d3137fc36140256e12c1737b12348f66848de839a2f0732ea4f3944"}
Jul 07 22:22:47.941648 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:47.941336    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4" event={"ID":"4aefb39e-3576-4e0d-9af0-a91f2e249893","Type":"ContainerDied","Data":"729af4412b2cbd4588977cb9653607c92f1038d45c6df5385ea1a3fb350cf80f"}
Jul 07 22:22:47.941648 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:47.941356    2555 scope.go:117] "RemoveContainer" containerID="1d0596956d3137fc36140256e12c1737b12348f66848de839a2f0732ea4f3944"
Jul 07 22:22:47.949662 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:47.949460    2555 scope.go:117] "RemoveContainer" containerID="1d0596956d3137fc36140256e12c1737b12348f66848de839a2f0732ea4f3944"
Jul 07 22:22:47.949706 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:22:47.949658    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d0596956d3137fc36140256e12c1737b12348f66848de839a2f0732ea4f3944\": container with ID starting with 1d0596956d3137fc36140256e12c1737b12348f66848de839a2f0732ea4f3944 not found: ID does not exist" containerID="1d0596956d3137fc36140256e12c1737b12348f66848de839a2f0732ea4f3944"
Jul 07 22:22:47.949706 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:47.949678    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d0596956d3137fc36140256e12c1737b12348f66848de839a2f0732ea4f3944"} err="failed to get container status \"1d0596956d3137fc36140256e12c1737b12348f66848de839a2f0732ea4f3944\": rpc error: code = NotFound desc = could not find container \"1d0596956d3137fc36140256e12c1737b12348f66848de839a2f0732ea4f3944\": container with ID starting with 1d0596956d3137fc36140256e12c1737b12348f66848de839a2f0732ea4f3944 not found: ID does not exist"
Jul 07 22:22:47.962902 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:47.962858    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4"]
Jul 07 22:22:47.972506 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:47.972483    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lkbf4"]
Jul 07 22:22:48.924562 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:22:48.924532    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4aefb39e-3576-4e0d-9af0-a91f2e249893" path="/var/lib/kubelet/pods/4aefb39e-3576-4e0d-9af0-a91f2e249893/volumes"
Jul 07 22:23:03.807004 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:03.806969    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-h5xdc"]
Jul 07 22:23:03.807439 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:03.807287    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4aefb39e-3576-4e0d-9af0-a91f2e249893" containerName="manager"
Jul 07 22:23:03.807439 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:03.807298    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4aefb39e-3576-4e0d-9af0-a91f2e249893" containerName="manager"
Jul 07 22:23:03.807439 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:03.807322    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daa848f3-2535-4b90-9738-205902373a61" containerName="manager"
Jul 07 22:23:03.807439 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:03.807327    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="daa848f3-2535-4b90-9738-205902373a61" containerName="manager"
Jul 07 22:23:03.807439 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:03.807378    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="daa848f3-2535-4b90-9738-205902373a61" containerName="manager"
Jul 07 22:23:03.807439 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:03.807386    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4aefb39e-3576-4e0d-9af0-a91f2e249893" containerName="manager"
Jul 07 22:23:03.810343 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:03.810324    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-h5xdc"
Jul 07 22:23:03.812391 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:03.812369    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zmqw2\""
Jul 07 22:23:03.817784 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:03.817548    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-h5xdc"]
Jul 07 22:23:03.924594 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:03.924559    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dshvh\" (UniqueName: \"kubernetes.io/projected/985dba83-9e7c-4183-a722-e6b4924b6090-kube-api-access-dshvh\") pod \"authorino-58cd955675-h5xdc\" (UID: \"985dba83-9e7c-4183-a722-e6b4924b6090\") " pod="kuadrant-system/authorino-58cd955675-h5xdc"
Jul 07 22:23:03.933422 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:03.933390    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-tjmf7"]
Jul 07 22:23:03.936038 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:03.936022    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tjmf7"
Jul 07 22:23:03.940671 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:03.940645    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tjmf7"]
Jul 07 22:23:04.025458 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:04.025430    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2zvd\" (UniqueName: \"kubernetes.io/projected/fc9bd4db-2407-4314-8c79-809f5c0c8751-kube-api-access-b2zvd\") pod \"authorino-75d556bd77-tjmf7\" (UID: \"fc9bd4db-2407-4314-8c79-809f5c0c8751\") " pod="kuadrant-system/authorino-75d556bd77-tjmf7"
Jul 07 22:23:04.025661 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:04.025494    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dshvh\" (UniqueName: \"kubernetes.io/projected/985dba83-9e7c-4183-a722-e6b4924b6090-kube-api-access-dshvh\") pod \"authorino-58cd955675-h5xdc\" (UID: \"985dba83-9e7c-4183-a722-e6b4924b6090\") " pod="kuadrant-system/authorino-58cd955675-h5xdc"
Jul 07 22:23:04.032833 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:04.032804    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dshvh\" (UniqueName: \"kubernetes.io/projected/985dba83-9e7c-4183-a722-e6b4924b6090-kube-api-access-dshvh\") pod \"authorino-58cd955675-h5xdc\" (UID: \"985dba83-9e7c-4183-a722-e6b4924b6090\") " pod="kuadrant-system/authorino-58cd955675-h5xdc"
Jul 07 22:23:04.120246 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:04.120166    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-h5xdc"
Jul 07 22:23:04.125934 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:04.125908    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2zvd\" (UniqueName: \"kubernetes.io/projected/fc9bd4db-2407-4314-8c79-809f5c0c8751-kube-api-access-b2zvd\") pod \"authorino-75d556bd77-tjmf7\" (UID: \"fc9bd4db-2407-4314-8c79-809f5c0c8751\") " pod="kuadrant-system/authorino-75d556bd77-tjmf7"
Jul 07 22:23:04.133812 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:04.133786    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2zvd\" (UniqueName: \"kubernetes.io/projected/fc9bd4db-2407-4314-8c79-809f5c0c8751-kube-api-access-b2zvd\") pod \"authorino-75d556bd77-tjmf7\" (UID: \"fc9bd4db-2407-4314-8c79-809f5c0c8751\") " pod="kuadrant-system/authorino-75d556bd77-tjmf7"
Jul 07 22:23:04.245407 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:04.245381    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-h5xdc"]
Jul 07 22:23:04.245475 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:04.245415    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tjmf7"
Jul 07 22:23:04.247914 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:23:04.247865    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod985dba83_9e7c_4183_a722_e6b4924b6090.slice/crio-0ad3c4f8a2e352741f1a6a75922057d301348d58cdac5711790a528d34df2111 WatchSource:0}: Error finding container 0ad3c4f8a2e352741f1a6a75922057d301348d58cdac5711790a528d34df2111: Status 404 returned error can't find the container with id 0ad3c4f8a2e352741f1a6a75922057d301348d58cdac5711790a528d34df2111
Jul 07 22:23:04.372206 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:04.372181    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tjmf7"]
Jul 07 22:23:04.374117 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:23:04.374086    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc9bd4db_2407_4314_8c79_809f5c0c8751.slice/crio-6f4a1d67f2a35fdd0e540987599d8a239021aa63eedd10165a417315670fb9b9 WatchSource:0}: Error finding container 6f4a1d67f2a35fdd0e540987599d8a239021aa63eedd10165a417315670fb9b9: Status 404 returned error can't find the container with id 6f4a1d67f2a35fdd0e540987599d8a239021aa63eedd10165a417315670fb9b9
Jul 07 22:23:05.023408 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:05.023300    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tjmf7" event={"ID":"fc9bd4db-2407-4314-8c79-809f5c0c8751","Type":"ContainerStarted","Data":"6f4a1d67f2a35fdd0e540987599d8a239021aa63eedd10165a417315670fb9b9"}
Jul 07 22:23:05.026924 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:05.026853    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-h5xdc" event={"ID":"985dba83-9e7c-4183-a722-e6b4924b6090","Type":"ContainerStarted","Data":"0ad3c4f8a2e352741f1a6a75922057d301348d58cdac5711790a528d34df2111"}
Jul 07 22:23:08.046934 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:08.046816    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tjmf7" event={"ID":"fc9bd4db-2407-4314-8c79-809f5c0c8751","Type":"ContainerStarted","Data":"6f8e88ea15596025d3f4661a6add0aa9d970bdbbfef2417dd89ef220ec83c27d"}
Jul 07 22:23:08.048350 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:08.048324    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-h5xdc" event={"ID":"985dba83-9e7c-4183-a722-e6b4924b6090","Type":"ContainerStarted","Data":"6d5aebc89e9f35bfa90b04b9b07eaaae131e260bc779a10f4aab8b0ec47bc97e"}
Jul 07 22:23:08.061096 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:08.061055    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-tjmf7" podStartSLOduration=1.640775651 podStartE2EDuration="5.061040565s" totalImagesPullingTime="3.420264914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:23:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:23:08.060185639 +0000 UTC m=+485.732404630" watchObservedRunningTime="2026-07-07 22:23:08.061040565 +0000 UTC m=+485.733259553"
Jul 07 22:23:08.074367 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:08.074327    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-h5xdc" podStartSLOduration=1.538894824 podStartE2EDuration="5.074313007s" totalImagesPullingTime="3.535418183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:23:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:23:08.072789544 +0000 UTC m=+485.745008535" watchObservedRunningTime="2026-07-07 22:23:08.074313007 +0000 UTC m=+485.746531994"
Jul 07 22:23:08.093899 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:08.093848    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-h5xdc"]
Jul 07 22:23:10.056614 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:10.056579    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-h5xdc" podUID="985dba83-9e7c-4183-a722-e6b4924b6090" containerName="authorino" containerID="cri-o://6d5aebc89e9f35bfa90b04b9b07eaaae131e260bc779a10f4aab8b0ec47bc97e" gracePeriod=30
Jul 07 22:23:10.301789 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:10.301762    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-h5xdc"
Jul 07 22:23:10.385405 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:10.385311    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dshvh\" (UniqueName: \"kubernetes.io/projected/985dba83-9e7c-4183-a722-e6b4924b6090-kube-api-access-dshvh\") pod \"985dba83-9e7c-4183-a722-e6b4924b6090\" (UID: \"985dba83-9e7c-4183-a722-e6b4924b6090\") "
Jul 07 22:23:10.387263 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:10.387240    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/985dba83-9e7c-4183-a722-e6b4924b6090-kube-api-access-dshvh" (OuterVolumeSpecName: "kube-api-access-dshvh") pod "985dba83-9e7c-4183-a722-e6b4924b6090" (UID: "985dba83-9e7c-4183-a722-e6b4924b6090"). InnerVolumeSpecName "kube-api-access-dshvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:23:10.486838 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:10.486804    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dshvh\" (UniqueName: \"kubernetes.io/projected/985dba83-9e7c-4183-a722-e6b4924b6090-kube-api-access-dshvh\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:23:11.061304 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:11.061269    2555 generic.go:356] "Generic (PLEG): container finished" podID="985dba83-9e7c-4183-a722-e6b4924b6090" containerID="6d5aebc89e9f35bfa90b04b9b07eaaae131e260bc779a10f4aab8b0ec47bc97e" exitCode=0
Jul 07 22:23:11.061683 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:11.061312    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-h5xdc"
Jul 07 22:23:11.061683 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:11.061327    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-h5xdc" event={"ID":"985dba83-9e7c-4183-a722-e6b4924b6090","Type":"ContainerDied","Data":"6d5aebc89e9f35bfa90b04b9b07eaaae131e260bc779a10f4aab8b0ec47bc97e"}
Jul 07 22:23:11.061683 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:11.061364    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-h5xdc" event={"ID":"985dba83-9e7c-4183-a722-e6b4924b6090","Type":"ContainerDied","Data":"0ad3c4f8a2e352741f1a6a75922057d301348d58cdac5711790a528d34df2111"}
Jul 07 22:23:11.061683 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:11.061385    2555 scope.go:117] "RemoveContainer" containerID="6d5aebc89e9f35bfa90b04b9b07eaaae131e260bc779a10f4aab8b0ec47bc97e"
Jul 07 22:23:11.069275 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:11.069255    2555 scope.go:117] "RemoveContainer" containerID="6d5aebc89e9f35bfa90b04b9b07eaaae131e260bc779a10f4aab8b0ec47bc97e"
Jul 07 22:23:11.069532 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:23:11.069512    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d5aebc89e9f35bfa90b04b9b07eaaae131e260bc779a10f4aab8b0ec47bc97e\": container with ID starting with 6d5aebc89e9f35bfa90b04b9b07eaaae131e260bc779a10f4aab8b0ec47bc97e not found: ID does not exist" containerID="6d5aebc89e9f35bfa90b04b9b07eaaae131e260bc779a10f4aab8b0ec47bc97e"
Jul 07 22:23:11.069590 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:11.069542    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d5aebc89e9f35bfa90b04b9b07eaaae131e260bc779a10f4aab8b0ec47bc97e"} err="failed to get container status \"6d5aebc89e9f35bfa90b04b9b07eaaae131e260bc779a10f4aab8b0ec47bc97e\": rpc error: code = NotFound desc = could not find container \"6d5aebc89e9f35bfa90b04b9b07eaaae131e260bc779a10f4aab8b0ec47bc97e\": container with ID starting with 6d5aebc89e9f35bfa90b04b9b07eaaae131e260bc779a10f4aab8b0ec47bc97e not found: ID does not exist"
Jul 07 22:23:11.076479 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:11.076456    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-h5xdc"]
Jul 07 22:23:11.081880 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:11.081850    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-h5xdc"]
Jul 07 22:23:12.922029 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:12.922000    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="985dba83-9e7c-4183-a722-e6b4924b6090" path="/var/lib/kubelet/pods/985dba83-9e7c-4183-a722-e6b4924b6090/volumes"
Jul 07 22:23:32.251067 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:32.251035    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wv62h"]
Jul 07 22:23:32.251443 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:32.251354    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="985dba83-9e7c-4183-a722-e6b4924b6090" containerName="authorino"
Jul 07 22:23:32.251443 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:32.251365    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="985dba83-9e7c-4183-a722-e6b4924b6090" containerName="authorino"
Jul 07 22:23:32.251443 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:32.251416    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="985dba83-9e7c-4183-a722-e6b4924b6090" containerName="authorino"
Jul 07 22:23:32.254400 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:32.254382    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wv62h"
Jul 07 22:23:32.272215 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:32.272186    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wv62h"]
Jul 07 22:23:32.381300 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:32.381271    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v7pz\" (UniqueName: \"kubernetes.io/projected/fe9ec755-c1b2-43ef-9d2d-94b06dbfb43f-kube-api-access-8v7pz\") pod \"authorino-5d4b67c446-wv62h\" (UID: \"fe9ec755-c1b2-43ef-9d2d-94b06dbfb43f\") " pod="kuadrant-system/authorino-5d4b67c446-wv62h"
Jul 07 22:23:32.456991 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:32.456957    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wv62h"]
Jul 07 22:23:32.457278 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:23:32.457260    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-8v7pz], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-wv62h" podUID="fe9ec755-c1b2-43ef-9d2d-94b06dbfb43f"
Jul 07 22:23:32.482682 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:32.482652    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v7pz\" (UniqueName: \"kubernetes.io/projected/fe9ec755-c1b2-43ef-9d2d-94b06dbfb43f-kube-api-access-8v7pz\") pod \"authorino-5d4b67c446-wv62h\" (UID: \"fe9ec755-c1b2-43ef-9d2d-94b06dbfb43f\") " pod="kuadrant-system/authorino-5d4b67c446-wv62h"
Jul 07 22:23:32.495447 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:32.495423    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v7pz\" (UniqueName: \"kubernetes.io/projected/fe9ec755-c1b2-43ef-9d2d-94b06dbfb43f-kube-api-access-8v7pz\") pod \"authorino-5d4b67c446-wv62h\" (UID: \"fe9ec755-c1b2-43ef-9d2d-94b06dbfb43f\") " pod="kuadrant-system/authorino-5d4b67c446-wv62h"
Jul 07 22:23:32.698864 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:32.698790    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-d49cfb9f9-cdwxf"]
Jul 07 22:23:32.701093 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:32.701076    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d49cfb9f9-cdwxf"
Jul 07 22:23:32.703728 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:32.703704    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 07 22:23:32.710604 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:32.710583    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d49cfb9f9-cdwxf"]
Jul 07 22:23:32.886265 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:32.886234    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wm2p\" (UniqueName: \"kubernetes.io/projected/71d42205-9564-4374-bc25-a6e27772c274-kube-api-access-2wm2p\") pod \"authorino-d49cfb9f9-cdwxf\" (UID: \"71d42205-9564-4374-bc25-a6e27772c274\") " pod="kuadrant-system/authorino-d49cfb9f9-cdwxf"
Jul 07 22:23:32.886444 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:32.886272    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/71d42205-9564-4374-bc25-a6e27772c274-tls-cert\") pod \"authorino-d49cfb9f9-cdwxf\" (UID: \"71d42205-9564-4374-bc25-a6e27772c274\") " pod="kuadrant-system/authorino-d49cfb9f9-cdwxf"
Jul 07 22:23:32.987009 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:32.986980    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wm2p\" (UniqueName: \"kubernetes.io/projected/71d42205-9564-4374-bc25-a6e27772c274-kube-api-access-2wm2p\") pod \"authorino-d49cfb9f9-cdwxf\" (UID: \"71d42205-9564-4374-bc25-a6e27772c274\") " pod="kuadrant-system/authorino-d49cfb9f9-cdwxf"
Jul 07 22:23:32.987167 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:32.987024    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/71d42205-9564-4374-bc25-a6e27772c274-tls-cert\") pod \"authorino-d49cfb9f9-cdwxf\" (UID: \"71d42205-9564-4374-bc25-a6e27772c274\") " pod="kuadrant-system/authorino-d49cfb9f9-cdwxf"
Jul 07 22:23:32.989454 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:32.989429    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/71d42205-9564-4374-bc25-a6e27772c274-tls-cert\") pod \"authorino-d49cfb9f9-cdwxf\" (UID: \"71d42205-9564-4374-bc25-a6e27772c274\") " pod="kuadrant-system/authorino-d49cfb9f9-cdwxf"
Jul 07 22:23:32.996150 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:32.996129    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wm2p\" (UniqueName: \"kubernetes.io/projected/71d42205-9564-4374-bc25-a6e27772c274-kube-api-access-2wm2p\") pod \"authorino-d49cfb9f9-cdwxf\" (UID: \"71d42205-9564-4374-bc25-a6e27772c274\") " pod="kuadrant-system/authorino-d49cfb9f9-cdwxf"
Jul 07 22:23:33.010933 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:33.010908    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d49cfb9f9-cdwxf"
Jul 07 22:23:33.146497 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:33.146472    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d49cfb9f9-cdwxf"]
Jul 07 22:23:33.148650 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:23:33.148623    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71d42205_9564_4374_bc25_a6e27772c274.slice/crio-6ecd221c206083117c50b44586c6140871b3dc0a2c4d6e95b34d49a2f71f0577 WatchSource:0}: Error finding container 6ecd221c206083117c50b44586c6140871b3dc0a2c4d6e95b34d49a2f71f0577: Status 404 returned error can't find the container with id 6ecd221c206083117c50b44586c6140871b3dc0a2c4d6e95b34d49a2f71f0577
Jul 07 22:23:33.150022 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:33.149999    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wv62h"
Jul 07 22:23:33.156620 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:33.156602    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wv62h"
Jul 07 22:23:33.290703 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:33.290620    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8v7pz\" (UniqueName: \"kubernetes.io/projected/fe9ec755-c1b2-43ef-9d2d-94b06dbfb43f-kube-api-access-8v7pz\") pod \"fe9ec755-c1b2-43ef-9d2d-94b06dbfb43f\" (UID: \"fe9ec755-c1b2-43ef-9d2d-94b06dbfb43f\") "
Jul 07 22:23:33.292658 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:33.292636    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe9ec755-c1b2-43ef-9d2d-94b06dbfb43f-kube-api-access-8v7pz" (OuterVolumeSpecName: "kube-api-access-8v7pz") pod "fe9ec755-c1b2-43ef-9d2d-94b06dbfb43f" (UID: "fe9ec755-c1b2-43ef-9d2d-94b06dbfb43f"). InnerVolumeSpecName "kube-api-access-8v7pz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:23:33.391957 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:33.391926    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8v7pz\" (UniqueName: \"kubernetes.io/projected/fe9ec755-c1b2-43ef-9d2d-94b06dbfb43f-kube-api-access-8v7pz\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:23:34.156741 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:34.156701    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d49cfb9f9-cdwxf" event={"ID":"71d42205-9564-4374-bc25-a6e27772c274","Type":"ContainerStarted","Data":"a86585d97e6af7199188a57f1463ff274f39393c119e2812393dfc5db4fc0803"}
Jul 07 22:23:34.156741 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:34.156746    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d49cfb9f9-cdwxf" event={"ID":"71d42205-9564-4374-bc25-a6e27772c274","Type":"ContainerStarted","Data":"6ecd221c206083117c50b44586c6140871b3dc0a2c4d6e95b34d49a2f71f0577"}
Jul 07 22:23:34.157109 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:34.156718    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wv62h"
Jul 07 22:23:34.172228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:34.172184    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-d49cfb9f9-cdwxf" podStartSLOduration=1.8914420619999999 podStartE2EDuration="2.17216754s" totalImagesPullingTime="280.725478ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:23:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:23:34.170344227 +0000 UTC m=+511.842563245" watchObservedRunningTime="2026-07-07 22:23:34.17216754 +0000 UTC m=+511.844386536"
Jul 07 22:23:34.191959 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:34.191932    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wv62h"]
Jul 07 22:23:34.198016 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:34.197993    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tjmf7"]
Jul 07 22:23:34.198207 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:34.198184    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-tjmf7" podUID="fc9bd4db-2407-4314-8c79-809f5c0c8751" containerName="authorino" containerID="cri-o://6f8e88ea15596025d3f4661a6add0aa9d970bdbbfef2417dd89ef220ec83c27d" gracePeriod=30
Jul 07 22:23:34.201028 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:34.200998    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wv62h"]
Jul 07 22:23:34.463898 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:34.463859    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tjmf7"
Jul 07 22:23:34.502232 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:34.502204    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2zvd\" (UniqueName: \"kubernetes.io/projected/fc9bd4db-2407-4314-8c79-809f5c0c8751-kube-api-access-b2zvd\") pod \"fc9bd4db-2407-4314-8c79-809f5c0c8751\" (UID: \"fc9bd4db-2407-4314-8c79-809f5c0c8751\") "
Jul 07 22:23:34.504206 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:34.504182    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc9bd4db-2407-4314-8c79-809f5c0c8751-kube-api-access-b2zvd" (OuterVolumeSpecName: "kube-api-access-b2zvd") pod "fc9bd4db-2407-4314-8c79-809f5c0c8751" (UID: "fc9bd4db-2407-4314-8c79-809f5c0c8751"). InnerVolumeSpecName "kube-api-access-b2zvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:23:34.603184 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:34.603155    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2zvd\" (UniqueName: \"kubernetes.io/projected/fc9bd4db-2407-4314-8c79-809f5c0c8751-kube-api-access-b2zvd\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:23:34.922605 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:34.922573    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe9ec755-c1b2-43ef-9d2d-94b06dbfb43f" path="/var/lib/kubelet/pods/fe9ec755-c1b2-43ef-9d2d-94b06dbfb43f/volumes"
Jul 07 22:23:35.162389 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:35.162352    2555 generic.go:356] "Generic (PLEG): container finished" podID="fc9bd4db-2407-4314-8c79-809f5c0c8751" containerID="6f8e88ea15596025d3f4661a6add0aa9d970bdbbfef2417dd89ef220ec83c27d" exitCode=0
Jul 07 22:23:35.162389 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:35.162393    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tjmf7"
Jul 07 22:23:35.162612 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:35.162434    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tjmf7" event={"ID":"fc9bd4db-2407-4314-8c79-809f5c0c8751","Type":"ContainerDied","Data":"6f8e88ea15596025d3f4661a6add0aa9d970bdbbfef2417dd89ef220ec83c27d"}
Jul 07 22:23:35.162612 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:35.162477    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tjmf7" event={"ID":"fc9bd4db-2407-4314-8c79-809f5c0c8751","Type":"ContainerDied","Data":"6f4a1d67f2a35fdd0e540987599d8a239021aa63eedd10165a417315670fb9b9"}
Jul 07 22:23:35.162612 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:35.162495    2555 scope.go:117] "RemoveContainer" containerID="6f8e88ea15596025d3f4661a6add0aa9d970bdbbfef2417dd89ef220ec83c27d"
Jul 07 22:23:35.170371 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:35.170351    2555 scope.go:117] "RemoveContainer" containerID="6f8e88ea15596025d3f4661a6add0aa9d970bdbbfef2417dd89ef220ec83c27d"
Jul 07 22:23:35.170713 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:23:35.170693    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f8e88ea15596025d3f4661a6add0aa9d970bdbbfef2417dd89ef220ec83c27d\": container with ID starting with 6f8e88ea15596025d3f4661a6add0aa9d970bdbbfef2417dd89ef220ec83c27d not found: ID does not exist" containerID="6f8e88ea15596025d3f4661a6add0aa9d970bdbbfef2417dd89ef220ec83c27d"
Jul 07 22:23:35.170767 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:35.170722    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f8e88ea15596025d3f4661a6add0aa9d970bdbbfef2417dd89ef220ec83c27d"} err="failed to get container status \"6f8e88ea15596025d3f4661a6add0aa9d970bdbbfef2417dd89ef220ec83c27d\": rpc error: code = NotFound desc = could not find container \"6f8e88ea15596025d3f4661a6add0aa9d970bdbbfef2417dd89ef220ec83c27d\": container with ID starting with 6f8e88ea15596025d3f4661a6add0aa9d970bdbbfef2417dd89ef220ec83c27d not found: ID does not exist"
Jul 07 22:23:35.180496 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:35.180451    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tjmf7"]
Jul 07 22:23:35.184041 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:35.184022    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tjmf7"]
Jul 07 22:23:36.921728 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:23:36.921700    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc9bd4db-2407-4314-8c79-809f5c0c8751" path="/var/lib/kubelet/pods/fc9bd4db-2407-4314-8c79-809f5c0c8751/volumes"
Jul 07 22:24:56.076169 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.076132    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"]
Jul 07 22:24:56.076616 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.076604    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc9bd4db-2407-4314-8c79-809f5c0c8751" containerName="authorino"
Jul 07 22:24:56.076662 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.076618    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc9bd4db-2407-4314-8c79-809f5c0c8751" containerName="authorino"
Jul 07 22:24:56.076710 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.076698    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc9bd4db-2407-4314-8c79-809f5c0c8751" containerName="authorino"
Jul 07 22:24:56.079953 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.079931    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:24:56.083449 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.083425    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 07 22:24:56.083557 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.083455    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 07 22:24:56.083557 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.083457    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-nwrnv\""
Jul 07 22:24:56.083557 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.083425    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 07 22:24:56.086700 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.086680    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"]
Jul 07 22:24:56.205298 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.205268    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzfxm\" (UniqueName: \"kubernetes.io/projected/0df0bb34-ea82-4f69-bf02-11d07c910f4d-kube-api-access-pzfxm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n7m74\" (UID: \"0df0bb34-ea82-4f69-bf02-11d07c910f4d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:24:56.205462 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.205323    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0df0bb34-ea82-4f69-bf02-11d07c910f4d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n7m74\" (UID: \"0df0bb34-ea82-4f69-bf02-11d07c910f4d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:24:56.205462 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.205368    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0df0bb34-ea82-4f69-bf02-11d07c910f4d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n7m74\" (UID: \"0df0bb34-ea82-4f69-bf02-11d07c910f4d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:24:56.205462 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.205388    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0df0bb34-ea82-4f69-bf02-11d07c910f4d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n7m74\" (UID: \"0df0bb34-ea82-4f69-bf02-11d07c910f4d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:24:56.205462 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.205407    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0df0bb34-ea82-4f69-bf02-11d07c910f4d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n7m74\" (UID: \"0df0bb34-ea82-4f69-bf02-11d07c910f4d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:24:56.205462 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.205448    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0df0bb34-ea82-4f69-bf02-11d07c910f4d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n7m74\" (UID: \"0df0bb34-ea82-4f69-bf02-11d07c910f4d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:24:56.306222 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.306188    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0df0bb34-ea82-4f69-bf02-11d07c910f4d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n7m74\" (UID: \"0df0bb34-ea82-4f69-bf02-11d07c910f4d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:24:56.306222 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.306230    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0df0bb34-ea82-4f69-bf02-11d07c910f4d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n7m74\" (UID: \"0df0bb34-ea82-4f69-bf02-11d07c910f4d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:24:56.306477 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.306251    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0df0bb34-ea82-4f69-bf02-11d07c910f4d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n7m74\" (UID: \"0df0bb34-ea82-4f69-bf02-11d07c910f4d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:24:56.306477 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.306271    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0df0bb34-ea82-4f69-bf02-11d07c910f4d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n7m74\" (UID: \"0df0bb34-ea82-4f69-bf02-11d07c910f4d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:24:56.306477 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.306286    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0df0bb34-ea82-4f69-bf02-11d07c910f4d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n7m74\" (UID: \"0df0bb34-ea82-4f69-bf02-11d07c910f4d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:24:56.306477 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.306331    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzfxm\" (UniqueName: \"kubernetes.io/projected/0df0bb34-ea82-4f69-bf02-11d07c910f4d-kube-api-access-pzfxm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n7m74\" (UID: \"0df0bb34-ea82-4f69-bf02-11d07c910f4d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:24:56.306684 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.306595    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0df0bb34-ea82-4f69-bf02-11d07c910f4d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n7m74\" (UID: \"0df0bb34-ea82-4f69-bf02-11d07c910f4d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:24:56.306737 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.306697    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0df0bb34-ea82-4f69-bf02-11d07c910f4d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n7m74\" (UID: \"0df0bb34-ea82-4f69-bf02-11d07c910f4d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:24:56.306774 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.306748    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0df0bb34-ea82-4f69-bf02-11d07c910f4d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n7m74\" (UID: \"0df0bb34-ea82-4f69-bf02-11d07c910f4d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:24:56.308579 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.308552    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0df0bb34-ea82-4f69-bf02-11d07c910f4d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n7m74\" (UID: \"0df0bb34-ea82-4f69-bf02-11d07c910f4d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:24:56.308757 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.308738    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0df0bb34-ea82-4f69-bf02-11d07c910f4d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n7m74\" (UID: \"0df0bb34-ea82-4f69-bf02-11d07c910f4d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:24:56.313908 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.313863    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzfxm\" (UniqueName: \"kubernetes.io/projected/0df0bb34-ea82-4f69-bf02-11d07c910f4d-kube-api-access-pzfxm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n7m74\" (UID: \"0df0bb34-ea82-4f69-bf02-11d07c910f4d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:24:56.391574 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.391507    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:24:56.515669 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:56.515641    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"]
Jul 07 22:24:56.518260 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:24:56.518228    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0df0bb34_ea82_4f69_bf02_11d07c910f4d.slice/crio-d9502c960eb36c9d68c121bc5977b27b3556a05fd3b1df436f3a9eba2b7c9146 WatchSource:0}: Error finding container d9502c960eb36c9d68c121bc5977b27b3556a05fd3b1df436f3a9eba2b7c9146: Status 404 returned error can't find the container with id d9502c960eb36c9d68c121bc5977b27b3556a05fd3b1df436f3a9eba2b7c9146
Jul 07 22:24:57.514910 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:57.514856    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74" event={"ID":"0df0bb34-ea82-4f69-bf02-11d07c910f4d","Type":"ContainerStarted","Data":"d9502c960eb36c9d68c121bc5977b27b3556a05fd3b1df436f3a9eba2b7c9146"}
Jul 07 22:24:58.520018 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:58.519985    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74" event={"ID":"0df0bb34-ea82-4f69-bf02-11d07c910f4d","Type":"ContainerStarted","Data":"7b1f7547594c587a7ba4ef5f5d18c56043219bc1036078e6d793aedb5547f7f1"}
Jul 07 22:24:58.536166 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:24:58.536114    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74" podStartSLOduration=1.408115071 podStartE2EDuration="2.536097063s" totalImagesPullingTime="1.127981992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:24:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:24:58.535053985 +0000 UTC m=+596.207272991" watchObservedRunningTime="2026-07-07 22:24:58.536097063 +0000 UTC m=+596.208316050"
Jul 07 22:25:02.794205 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:02.794180    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:25:02.794710 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:02.794252    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:25:06.392392 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:06.392356    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:25:06.392782 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:06.392436    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:25:06.404786 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:06.404762    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:25:06.564099 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:06.564070    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n7m74"
Jul 07 22:25:32.476968 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.476939    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"]
Jul 07 22:25:32.480222 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.480207    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:32.482692 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.482669    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 07 22:25:32.488437 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.488415    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"]
Jul 07 22:25:32.636704 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.636675    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/889bf831-e958-4021-a98a-6c3c0375924a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8rljm\" (UID: \"889bf831-e958-4021-a98a-6c3c0375924a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:32.636704 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.636709    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/889bf831-e958-4021-a98a-6c3c0375924a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8rljm\" (UID: \"889bf831-e958-4021-a98a-6c3c0375924a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:32.636932 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.636788    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4rns\" (UniqueName: \"kubernetes.io/projected/889bf831-e958-4021-a98a-6c3c0375924a-kube-api-access-g4rns\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8rljm\" (UID: \"889bf831-e958-4021-a98a-6c3c0375924a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:32.636932 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.636842    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/889bf831-e958-4021-a98a-6c3c0375924a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8rljm\" (UID: \"889bf831-e958-4021-a98a-6c3c0375924a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:32.636932 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.636891    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/889bf831-e958-4021-a98a-6c3c0375924a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8rljm\" (UID: \"889bf831-e958-4021-a98a-6c3c0375924a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:32.637035 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.636941    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/889bf831-e958-4021-a98a-6c3c0375924a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8rljm\" (UID: \"889bf831-e958-4021-a98a-6c3c0375924a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:32.737848 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.737764    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/889bf831-e958-4021-a98a-6c3c0375924a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8rljm\" (UID: \"889bf831-e958-4021-a98a-6c3c0375924a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:32.737848 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.737804    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/889bf831-e958-4021-a98a-6c3c0375924a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8rljm\" (UID: \"889bf831-e958-4021-a98a-6c3c0375924a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:32.737848 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.737827    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/889bf831-e958-4021-a98a-6c3c0375924a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8rljm\" (UID: \"889bf831-e958-4021-a98a-6c3c0375924a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:32.738140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.737900    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4rns\" (UniqueName: \"kubernetes.io/projected/889bf831-e958-4021-a98a-6c3c0375924a-kube-api-access-g4rns\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8rljm\" (UID: \"889bf831-e958-4021-a98a-6c3c0375924a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:32.738140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.737950    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/889bf831-e958-4021-a98a-6c3c0375924a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8rljm\" (UID: \"889bf831-e958-4021-a98a-6c3c0375924a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:32.738140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.737978    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/889bf831-e958-4021-a98a-6c3c0375924a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8rljm\" (UID: \"889bf831-e958-4021-a98a-6c3c0375924a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:32.738278 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.738246    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/889bf831-e958-4021-a98a-6c3c0375924a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8rljm\" (UID: \"889bf831-e958-4021-a98a-6c3c0375924a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:32.738326 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.738274    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/889bf831-e958-4021-a98a-6c3c0375924a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8rljm\" (UID: \"889bf831-e958-4021-a98a-6c3c0375924a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:32.738326 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.738301    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/889bf831-e958-4021-a98a-6c3c0375924a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8rljm\" (UID: \"889bf831-e958-4021-a98a-6c3c0375924a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:32.740249 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.740222    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/889bf831-e958-4021-a98a-6c3c0375924a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8rljm\" (UID: \"889bf831-e958-4021-a98a-6c3c0375924a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:32.740505 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.740488    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/889bf831-e958-4021-a98a-6c3c0375924a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8rljm\" (UID: \"889bf831-e958-4021-a98a-6c3c0375924a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:32.744822 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.744804    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4rns\" (UniqueName: \"kubernetes.io/projected/889bf831-e958-4021-a98a-6c3c0375924a-kube-api-access-g4rns\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8rljm\" (UID: \"889bf831-e958-4021-a98a-6c3c0375924a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:32.790498 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.790472    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:32.914433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.914406    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"]
Jul 07 22:25:32.916991 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:25:32.916963    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod889bf831_e958_4021_a98a_6c3c0375924a.slice/crio-099695a94f6335340acc7c717d3acc141a9463cdd20972f456f6e218ad5661d3 WatchSource:0}: Error finding container 099695a94f6335340acc7c717d3acc141a9463cdd20972f456f6e218ad5661d3: Status 404 returned error can't find the container with id 099695a94f6335340acc7c717d3acc141a9463cdd20972f456f6e218ad5661d3
Jul 07 22:25:32.918508 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:32.918491    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 22:25:33.677538 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:33.677507    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm" event={"ID":"889bf831-e958-4021-a98a-6c3c0375924a","Type":"ContainerStarted","Data":"440da51aebd64b1418e051c9f35bf880e14b1a8030e3b30cb335e41e499bceed"}
Jul 07 22:25:33.677966 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:33.677597    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm" event={"ID":"889bf831-e958-4021-a98a-6c3c0375924a","Type":"ContainerStarted","Data":"099695a94f6335340acc7c717d3acc141a9463cdd20972f456f6e218ad5661d3"}
Jul 07 22:25:33.693904 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:33.693837    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm" podStartSLOduration=1.5189379939999998 podStartE2EDuration="1.693823844s" totalImagesPullingTime="174.88585ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:25:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:25:33.693046098 +0000 UTC m=+631.365265087" watchObservedRunningTime="2026-07-07 22:25:33.693823844 +0000 UTC m=+631.366042832"
Jul 07 22:25:38.869890 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:38.869837    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"]
Jul 07 22:25:38.873376 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:38.873356    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:38.875818 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:38.875795    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 07 22:25:38.882764 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:38.882733    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"]
Jul 07 22:25:39.000614 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.000572    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7192764-ed13-4253-8ec6-ea3c845bb27a-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl\" (UID: \"f7192764-ed13-4253-8ec6-ea3c845bb27a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:39.000614 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.000617    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7192764-ed13-4253-8ec6-ea3c845bb27a-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl\" (UID: \"f7192764-ed13-4253-8ec6-ea3c845bb27a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:39.000863 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.000656    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7192764-ed13-4253-8ec6-ea3c845bb27a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl\" (UID: \"f7192764-ed13-4253-8ec6-ea3c845bb27a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:39.000863 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.000680    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7192764-ed13-4253-8ec6-ea3c845bb27a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl\" (UID: \"f7192764-ed13-4253-8ec6-ea3c845bb27a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:39.000863 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.000754    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7192764-ed13-4253-8ec6-ea3c845bb27a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl\" (UID: \"f7192764-ed13-4253-8ec6-ea3c845bb27a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:39.000863 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.000822    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdhfr\" (UniqueName: \"kubernetes.io/projected/f7192764-ed13-4253-8ec6-ea3c845bb27a-kube-api-access-fdhfr\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl\" (UID: \"f7192764-ed13-4253-8ec6-ea3c845bb27a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:39.101768 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.101734    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7192764-ed13-4253-8ec6-ea3c845bb27a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl\" (UID: \"f7192764-ed13-4253-8ec6-ea3c845bb27a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:39.101768 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.101768    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7192764-ed13-4253-8ec6-ea3c845bb27a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl\" (UID: \"f7192764-ed13-4253-8ec6-ea3c845bb27a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:39.102043 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.101803    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7192764-ed13-4253-8ec6-ea3c845bb27a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl\" (UID: \"f7192764-ed13-4253-8ec6-ea3c845bb27a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:39.102043 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.101844    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdhfr\" (UniqueName: \"kubernetes.io/projected/f7192764-ed13-4253-8ec6-ea3c845bb27a-kube-api-access-fdhfr\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl\" (UID: \"f7192764-ed13-4253-8ec6-ea3c845bb27a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:39.102043 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.101897    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7192764-ed13-4253-8ec6-ea3c845bb27a-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl\" (UID: \"f7192764-ed13-4253-8ec6-ea3c845bb27a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:39.102043 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.102032    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7192764-ed13-4253-8ec6-ea3c845bb27a-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl\" (UID: \"f7192764-ed13-4253-8ec6-ea3c845bb27a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:39.102294 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.102272    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7192764-ed13-4253-8ec6-ea3c845bb27a-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl\" (UID: \"f7192764-ed13-4253-8ec6-ea3c845bb27a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:39.102348 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.102287    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7192764-ed13-4253-8ec6-ea3c845bb27a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl\" (UID: \"f7192764-ed13-4253-8ec6-ea3c845bb27a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:39.102384 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.102350    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7192764-ed13-4253-8ec6-ea3c845bb27a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl\" (UID: \"f7192764-ed13-4253-8ec6-ea3c845bb27a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:39.104104 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.104086    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7192764-ed13-4253-8ec6-ea3c845bb27a-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl\" (UID: \"f7192764-ed13-4253-8ec6-ea3c845bb27a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:39.104273 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.104256    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7192764-ed13-4253-8ec6-ea3c845bb27a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl\" (UID: \"f7192764-ed13-4253-8ec6-ea3c845bb27a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:39.110171 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.110148    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdhfr\" (UniqueName: \"kubernetes.io/projected/f7192764-ed13-4253-8ec6-ea3c845bb27a-kube-api-access-fdhfr\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl\" (UID: \"f7192764-ed13-4253-8ec6-ea3c845bb27a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:39.184900 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.184798    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:39.312791 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.312735    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"]
Jul 07 22:25:39.315196 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:25:39.315171    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7192764_ed13_4253_8ec6_ea3c845bb27a.slice/crio-1873b6a9b9c51e6fd35ee6b7d2e36f4d89ab9ac6778411644c475ff8443edb7c WatchSource:0}: Error finding container 1873b6a9b9c51e6fd35ee6b7d2e36f4d89ab9ac6778411644c475ff8443edb7c: Status 404 returned error can't find the container with id 1873b6a9b9c51e6fd35ee6b7d2e36f4d89ab9ac6778411644c475ff8443edb7c
Jul 07 22:25:39.702955 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.702916    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl" event={"ID":"f7192764-ed13-4253-8ec6-ea3c845bb27a","Type":"ContainerStarted","Data":"db48a326c7683a0ab566af77a5e4850f9daf14df2cab9a6c1022f727c3bb618a"}
Jul 07 22:25:39.702955 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.702961    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl" event={"ID":"f7192764-ed13-4253-8ec6-ea3c845bb27a","Type":"ContainerStarted","Data":"1873b6a9b9c51e6fd35ee6b7d2e36f4d89ab9ac6778411644c475ff8443edb7c"}
Jul 07 22:25:39.719128 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:39.719082    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl" podStartSLOduration=1.5576266909999998 podStartE2EDuration="1.719066592s" totalImagesPullingTime="161.439901ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:25:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:25:39.717789392 +0000 UTC m=+637.390008378" watchObservedRunningTime="2026-07-07 22:25:39.719066592 +0000 UTC m=+637.391285580"
Jul 07 22:25:42.791575 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:42.791542    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:42.791992 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:42.791605    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:42.804100 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:42.804077    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:43.733545 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:43.733515    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8rljm"
Jul 07 22:25:49.185860 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:49.185818    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:49.186267 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:49.185896    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:49.198360 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:49.198333    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:25:49.759028 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:25:49.759001    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl"
Jul 07 22:26:52.728729 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:52.728694    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-846f7dc586-zbgh4"]
Jul 07 22:26:52.732092 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:52.732076    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-846f7dc586-zbgh4"
Jul 07 22:26:52.738121 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:52.738097    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-846f7dc586-zbgh4"]
Jul 07 22:26:52.870142 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:52.870110    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c099867-be01-4483-b206-670e283e6880-tls-cert\") pod \"authorino-846f7dc586-zbgh4\" (UID: \"1c099867-be01-4483-b206-670e283e6880\") " pod="kuadrant-system/authorino-846f7dc586-zbgh4"
Jul 07 22:26:52.870142 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:52.870149    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwkhq\" (UniqueName: \"kubernetes.io/projected/1c099867-be01-4483-b206-670e283e6880-kube-api-access-rwkhq\") pod \"authorino-846f7dc586-zbgh4\" (UID: \"1c099867-be01-4483-b206-670e283e6880\") " pod="kuadrant-system/authorino-846f7dc586-zbgh4"
Jul 07 22:26:52.970763 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:52.970708    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c099867-be01-4483-b206-670e283e6880-tls-cert\") pod \"authorino-846f7dc586-zbgh4\" (UID: \"1c099867-be01-4483-b206-670e283e6880\") " pod="kuadrant-system/authorino-846f7dc586-zbgh4"
Jul 07 22:26:52.970763 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:52.970764    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwkhq\" (UniqueName: \"kubernetes.io/projected/1c099867-be01-4483-b206-670e283e6880-kube-api-access-rwkhq\") pod \"authorino-846f7dc586-zbgh4\" (UID: \"1c099867-be01-4483-b206-670e283e6880\") " pod="kuadrant-system/authorino-846f7dc586-zbgh4"
Jul 07 22:26:52.973414 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:52.973388    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c099867-be01-4483-b206-670e283e6880-tls-cert\") pod \"authorino-846f7dc586-zbgh4\" (UID: \"1c099867-be01-4483-b206-670e283e6880\") " pod="kuadrant-system/authorino-846f7dc586-zbgh4"
Jul 07 22:26:52.978304 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:52.978280    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwkhq\" (UniqueName: \"kubernetes.io/projected/1c099867-be01-4483-b206-670e283e6880-kube-api-access-rwkhq\") pod \"authorino-846f7dc586-zbgh4\" (UID: \"1c099867-be01-4483-b206-670e283e6880\") " pod="kuadrant-system/authorino-846f7dc586-zbgh4"
Jul 07 22:26:53.042567 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:53.042482    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-846f7dc586-zbgh4"
Jul 07 22:26:53.169895 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:53.169857    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-846f7dc586-zbgh4"]
Jul 07 22:26:53.172044 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:26:53.172009    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c099867_be01_4483_b206_670e283e6880.slice/crio-92fb1cd3742a7bedb45788ebf33d0369a179d6ad2523a136c58fd812debb43db WatchSource:0}: Error finding container 92fb1cd3742a7bedb45788ebf33d0369a179d6ad2523a136c58fd812debb43db: Status 404 returned error can't find the container with id 92fb1cd3742a7bedb45788ebf33d0369a179d6ad2523a136c58fd812debb43db
Jul 07 22:26:54.025278 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:54.025245    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-846f7dc586-zbgh4" event={"ID":"1c099867-be01-4483-b206-670e283e6880","Type":"ContainerStarted","Data":"b67737cdc0b9118732903426a2e8d2264925a13a9c87288124ca424a7f9e63f9"}
Jul 07 22:26:54.025663 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:54.025287    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-846f7dc586-zbgh4" event={"ID":"1c099867-be01-4483-b206-670e283e6880","Type":"ContainerStarted","Data":"92fb1cd3742a7bedb45788ebf33d0369a179d6ad2523a136c58fd812debb43db"}
Jul 07 22:26:54.043680 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:54.043633    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-846f7dc586-zbgh4" podStartSLOduration=1.633594757 podStartE2EDuration="2.043616733s" totalImagesPullingTime="410.021976ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:26:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:26:54.042263736 +0000 UTC m=+711.714482724" watchObservedRunningTime="2026-07-07 22:26:54.043616733 +0000 UTC m=+711.715835721"
Jul 07 22:26:54.070581 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:54.070542    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d49cfb9f9-cdwxf"]
Jul 07 22:26:54.070829 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:54.070796    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-d49cfb9f9-cdwxf" podUID="71d42205-9564-4374-bc25-a6e27772c274" containerName="authorino" containerID="cri-o://a86585d97e6af7199188a57f1463ff274f39393c119e2812393dfc5db4fc0803" gracePeriod=30
Jul 07 22:26:54.343102 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:54.343070    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d49cfb9f9-cdwxf"
Jul 07 22:26:54.485405 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:54.485374    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wm2p\" (UniqueName: \"kubernetes.io/projected/71d42205-9564-4374-bc25-a6e27772c274-kube-api-access-2wm2p\") pod \"71d42205-9564-4374-bc25-a6e27772c274\" (UID: \"71d42205-9564-4374-bc25-a6e27772c274\") "
Jul 07 22:26:54.485580 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:54.485422    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/71d42205-9564-4374-bc25-a6e27772c274-tls-cert\") pod \"71d42205-9564-4374-bc25-a6e27772c274\" (UID: \"71d42205-9564-4374-bc25-a6e27772c274\") "
Jul 07 22:26:54.487461 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:54.487430    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71d42205-9564-4374-bc25-a6e27772c274-kube-api-access-2wm2p" (OuterVolumeSpecName: "kube-api-access-2wm2p") pod "71d42205-9564-4374-bc25-a6e27772c274" (UID: "71d42205-9564-4374-bc25-a6e27772c274"). InnerVolumeSpecName "kube-api-access-2wm2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:26:54.496003 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:54.495977    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71d42205-9564-4374-bc25-a6e27772c274-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "71d42205-9564-4374-bc25-a6e27772c274" (UID: "71d42205-9564-4374-bc25-a6e27772c274"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 22:26:54.586699 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:54.586610    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wm2p\" (UniqueName: \"kubernetes.io/projected/71d42205-9564-4374-bc25-a6e27772c274-kube-api-access-2wm2p\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:26:54.586699 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:54.586644    2555 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/71d42205-9564-4374-bc25-a6e27772c274-tls-cert\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:26:55.031195 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:55.031153    2555 generic.go:356] "Generic (PLEG): container finished" podID="71d42205-9564-4374-bc25-a6e27772c274" containerID="a86585d97e6af7199188a57f1463ff274f39393c119e2812393dfc5db4fc0803" exitCode=0
Jul 07 22:26:55.031195 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:55.031191    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d49cfb9f9-cdwxf"
Jul 07 22:26:55.031673 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:55.031231    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d49cfb9f9-cdwxf" event={"ID":"71d42205-9564-4374-bc25-a6e27772c274","Type":"ContainerDied","Data":"a86585d97e6af7199188a57f1463ff274f39393c119e2812393dfc5db4fc0803"}
Jul 07 22:26:55.031673 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:55.031271    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d49cfb9f9-cdwxf" event={"ID":"71d42205-9564-4374-bc25-a6e27772c274","Type":"ContainerDied","Data":"6ecd221c206083117c50b44586c6140871b3dc0a2c4d6e95b34d49a2f71f0577"}
Jul 07 22:26:55.031673 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:55.031288    2555 scope.go:117] "RemoveContainer" containerID="a86585d97e6af7199188a57f1463ff274f39393c119e2812393dfc5db4fc0803"
Jul 07 22:26:55.039515 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:55.039492    2555 scope.go:117] "RemoveContainer" containerID="a86585d97e6af7199188a57f1463ff274f39393c119e2812393dfc5db4fc0803"
Jul 07 22:26:55.039808 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:26:55.039788    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a86585d97e6af7199188a57f1463ff274f39393c119e2812393dfc5db4fc0803\": container with ID starting with a86585d97e6af7199188a57f1463ff274f39393c119e2812393dfc5db4fc0803 not found: ID does not exist" containerID="a86585d97e6af7199188a57f1463ff274f39393c119e2812393dfc5db4fc0803"
Jul 07 22:26:55.039863 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:55.039817    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a86585d97e6af7199188a57f1463ff274f39393c119e2812393dfc5db4fc0803"} err="failed to get container status \"a86585d97e6af7199188a57f1463ff274f39393c119e2812393dfc5db4fc0803\": rpc error: code = NotFound desc = could not find container \"a86585d97e6af7199188a57f1463ff274f39393c119e2812393dfc5db4fc0803\": container with ID starting with a86585d97e6af7199188a57f1463ff274f39393c119e2812393dfc5db4fc0803 not found: ID does not exist"
Jul 07 22:26:55.046408 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:55.046387    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d49cfb9f9-cdwxf"]
Jul 07 22:26:55.050627 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:55.050605    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-d49cfb9f9-cdwxf"]
Jul 07 22:26:56.922268 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:26:56.922234    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71d42205-9564-4374-bc25-a6e27772c274" path="/var/lib/kubelet/pods/71d42205-9564-4374-bc25-a6e27772c274/volumes"
Jul 07 22:30:02.826754 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:30:02.826678    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:30:02.827911 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:30:02.827891    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:32:43.589740 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:32:43.589700    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77"]
Jul 07 22:32:43.590318 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:32:43.589941    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77" podUID="f2b5a22d-6987-4746-b33b-7b35c97d2220" containerName="manager" containerID="cri-o://47728590bc8b588a61c432cdefff70a50d007c382ff25e8b018713f0eafa8998" gracePeriod=10
Jul 07 22:32:46.129059 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:32:46.129033    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77"
Jul 07 22:32:46.203789 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:32:46.203753    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2b5a22d-6987-4746-b33b-7b35c97d2220-extensions-socket-volume\") pod \"f2b5a22d-6987-4746-b33b-7b35c97d2220\" (UID: \"f2b5a22d-6987-4746-b33b-7b35c97d2220\") "
Jul 07 22:32:46.203957 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:32:46.203812    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pc2sv\" (UniqueName: \"kubernetes.io/projected/f2b5a22d-6987-4746-b33b-7b35c97d2220-kube-api-access-pc2sv\") pod \"f2b5a22d-6987-4746-b33b-7b35c97d2220\" (UID: \"f2b5a22d-6987-4746-b33b-7b35c97d2220\") "
Jul 07 22:32:46.204129 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:32:46.204105    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2b5a22d-6987-4746-b33b-7b35c97d2220-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f2b5a22d-6987-4746-b33b-7b35c97d2220" (UID: "f2b5a22d-6987-4746-b33b-7b35c97d2220"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:32:46.205761 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:32:46.205741    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2b5a22d-6987-4746-b33b-7b35c97d2220-kube-api-access-pc2sv" (OuterVolumeSpecName: "kube-api-access-pc2sv") pod "f2b5a22d-6987-4746-b33b-7b35c97d2220" (UID: "f2b5a22d-6987-4746-b33b-7b35c97d2220"). InnerVolumeSpecName "kube-api-access-pc2sv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:32:46.304632 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:32:46.304597    2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2b5a22d-6987-4746-b33b-7b35c97d2220-extensions-socket-volume\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:32:46.304632 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:32:46.304625    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pc2sv\" (UniqueName: \"kubernetes.io/projected/f2b5a22d-6987-4746-b33b-7b35c97d2220-kube-api-access-pc2sv\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:32:46.573111 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:32:46.573019    2555 generic.go:356] "Generic (PLEG): container finished" podID="f2b5a22d-6987-4746-b33b-7b35c97d2220" containerID="47728590bc8b588a61c432cdefff70a50d007c382ff25e8b018713f0eafa8998" exitCode=0
Jul 07 22:32:46.573111 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:32:46.573064    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77"
Jul 07 22:32:46.573111 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:32:46.573059    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77" event={"ID":"f2b5a22d-6987-4746-b33b-7b35c97d2220","Type":"ContainerDied","Data":"47728590bc8b588a61c432cdefff70a50d007c382ff25e8b018713f0eafa8998"}
Jul 07 22:32:46.573368 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:32:46.573129    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77" event={"ID":"f2b5a22d-6987-4746-b33b-7b35c97d2220","Type":"ContainerDied","Data":"abbbf3f335ee7491270ea863082b643afd4df926d38086364d9512e0ef1857c7"}
Jul 07 22:32:46.573368 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:32:46.573159    2555 scope.go:117] "RemoveContainer" containerID="47728590bc8b588a61c432cdefff70a50d007c382ff25e8b018713f0eafa8998"
Jul 07 22:32:46.582612 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:32:46.582595    2555 scope.go:117] "RemoveContainer" containerID="47728590bc8b588a61c432cdefff70a50d007c382ff25e8b018713f0eafa8998"
Jul 07 22:32:46.582828 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:32:46.582811    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47728590bc8b588a61c432cdefff70a50d007c382ff25e8b018713f0eafa8998\": container with ID starting with 47728590bc8b588a61c432cdefff70a50d007c382ff25e8b018713f0eafa8998 not found: ID does not exist" containerID="47728590bc8b588a61c432cdefff70a50d007c382ff25e8b018713f0eafa8998"
Jul 07 22:32:46.582896 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:32:46.582837    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47728590bc8b588a61c432cdefff70a50d007c382ff25e8b018713f0eafa8998"} err="failed to get container status \"47728590bc8b588a61c432cdefff70a50d007c382ff25e8b018713f0eafa8998\": rpc error: code = NotFound desc = could not find container \"47728590bc8b588a61c432cdefff70a50d007c382ff25e8b018713f0eafa8998\": container with ID starting with 47728590bc8b588a61c432cdefff70a50d007c382ff25e8b018713f0eafa8998 not found: ID does not exist"
Jul 07 22:32:46.593835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:32:46.593815    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77"]
Jul 07 22:32:46.597163 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:32:46.597143    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42f77"]
Jul 07 22:32:46.922249 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:32:46.922176    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2b5a22d-6987-4746-b33b-7b35c97d2220" path="/var/lib/kubelet/pods/f2b5a22d-6987-4746-b33b-7b35c97d2220/volumes"
Jul 07 22:35:02.852140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:35:02.852113    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:35:02.853504 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:35:02.853483    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:40:02.879009 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:02.878982    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:40:02.881316 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:02.881297    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:40:16.823607 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.823578    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"]
Jul 07 22:40:16.824129 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.824106    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71d42205-9564-4374-bc25-a6e27772c274" containerName="authorino"
Jul 07 22:40:16.824129 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.824126    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="71d42205-9564-4374-bc25-a6e27772c274" containerName="authorino"
Jul 07 22:40:16.824297 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.824160    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2b5a22d-6987-4746-b33b-7b35c97d2220" containerName="manager"
Jul 07 22:40:16.824297 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.824169    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2b5a22d-6987-4746-b33b-7b35c97d2220" containerName="manager"
Jul 07 22:40:16.824297 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.824247    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="71d42205-9564-4374-bc25-a6e27772c274" containerName="authorino"
Jul 07 22:40:16.824297 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.824261    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2b5a22d-6987-4746-b33b-7b35c97d2220" containerName="manager"
Jul 07 22:40:16.826976 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.826957    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.830464 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.830440    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 07 22:40:16.830705 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.830688    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-fe0013e5-openshift-default-dockercfg-6j656\""
Jul 07 22:40:16.840061 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.840040    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"]
Jul 07 22:40:16.888211 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.888184    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istiod-ca-cert\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.888351 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.888281    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-data\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.888419 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.888351    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-podinfo\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.888419 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.888374    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-token\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.888419 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.888411    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-envoy\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.888548 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.888456    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-workload-certs\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.888548 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.888534    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjflw\" (UniqueName: \"kubernetes.io/projected/8728778c-db1b-45ec-bbf6-cdf95b5e888b-kube-api-access-wjflw\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.888647 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.888553    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-workload-socket\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.888647 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.888581    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-credential-socket\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.989948 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.989914    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-envoy\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.989948 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.989953    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-workload-certs\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.990185 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.990010    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjflw\" (UniqueName: \"kubernetes.io/projected/8728778c-db1b-45ec-bbf6-cdf95b5e888b-kube-api-access-wjflw\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.990185 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.990036    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-workload-socket\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.990185 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.990101    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-credential-socket\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.990341 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.990211    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istiod-ca-cert\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.990341 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.990275    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-data\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.990341 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.990319    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-podinfo\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.990478 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.990345    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-token\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.990478 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.990415    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-workload-socket\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.990478 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.990459    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-workload-certs\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.990727 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.990700    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-credential-socket\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.990849 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.990759    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-data\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.991062 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.991041    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istiod-ca-cert\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.992240 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.992223    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-envoy\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.992798 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.992774    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-podinfo\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.998683 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.998663    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjflw\" (UniqueName: \"kubernetes.io/projected/8728778c-db1b-45ec-bbf6-cdf95b5e888b-kube-api-access-wjflw\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:16.998806 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:16.998790    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-token\") pod \"e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") " pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:17.140224 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:17.140129    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:17.292698 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:17.292670    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"]
Jul 07 22:40:17.293959 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:40:17.293935    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8728778c_db1b_45ec_bbf6_cdf95b5e888b.slice/crio-e352cddc4b50862b6c4446907b0d97f1093d6e7660d8fb14e47c283e028ee397 WatchSource:0}: Error finding container e352cddc4b50862b6c4446907b0d97f1093d6e7660d8fb14e47c283e028ee397: Status 404 returned error can't find the container with id e352cddc4b50862b6c4446907b0d97f1093d6e7660d8fb14e47c283e028ee397
Jul 07 22:40:17.295732 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:17.295717    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 22:40:17.517025 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:17.516993    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x" event={"ID":"8728778c-db1b-45ec-bbf6-cdf95b5e888b","Type":"ContainerStarted","Data":"e352cddc4b50862b6c4446907b0d97f1093d6e7660d8fb14e47c283e028ee397"}
Jul 07 22:40:18.009077 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:18.009043    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5"]
Jul 07 22:40:18.013388 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:18.013366    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5"
Jul 07 22:40:18.015640 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:18.015610    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-fe0013e5-serving-cert\""
Jul 07 22:40:18.015775 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:18.015684    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-r9rv4\""
Jul 07 22:40:18.022583 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:18.022559    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5"]
Jul 07 22:40:18.101706 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:18.101673    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-openshift-service-ca\") pod \"maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5\" (UID: \"1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908\") " pod="opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5"
Jul 07 22:40:18.101892 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:18.101728    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtmws\" (UniqueName: \"kubernetes.io/projected/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-kube-api-access-rtmws\") pod \"maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5\" (UID: \"1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908\") " pod="opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5"
Jul 07 22:40:18.101892 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:18.101859    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-maas-api-tls\") pod \"maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5\" (UID: \"1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908\") " pod="opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5"
Jul 07 22:40:18.202931 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:18.202898    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-maas-api-tls\") pod \"maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5\" (UID: \"1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908\") " pod="opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5"
Jul 07 22:40:18.203126 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:18.202953    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-openshift-service-ca\") pod \"maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5\" (UID: \"1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908\") " pod="opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5"
Jul 07 22:40:18.203126 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:18.202984    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtmws\" (UniqueName: \"kubernetes.io/projected/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-kube-api-access-rtmws\") pod \"maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5\" (UID: \"1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908\") " pod="opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5"
Jul 07 22:40:18.203702 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:18.203681    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-openshift-service-ca\") pod \"maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5\" (UID: \"1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908\") " pod="opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5"
Jul 07 22:40:18.205277 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:18.205255    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-maas-api-tls\") pod \"maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5\" (UID: \"1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908\") " pod="opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5"
Jul 07 22:40:18.211893 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:18.211854    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtmws\" (UniqueName: \"kubernetes.io/projected/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-kube-api-access-rtmws\") pod \"maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5\" (UID: \"1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908\") " pod="opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5"
Jul 07 22:40:18.324707 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:18.324628    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5"
Jul 07 22:40:18.451414 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:18.451357    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5"]
Jul 07 22:40:18.453948 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:40:18.453915    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eb2f0b2_c1f0_4f40_b1ce_a0a28b06d908.slice/crio-16d1d61e9f3ac077d6f587b2957dcd91d9beeac315283643f613ec91f31a6e38 WatchSource:0}: Error finding container 16d1d61e9f3ac077d6f587b2957dcd91d9beeac315283643f613ec91f31a6e38: Status 404 returned error can't find the container with id 16d1d61e9f3ac077d6f587b2957dcd91d9beeac315283643f613ec91f31a6e38
Jul 07 22:40:18.525035 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:18.525002    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5" event={"ID":"1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908","Type":"ContainerStarted","Data":"16d1d61e9f3ac077d6f587b2957dcd91d9beeac315283643f613ec91f31a6e38"}
Jul 07 22:40:21.536811 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:21.536764    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:40:21.537245 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:21.536863    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:40:21.537245 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:21.536934    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:40:22.545184 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:22.545147    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5" event={"ID":"1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908","Type":"ContainerStarted","Data":"e5ca5c560268aff9f33c77b92927a22b1981fc626ba2cef7dcc828daed3620d1"}
Jul 07 22:40:22.545625 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:22.545243    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5"
Jul 07 22:40:22.546624 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:22.546604    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x" event={"ID":"8728778c-db1b-45ec-bbf6-cdf95b5e888b","Type":"ContainerStarted","Data":"9c64a1ea7b1f2147052870460149e8aae44a7cfb07aa203d641494773e03e8ac"}
Jul 07 22:40:22.561429 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:22.561382    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5" podStartSLOduration=2.440554943 podStartE2EDuration="5.561362407s" totalImagesPullingTime="3.120807464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:40:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:40:22.560250272 +0000 UTC m=+1520.232469260" watchObservedRunningTime="2026-07-07 22:40:22.561362407 +0000 UTC m=+1520.233581379"
Jul 07 22:40:22.582490 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:22.582425    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x" podStartSLOduration=2.341794235 podStartE2EDuration="6.582401424s" totalImagesPullingTime="4.240607189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:40:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:40:22.580198366 +0000 UTC m=+1520.252417393" watchObservedRunningTime="2026-07-07 22:40:22.582401424 +0000 UTC m=+1520.254620415"
Jul 07 22:40:23.140551 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:23.140522    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:23.283197 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:23.283155    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5"]
Jul 07 22:40:23.657455 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:40:23.657418    2555 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-fe0013e5-serving-cert: secret "maas-api-e2e-ait-fe0013e5-serving-cert" not found
Jul 07 22:40:23.657850 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:40:23.657507    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-maas-api-tls podName:1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908 nodeName:}" failed. No retries permitted until 2026-07-07 22:40:24.15748491 +0000 UTC m=+1521.829703896 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-maas-api-tls") pod "maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5" (UID: "1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908") : secret "maas-api-e2e-ait-fe0013e5-serving-cert" not found
Jul 07 22:40:24.159258 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:24.159212    2555 patch_prober.go:28] interesting pod/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 07 22:40:24.159464 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:24.159323    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x" podUID="8728778c-db1b-45ec-bbf6-cdf95b5e888b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 07 22:40:24.163825 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:40:24.163777    2555 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-fe0013e5-serving-cert: secret "maas-api-e2e-ait-fe0013e5-serving-cert" not found
Jul 07 22:40:24.163974 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:40:24.163891    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-maas-api-tls podName:1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908 nodeName:}" failed. No retries permitted until 2026-07-07 22:40:25.163851369 +0000 UTC m=+1522.836070349 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-maas-api-tls") pod "maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5" (UID: "1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908") : secret "maas-api-e2e-ait-fe0013e5-serving-cert" not found
Jul 07 22:40:24.558109 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:24.558063    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5" podUID="1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908" containerName="maas-api" containerID="cri-o://e5ca5c560268aff9f33c77b92927a22b1981fc626ba2cef7dcc828daed3620d1" gracePeriod=30
Jul 07 22:40:24.826029 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:24.826004    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5"
Jul 07 22:40:24.985973 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:24.985229    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-openshift-service-ca\") pod \"1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908\" (UID: \"1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908\") "
Jul 07 22:40:24.985973 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:24.985292    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtmws\" (UniqueName: \"kubernetes.io/projected/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-kube-api-access-rtmws\") pod \"1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908\" (UID: \"1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908\") "
Jul 07 22:40:24.985973 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:24.985386    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-maas-api-tls\") pod \"1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908\" (UID: \"1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908\") "
Jul 07 22:40:24.986260 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:24.986200    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908" (UID: "1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 22:40:24.988999 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:24.988965    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908" (UID: "1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 22:40:24.989484 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:24.989457    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-kube-api-access-rtmws" (OuterVolumeSpecName: "kube-api-access-rtmws") pod "1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908" (UID: "1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908"). InnerVolumeSpecName "kube-api-access-rtmws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:40:25.086178 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:25.086095    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-maas-api-tls\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:25.086178 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:25.086124    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-openshift-service-ca\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:25.086178 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:25.086134    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtmws\" (UniqueName: \"kubernetes.io/projected/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908-kube-api-access-rtmws\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:25.160408 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:25.160367    2555 patch_prober.go:28] interesting pod/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 07 22:40:25.160572 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:25.160439    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x" podUID="8728778c-db1b-45ec-bbf6-cdf95b5e888b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 07 22:40:25.565591 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:25.565556    2555 generic.go:356] "Generic (PLEG): container finished" podID="1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908" containerID="e5ca5c560268aff9f33c77b92927a22b1981fc626ba2cef7dcc828daed3620d1" exitCode=0
Jul 07 22:40:25.565865 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:25.565601    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5"
Jul 07 22:40:25.566567 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:25.565619    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5" event={"ID":"1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908","Type":"ContainerDied","Data":"e5ca5c560268aff9f33c77b92927a22b1981fc626ba2cef7dcc828daed3620d1"}
Jul 07 22:40:25.566567 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:25.566103    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5" event={"ID":"1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908","Type":"ContainerDied","Data":"16d1d61e9f3ac077d6f587b2957dcd91d9beeac315283643f613ec91f31a6e38"}
Jul 07 22:40:25.566567 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:25.566138    2555 scope.go:117] "RemoveContainer" containerID="e5ca5c560268aff9f33c77b92927a22b1981fc626ba2cef7dcc828daed3620d1"
Jul 07 22:40:25.582248 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:25.582150    2555 scope.go:117] "RemoveContainer" containerID="e5ca5c560268aff9f33c77b92927a22b1981fc626ba2cef7dcc828daed3620d1"
Jul 07 22:40:25.582833 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:40:25.582657    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5ca5c560268aff9f33c77b92927a22b1981fc626ba2cef7dcc828daed3620d1\": container with ID starting with e5ca5c560268aff9f33c77b92927a22b1981fc626ba2cef7dcc828daed3620d1 not found: ID does not exist" containerID="e5ca5c560268aff9f33c77b92927a22b1981fc626ba2cef7dcc828daed3620d1"
Jul 07 22:40:25.582833 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:25.582692    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5ca5c560268aff9f33c77b92927a22b1981fc626ba2cef7dcc828daed3620d1"} err="failed to get container status \"e5ca5c560268aff9f33c77b92927a22b1981fc626ba2cef7dcc828daed3620d1\": rpc error: code = NotFound desc = could not find container \"e5ca5c560268aff9f33c77b92927a22b1981fc626ba2cef7dcc828daed3620d1\": container with ID starting with e5ca5c560268aff9f33c77b92927a22b1981fc626ba2cef7dcc828daed3620d1 not found: ID does not exist"
Jul 07 22:40:25.599857 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:25.599835    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5"]
Jul 07 22:40:25.603393 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:25.603373    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-fe0013e5-74b67784b4-wmwz5"]
Jul 07 22:40:26.161812 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:26.161763    2555 patch_prober.go:28] interesting pod/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 07 22:40:26.162430 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:26.162396    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x" podUID="8728778c-db1b-45ec-bbf6-cdf95b5e888b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 07 22:40:26.923330 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:26.923291    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908" path="/var/lib/kubelet/pods/1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908/volumes"
Jul 07 22:40:27.204051 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:27.204014    2555 patch_prober.go:28] interesting pod/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 07 22:40:27.204464 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:27.204094    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x" podUID="8728778c-db1b-45ec-bbf6-cdf95b5e888b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 07 22:40:28.246019 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:28.245980    2555 patch_prober.go:28] interesting pod/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 07 22:40:28.246431 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:28.246053    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x" podUID="8728778c-db1b-45ec-bbf6-cdf95b5e888b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 07 22:40:29.063169 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.063133    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"]
Jul 07 22:40:29.063345 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.063307    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x" podUID="8728778c-db1b-45ec-bbf6-cdf95b5e888b" containerName="istio-proxy" containerID="cri-o://9c64a1ea7b1f2147052870460149e8aae44a7cfb07aa203d641494773e03e8ac" gracePeriod=30
Jul 07 22:40:29.192696 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.192665    2555 patch_prober.go:28] interesting pod/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": read tcp 10.133.0.2:59440->10.133.0.32:15021: read: connection reset by peer" start-of-body=
Jul 07 22:40:29.192922 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.192730    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x" podUID="8728778c-db1b-45ec-bbf6-cdf95b5e888b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": read tcp 10.133.0.2:59440->10.133.0.32:15021: read: connection reset by peer"
Jul 07 22:40:29.193065 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.193033    2555 patch_prober.go:28] interesting pod/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Jul 07 22:40:29.193153 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.193092    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x" podUID="8728778c-db1b-45ec-bbf6-cdf95b5e888b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Jul 07 22:40:29.422660 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.422631    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:29.426260 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.426233    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-podinfo\") pod \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") "
Jul 07 22:40:29.426409 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.426275    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-data\") pod \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") "
Jul 07 22:40:29.426409 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.426303    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-token\") pod \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") "
Jul 07 22:40:29.426409 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.426327    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjflw\" (UniqueName: \"kubernetes.io/projected/8728778c-db1b-45ec-bbf6-cdf95b5e888b-kube-api-access-wjflw\") pod \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") "
Jul 07 22:40:29.426409 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.426360    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-envoy\") pod \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") "
Jul 07 22:40:29.426409 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.426382    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-workload-certs\") pod \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") "
Jul 07 22:40:29.426656 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.426469    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istiod-ca-cert\") pod \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") "
Jul 07 22:40:29.426656 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.426499    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-workload-socket\") pod \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") "
Jul 07 22:40:29.426656 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.426551    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-credential-socket\") pod \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\" (UID: \"8728778c-db1b-45ec-bbf6-cdf95b5e888b\") "
Jul 07 22:40:29.427066 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.427037    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8728778c-db1b-45ec-bbf6-cdf95b5e888b" (UID: "8728778c-db1b-45ec-bbf6-cdf95b5e888b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:40:29.427243 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.427221    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8728778c-db1b-45ec-bbf6-cdf95b5e888b" (UID: "8728778c-db1b-45ec-bbf6-cdf95b5e888b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:40:29.427317 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.427260    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-data" (OuterVolumeSpecName: "istio-data") pod "8728778c-db1b-45ec-bbf6-cdf95b5e888b" (UID: "8728778c-db1b-45ec-bbf6-cdf95b5e888b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:40:29.427470 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.427420    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8728778c-db1b-45ec-bbf6-cdf95b5e888b" (UID: "8728778c-db1b-45ec-bbf6-cdf95b5e888b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:40:29.427526 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.427473    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8728778c-db1b-45ec-bbf6-cdf95b5e888b" (UID: "8728778c-db1b-45ec-bbf6-cdf95b5e888b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 22:40:29.429188 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.429165    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8728778c-db1b-45ec-bbf6-cdf95b5e888b" (UID: "8728778c-db1b-45ec-bbf6-cdf95b5e888b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 22:40:29.429329 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.429196    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8728778c-db1b-45ec-bbf6-cdf95b5e888b-kube-api-access-wjflw" (OuterVolumeSpecName: "kube-api-access-wjflw") pod "8728778c-db1b-45ec-bbf6-cdf95b5e888b" (UID: "8728778c-db1b-45ec-bbf6-cdf95b5e888b"). InnerVolumeSpecName "kube-api-access-wjflw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:40:29.429329 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.429239    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-token" (OuterVolumeSpecName: "istio-token") pod "8728778c-db1b-45ec-bbf6-cdf95b5e888b" (UID: "8728778c-db1b-45ec-bbf6-cdf95b5e888b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:40:29.430629 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.430603    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8728778c-db1b-45ec-bbf6-cdf95b5e888b" (UID: "8728778c-db1b-45ec-bbf6-cdf95b5e888b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:40:29.528142 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.528102    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-workload-socket\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:29.528142 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.528137    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-credential-socket\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:29.528334 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.528151    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-podinfo\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:29.528334 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.528164    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-data\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:29.528334 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.528175    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-token\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:29.528334 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.528187    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjflw\" (UniqueName: \"kubernetes.io/projected/8728778c-db1b-45ec-bbf6-cdf95b5e888b-kube-api-access-wjflw\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:29.528334 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.528200    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istio-envoy\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:29.528334 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.528213    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8728778c-db1b-45ec-bbf6-cdf95b5e888b-workload-certs\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:29.528334 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.528224    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8728778c-db1b-45ec-bbf6-cdf95b5e888b-istiod-ca-cert\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:29.586050 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.585967    2555 generic.go:356] "Generic (PLEG): container finished" podID="8728778c-db1b-45ec-bbf6-cdf95b5e888b" containerID="9c64a1ea7b1f2147052870460149e8aae44a7cfb07aa203d641494773e03e8ac" exitCode=0
Jul 07 22:40:29.586050 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.586018    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"
Jul 07 22:40:29.586226 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.586015    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x" event={"ID":"8728778c-db1b-45ec-bbf6-cdf95b5e888b","Type":"ContainerDied","Data":"9c64a1ea7b1f2147052870460149e8aae44a7cfb07aa203d641494773e03e8ac"}
Jul 07 22:40:29.586226 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.586161    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x" event={"ID":"8728778c-db1b-45ec-bbf6-cdf95b5e888b","Type":"ContainerDied","Data":"e352cddc4b50862b6c4446907b0d97f1093d6e7660d8fb14e47c283e028ee397"}
Jul 07 22:40:29.586226 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.586190    2555 scope.go:117] "RemoveContainer" containerID="9c64a1ea7b1f2147052870460149e8aae44a7cfb07aa203d641494773e03e8ac"
Jul 07 22:40:29.595684 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.595668    2555 scope.go:117] "RemoveContainer" containerID="9c64a1ea7b1f2147052870460149e8aae44a7cfb07aa203d641494773e03e8ac"
Jul 07 22:40:29.595990 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:40:29.595973    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c64a1ea7b1f2147052870460149e8aae44a7cfb07aa203d641494773e03e8ac\": container with ID starting with 9c64a1ea7b1f2147052870460149e8aae44a7cfb07aa203d641494773e03e8ac not found: ID does not exist" containerID="9c64a1ea7b1f2147052870460149e8aae44a7cfb07aa203d641494773e03e8ac"
Jul 07 22:40:29.596050 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.596000    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c64a1ea7b1f2147052870460149e8aae44a7cfb07aa203d641494773e03e8ac"} err="failed to get container status \"9c64a1ea7b1f2147052870460149e8aae44a7cfb07aa203d641494773e03e8ac\": rpc error: code = NotFound desc = could not find container \"9c64a1ea7b1f2147052870460149e8aae44a7cfb07aa203d641494773e03e8ac\": container with ID starting with 9c64a1ea7b1f2147052870460149e8aae44a7cfb07aa203d641494773e03e8ac not found: ID does not exist"
Jul 07 22:40:29.607430 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.607406    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"]
Jul 07 22:40:29.610803 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:29.610782    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-fe0013e5-openshift-default-5b6b9878cc-dmf4x"]
Jul 07 22:40:30.922754 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:30.922720    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8728778c-db1b-45ec-bbf6-cdf95b5e888b" path="/var/lib/kubelet/pods/8728778c-db1b-45ec-bbf6-cdf95b5e888b/volumes"
Jul 07 22:40:34.943407 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:34.943377    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"]
Jul 07 22:40:34.943790 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:34.943698    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908" containerName="maas-api"
Jul 07 22:40:34.943790 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:34.943711    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908" containerName="maas-api"
Jul 07 22:40:34.943790 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:34.943724    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8728778c-db1b-45ec-bbf6-cdf95b5e888b" containerName="istio-proxy"
Jul 07 22:40:34.943790 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:34.943729    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="8728778c-db1b-45ec-bbf6-cdf95b5e888b" containerName="istio-proxy"
Jul 07 22:40:34.943790 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:34.943790    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="1eb2f0b2-c1f0-4f40-b1ce-a0a28b06d908" containerName="maas-api"
Jul 07 22:40:34.943977 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:34.943802    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="8728778c-db1b-45ec-bbf6-cdf95b5e888b" containerName="istio-proxy"
Jul 07 22:40:34.945800 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:34.945784    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:34.948244 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:34.948222    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 07 22:40:34.948427 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:34.948407    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-1d73c518-openshift-default-dockercfg-s6gfx\""
Jul 07 22:40:34.960008 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:34.959985    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"]
Jul 07 22:40:34.978376 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:34.978347    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gg4g\" (UniqueName: \"kubernetes.io/projected/758c26bd-7c06-4f32-a660-deed4f04e12c-kube-api-access-4gg4g\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:34.978477 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:34.978381    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-envoy\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:34.978477 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:34.978415    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-credential-socket\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:34.978547 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:34.978508    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-workload-socket\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:34.978590 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:34.978549    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-data\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:34.978590 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:34.978580    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-token\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:34.978677 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:34.978626    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-workload-certs\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:34.978725 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:34.978698    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-podinfo\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:34.978777 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:34.978738    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/758c26bd-7c06-4f32-a660-deed4f04e12c-istiod-ca-cert\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:35.080077 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.080051    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-workload-socket\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:35.080253 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.080084    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-data\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:35.080253 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.080103    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-token\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:35.080253 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.080166    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-workload-certs\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:35.080253 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.080223    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-podinfo\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:35.080253 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.080250    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/758c26bd-7c06-4f32-a660-deed4f04e12c-istiod-ca-cert\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:35.080519 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.080311    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gg4g\" (UniqueName: \"kubernetes.io/projected/758c26bd-7c06-4f32-a660-deed4f04e12c-kube-api-access-4gg4g\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:35.080519 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.080347    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-envoy\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:35.080519 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.080397    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-credential-socket\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:35.080666 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.080531    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-data\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:35.080666 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.080558    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-workload-certs\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:35.080666 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.080610    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-workload-socket\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:35.080817 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.080747    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-credential-socket\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:35.081126 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.081103    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/758c26bd-7c06-4f32-a660-deed4f04e12c-istiod-ca-cert\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:35.082617 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.082591    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-envoy\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:35.082789 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.082772    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-podinfo\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:35.088202 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.088175    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-token\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:35.088361 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.088340    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gg4g\" (UniqueName: \"kubernetes.io/projected/758c26bd-7c06-4f32-a660-deed4f04e12c-kube-api-access-4gg4g\") pod \"e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") " pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:35.255862 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.255826    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:35.390489 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.390462    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"]
Jul 07 22:40:35.393184 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:40:35.393157    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod758c26bd_7c06_4f32_a660_deed4f04e12c.slice/crio-448aad43bdcc0531bbae65689ae89e330fcfccfcdf04a67ba353100bf52824c1 WatchSource:0}: Error finding container 448aad43bdcc0531bbae65689ae89e330fcfccfcdf04a67ba353100bf52824c1: Status 404 returned error can't find the container with id 448aad43bdcc0531bbae65689ae89e330fcfccfcdf04a67ba353100bf52824c1
Jul 07 22:40:35.395734 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.395692    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:40:35.395812 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.395785    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:40:35.395853 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.395826    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:40:35.622385 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.622297    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4" event={"ID":"758c26bd-7c06-4f32-a660-deed4f04e12c","Type":"ContainerStarted","Data":"d00bc9fad8413d2ca22a35540ae86457dc922703a048164750336e638eda287d"}
Jul 07 22:40:35.622385 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.622359    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4" event={"ID":"758c26bd-7c06-4f32-a660-deed4f04e12c","Type":"ContainerStarted","Data":"448aad43bdcc0531bbae65689ae89e330fcfccfcdf04a67ba353100bf52824c1"}
Jul 07 22:40:35.645219 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:35.645173    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4" podStartSLOduration=1.645158729 podStartE2EDuration="1.645158729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:40:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:40:35.643545757 +0000 UTC m=+1533.315764752" watchObservedRunningTime="2026-07-07 22:40:35.645158729 +0000 UTC m=+1533.317377717"
Jul 07 22:40:36.213587 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:36.213556    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx"]
Jul 07 22:40:36.215823 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:36.215803    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx"
Jul 07 22:40:36.218252 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:36.218234    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-1d73c518-serving-cert\""
Jul 07 22:40:36.218358 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:36.218240    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-r9rv4\""
Jul 07 22:40:36.226836 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:36.226815    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx"]
Jul 07 22:40:36.257013 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:36.256986    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:36.261768 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:36.261745    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:36.292087 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:36.292064    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78dafa91-e87b-41b4-aa78-f7dca53aa4ca-openshift-service-ca\") pod \"maas-api-e2e-derive-1d73c518-778b9d84db-77qwx\" (UID: \"78dafa91-e87b-41b4-aa78-f7dca53aa4ca\") " pod="opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx"
Jul 07 22:40:36.292226 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:36.292101    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfw6m\" (UniqueName: \"kubernetes.io/projected/78dafa91-e87b-41b4-aa78-f7dca53aa4ca-kube-api-access-qfw6m\") pod \"maas-api-e2e-derive-1d73c518-778b9d84db-77qwx\" (UID: \"78dafa91-e87b-41b4-aa78-f7dca53aa4ca\") " pod="opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx"
Jul 07 22:40:36.292288 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:36.292239    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78dafa91-e87b-41b4-aa78-f7dca53aa4ca-maas-api-tls\") pod \"maas-api-e2e-derive-1d73c518-778b9d84db-77qwx\" (UID: \"78dafa91-e87b-41b4-aa78-f7dca53aa4ca\") " pod="opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx"
Jul 07 22:40:36.392720 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:36.392685    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78dafa91-e87b-41b4-aa78-f7dca53aa4ca-maas-api-tls\") pod \"maas-api-e2e-derive-1d73c518-778b9d84db-77qwx\" (UID: \"78dafa91-e87b-41b4-aa78-f7dca53aa4ca\") " pod="opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx"
Jul 07 22:40:36.392910 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:36.392749    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78dafa91-e87b-41b4-aa78-f7dca53aa4ca-openshift-service-ca\") pod \"maas-api-e2e-derive-1d73c518-778b9d84db-77qwx\" (UID: \"78dafa91-e87b-41b4-aa78-f7dca53aa4ca\") " pod="opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx"
Jul 07 22:40:36.392910 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:40:36.392831    2555 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-1d73c518-serving-cert: secret "maas-api-e2e-derive-1d73c518-serving-cert" not found
Jul 07 22:40:36.393010 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:40:36.392921    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78dafa91-e87b-41b4-aa78-f7dca53aa4ca-maas-api-tls podName:78dafa91-e87b-41b4-aa78-f7dca53aa4ca nodeName:}" failed. No retries permitted until 2026-07-07 22:40:36.892900391 +0000 UTC m=+1534.565119369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/78dafa91-e87b-41b4-aa78-f7dca53aa4ca-maas-api-tls") pod "maas-api-e2e-derive-1d73c518-778b9d84db-77qwx" (UID: "78dafa91-e87b-41b4-aa78-f7dca53aa4ca") : secret "maas-api-e2e-derive-1d73c518-serving-cert" not found
Jul 07 22:40:36.393010 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:36.392915    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfw6m\" (UniqueName: \"kubernetes.io/projected/78dafa91-e87b-41b4-aa78-f7dca53aa4ca-kube-api-access-qfw6m\") pod \"maas-api-e2e-derive-1d73c518-778b9d84db-77qwx\" (UID: \"78dafa91-e87b-41b4-aa78-f7dca53aa4ca\") " pod="opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx"
Jul 07 22:40:36.393421 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:36.393403    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78dafa91-e87b-41b4-aa78-f7dca53aa4ca-openshift-service-ca\") pod \"maas-api-e2e-derive-1d73c518-778b9d84db-77qwx\" (UID: \"78dafa91-e87b-41b4-aa78-f7dca53aa4ca\") " pod="opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx"
Jul 07 22:40:36.403838 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:36.403815    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfw6m\" (UniqueName: \"kubernetes.io/projected/78dafa91-e87b-41b4-aa78-f7dca53aa4ca-kube-api-access-qfw6m\") pod \"maas-api-e2e-derive-1d73c518-778b9d84db-77qwx\" (UID: \"78dafa91-e87b-41b4-aa78-f7dca53aa4ca\") " pod="opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx"
Jul 07 22:40:36.626724 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:36.626641    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:36.627496 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:36.627467    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:36.897614 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:36.897527    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78dafa91-e87b-41b4-aa78-f7dca53aa4ca-maas-api-tls\") pod \"maas-api-e2e-derive-1d73c518-778b9d84db-77qwx\" (UID: \"78dafa91-e87b-41b4-aa78-f7dca53aa4ca\") " pod="opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx"
Jul 07 22:40:36.899760 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:36.899743    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78dafa91-e87b-41b4-aa78-f7dca53aa4ca-maas-api-tls\") pod \"maas-api-e2e-derive-1d73c518-778b9d84db-77qwx\" (UID: \"78dafa91-e87b-41b4-aa78-f7dca53aa4ca\") " pod="opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx"
Jul 07 22:40:37.127706 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:37.127675    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx"
Jul 07 22:40:37.254655 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:37.254603    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx"]
Jul 07 22:40:37.257409 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:40:37.257385    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78dafa91_e87b_41b4_aa78_f7dca53aa4ca.slice/crio-001650853d93ff51fe4f01ecb4b5501a4ce60e975214c1e0f06ecaa5b0674feb WatchSource:0}: Error finding container 001650853d93ff51fe4f01ecb4b5501a4ce60e975214c1e0f06ecaa5b0674feb: Status 404 returned error can't find the container with id 001650853d93ff51fe4f01ecb4b5501a4ce60e975214c1e0f06ecaa5b0674feb
Jul 07 22:40:37.633502 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:37.633463    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx" event={"ID":"78dafa91-e87b-41b4-aa78-f7dca53aa4ca","Type":"ContainerStarted","Data":"001650853d93ff51fe4f01ecb4b5501a4ce60e975214c1e0f06ecaa5b0674feb"}
Jul 07 22:40:38.639835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:38.639803    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx" event={"ID":"78dafa91-e87b-41b4-aa78-f7dca53aa4ca","Type":"ContainerStarted","Data":"ad31e69d63b92c433c4c5faf51d4f465d601fdb94c99407e53c79061eb852e57"}
Jul 07 22:40:38.657609 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:38.657559    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx" podStartSLOduration=2.168333875 podStartE2EDuration="2.657543453s" totalImagesPullingTime="489.209578ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:40:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:40:38.657285134 +0000 UTC m=+1536.329504121" watchObservedRunningTime="2026-07-07 22:40:38.657543453 +0000 UTC m=+1536.329762441"
Jul 07 22:40:39.644461 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:39.644429    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx"
Jul 07 22:40:40.668784 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:40.668748    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"]
Jul 07 22:40:40.669580 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:40.669528    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4" podUID="758c26bd-7c06-4f32-a660-deed4f04e12c" containerName="istio-proxy" containerID="cri-o://d00bc9fad8413d2ca22a35540ae86457dc922703a048164750336e638eda287d" gracePeriod=30
Jul 07 22:40:40.864217 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:40.864164    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx"]
Jul 07 22:40:41.653203 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:41.653160    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx" podUID="78dafa91-e87b-41b4-aa78-f7dca53aa4ca" containerName="maas-api" containerID="cri-o://ad31e69d63b92c433c4c5faf51d4f465d601fdb94c99407e53c79061eb852e57" gracePeriod=30
Jul 07 22:40:41.927745 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:41.927714    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx"
Jul 07 22:40:42.049928 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:42.049850    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfw6m\" (UniqueName: \"kubernetes.io/projected/78dafa91-e87b-41b4-aa78-f7dca53aa4ca-kube-api-access-qfw6m\") pod \"78dafa91-e87b-41b4-aa78-f7dca53aa4ca\" (UID: \"78dafa91-e87b-41b4-aa78-f7dca53aa4ca\") "
Jul 07 22:40:42.050094 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:42.049956    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78dafa91-e87b-41b4-aa78-f7dca53aa4ca-maas-api-tls\") pod \"78dafa91-e87b-41b4-aa78-f7dca53aa4ca\" (UID: \"78dafa91-e87b-41b4-aa78-f7dca53aa4ca\") "
Jul 07 22:40:42.050094 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:42.049992    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78dafa91-e87b-41b4-aa78-f7dca53aa4ca-openshift-service-ca\") pod \"78dafa91-e87b-41b4-aa78-f7dca53aa4ca\" (UID: \"78dafa91-e87b-41b4-aa78-f7dca53aa4ca\") "
Jul 07 22:40:42.050358 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:42.050334    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78dafa91-e87b-41b4-aa78-f7dca53aa4ca-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "78dafa91-e87b-41b4-aa78-f7dca53aa4ca" (UID: "78dafa91-e87b-41b4-aa78-f7dca53aa4ca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 22:40:42.051899 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:42.051862    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78dafa91-e87b-41b4-aa78-f7dca53aa4ca-kube-api-access-qfw6m" (OuterVolumeSpecName: "kube-api-access-qfw6m") pod "78dafa91-e87b-41b4-aa78-f7dca53aa4ca" (UID: "78dafa91-e87b-41b4-aa78-f7dca53aa4ca"). InnerVolumeSpecName "kube-api-access-qfw6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:40:42.052018 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:42.052002    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78dafa91-e87b-41b4-aa78-f7dca53aa4ca-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "78dafa91-e87b-41b4-aa78-f7dca53aa4ca" (UID: "78dafa91-e87b-41b4-aa78-f7dca53aa4ca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 22:40:42.151081 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:42.151037    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78dafa91-e87b-41b4-aa78-f7dca53aa4ca-openshift-service-ca\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:42.151081 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:42.151074    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfw6m\" (UniqueName: \"kubernetes.io/projected/78dafa91-e87b-41b4-aa78-f7dca53aa4ca-kube-api-access-qfw6m\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:42.151081 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:42.151085    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78dafa91-e87b-41b4-aa78-f7dca53aa4ca-maas-api-tls\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:42.662951 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:42.659624    2555 generic.go:356] "Generic (PLEG): container finished" podID="78dafa91-e87b-41b4-aa78-f7dca53aa4ca" containerID="ad31e69d63b92c433c4c5faf51d4f465d601fdb94c99407e53c79061eb852e57" exitCode=0
Jul 07 22:40:42.662951 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:42.659669    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx" event={"ID":"78dafa91-e87b-41b4-aa78-f7dca53aa4ca","Type":"ContainerDied","Data":"ad31e69d63b92c433c4c5faf51d4f465d601fdb94c99407e53c79061eb852e57"}
Jul 07 22:40:42.662951 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:42.659722    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx" event={"ID":"78dafa91-e87b-41b4-aa78-f7dca53aa4ca","Type":"ContainerDied","Data":"001650853d93ff51fe4f01ecb4b5501a4ce60e975214c1e0f06ecaa5b0674feb"}
Jul 07 22:40:42.662951 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:42.659762    2555 scope.go:117] "RemoveContainer" containerID="ad31e69d63b92c433c4c5faf51d4f465d601fdb94c99407e53c79061eb852e57"
Jul 07 22:40:42.662951 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:42.659953    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx"
Jul 07 22:40:42.689448 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:42.689419    2555 scope.go:117] "RemoveContainer" containerID="ad31e69d63b92c433c4c5faf51d4f465d601fdb94c99407e53c79061eb852e57"
Jul 07 22:40:42.689699 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:40:42.689681    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad31e69d63b92c433c4c5faf51d4f465d601fdb94c99407e53c79061eb852e57\": container with ID starting with ad31e69d63b92c433c4c5faf51d4f465d601fdb94c99407e53c79061eb852e57 not found: ID does not exist" containerID="ad31e69d63b92c433c4c5faf51d4f465d601fdb94c99407e53c79061eb852e57"
Jul 07 22:40:42.689752 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:42.689708    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad31e69d63b92c433c4c5faf51d4f465d601fdb94c99407e53c79061eb852e57"} err="failed to get container status \"ad31e69d63b92c433c4c5faf51d4f465d601fdb94c99407e53c79061eb852e57\": rpc error: code = NotFound desc = could not find container \"ad31e69d63b92c433c4c5faf51d4f465d601fdb94c99407e53c79061eb852e57\": container with ID starting with ad31e69d63b92c433c4c5faf51d4f465d601fdb94c99407e53c79061eb852e57 not found: ID does not exist"
Jul 07 22:40:42.692751 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:42.692726    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx"]
Jul 07 22:40:42.696485 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:42.696463    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-1d73c518-778b9d84db-77qwx"]
Jul 07 22:40:42.923442 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:42.923366    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78dafa91-e87b-41b4-aa78-f7dca53aa4ca" path="/var/lib/kubelet/pods/78dafa91-e87b-41b4-aa78-f7dca53aa4ca/volumes"
Jul 07 22:40:46.688212 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.688185    2555 generic.go:356] "Generic (PLEG): container finished" podID="758c26bd-7c06-4f32-a660-deed4f04e12c" containerID="d00bc9fad8413d2ca22a35540ae86457dc922703a048164750336e638eda287d" exitCode=0
Jul 07 22:40:46.688554 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.688259    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4" event={"ID":"758c26bd-7c06-4f32-a660-deed4f04e12c","Type":"ContainerDied","Data":"d00bc9fad8413d2ca22a35540ae86457dc922703a048164750336e638eda287d"}
Jul 07 22:40:46.705847 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.705827    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:46.807029 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.806950    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-workload-certs\") pod \"758c26bd-7c06-4f32-a660-deed4f04e12c\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") "
Jul 07 22:40:46.807029 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.806988    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/758c26bd-7c06-4f32-a660-deed4f04e12c-istiod-ca-cert\") pod \"758c26bd-7c06-4f32-a660-deed4f04e12c\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") "
Jul 07 22:40:46.807299 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.807034    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-token\") pod \"758c26bd-7c06-4f32-a660-deed4f04e12c\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") "
Jul 07 22:40:46.807299 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.807075    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-credential-socket\") pod \"758c26bd-7c06-4f32-a660-deed4f04e12c\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") "
Jul 07 22:40:46.807299 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.807097    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-workload-socket\") pod \"758c26bd-7c06-4f32-a660-deed4f04e12c\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") "
Jul 07 22:40:46.807299 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.807135    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gg4g\" (UniqueName: \"kubernetes.io/projected/758c26bd-7c06-4f32-a660-deed4f04e12c-kube-api-access-4gg4g\") pod \"758c26bd-7c06-4f32-a660-deed4f04e12c\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") "
Jul 07 22:40:46.807299 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.807148    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-envoy\") pod \"758c26bd-7c06-4f32-a660-deed4f04e12c\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") "
Jul 07 22:40:46.807299 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.807166    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-data\") pod \"758c26bd-7c06-4f32-a660-deed4f04e12c\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") "
Jul 07 22:40:46.807299 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.807186    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-podinfo\") pod \"758c26bd-7c06-4f32-a660-deed4f04e12c\" (UID: \"758c26bd-7c06-4f32-a660-deed4f04e12c\") "
Jul 07 22:40:46.807299 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.807271    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "758c26bd-7c06-4f32-a660-deed4f04e12c" (UID: "758c26bd-7c06-4f32-a660-deed4f04e12c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:40:46.807688 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.807365    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "758c26bd-7c06-4f32-a660-deed4f04e12c" (UID: "758c26bd-7c06-4f32-a660-deed4f04e12c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:40:46.807688 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.807389    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/758c26bd-7c06-4f32-a660-deed4f04e12c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "758c26bd-7c06-4f32-a660-deed4f04e12c" (UID: "758c26bd-7c06-4f32-a660-deed4f04e12c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 22:40:46.807688 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.807517    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-credential-socket\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:46.807688 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.807539    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-workload-certs\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:46.807688 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.807555    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/758c26bd-7c06-4f32-a660-deed4f04e12c-istiod-ca-cert\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:46.807961 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.807692    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "758c26bd-7c06-4f32-a660-deed4f04e12c" (UID: "758c26bd-7c06-4f32-a660-deed4f04e12c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:40:46.808018 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.807978    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-data" (OuterVolumeSpecName: "istio-data") pod "758c26bd-7c06-4f32-a660-deed4f04e12c" (UID: "758c26bd-7c06-4f32-a660-deed4f04e12c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:40:46.809398 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.809368    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-token" (OuterVolumeSpecName: "istio-token") pod "758c26bd-7c06-4f32-a660-deed4f04e12c" (UID: "758c26bd-7c06-4f32-a660-deed4f04e12c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:40:46.809398 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.809379    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "758c26bd-7c06-4f32-a660-deed4f04e12c" (UID: "758c26bd-7c06-4f32-a660-deed4f04e12c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:40:46.809672 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.809652    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "758c26bd-7c06-4f32-a660-deed4f04e12c" (UID: "758c26bd-7c06-4f32-a660-deed4f04e12c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 22:40:46.809735 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.809665    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/758c26bd-7c06-4f32-a660-deed4f04e12c-kube-api-access-4gg4g" (OuterVolumeSpecName: "kube-api-access-4gg4g") pod "758c26bd-7c06-4f32-a660-deed4f04e12c" (UID: "758c26bd-7c06-4f32-a660-deed4f04e12c"). InnerVolumeSpecName "kube-api-access-4gg4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:40:46.908593 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.908558    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-workload-socket\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:46.908593 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.908585    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gg4g\" (UniqueName: \"kubernetes.io/projected/758c26bd-7c06-4f32-a660-deed4f04e12c-kube-api-access-4gg4g\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:46.908593 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.908595    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-envoy\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:46.908807 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.908609    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-data\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:46.908807 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.908618    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-podinfo\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:46.908807 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:46.908626    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/758c26bd-7c06-4f32-a660-deed4f04e12c-istio-token\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:40:47.694110 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:47.694078    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4" event={"ID":"758c26bd-7c06-4f32-a660-deed4f04e12c","Type":"ContainerDied","Data":"448aad43bdcc0531bbae65689ae89e330fcfccfcdf04a67ba353100bf52824c1"}
Jul 07 22:40:47.694110 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:47.694096    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"
Jul 07 22:40:47.694596 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:47.694136    2555 scope.go:117] "RemoveContainer" containerID="d00bc9fad8413d2ca22a35540ae86457dc922703a048164750336e638eda287d"
Jul 07 22:40:47.713248 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:47.713224    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"]
Jul 07 22:40:47.718250 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:47.718230    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-1d73c518-openshift-default-6d7895f-jbnp4"]
Jul 07 22:40:48.922418 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:40:48.922387    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="758c26bd-7c06-4f32-a660-deed4f04e12c" path="/var/lib/kubelet/pods/758c26bd-7c06-4f32-a660-deed4f04e12c/volumes"
Jul 07 22:42:21.356900 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.356813    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"]
Jul 07 22:42:21.357329 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.357147    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="758c26bd-7c06-4f32-a660-deed4f04e12c" containerName="istio-proxy"
Jul 07 22:42:21.357329 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.357159    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="758c26bd-7c06-4f32-a660-deed4f04e12c" containerName="istio-proxy"
Jul 07 22:42:21.357329 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.357166    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78dafa91-e87b-41b4-aa78-f7dca53aa4ca" containerName="maas-api"
Jul 07 22:42:21.357329 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.357171    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="78dafa91-e87b-41b4-aa78-f7dca53aa4ca" containerName="maas-api"
Jul 07 22:42:21.357329 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.357230    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="78dafa91-e87b-41b4-aa78-f7dca53aa4ca" containerName="maas-api"
Jul 07 22:42:21.357329 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.357241    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="758c26bd-7c06-4f32-a660-deed4f04e12c" containerName="istio-proxy"
Jul 07 22:42:21.359028 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.359013    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.361393 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.361371    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-740f61f7-openshift-default-dockercfg-kxhkn\""
Jul 07 22:42:21.361536 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.361520    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 07 22:42:21.381567 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.381541    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"]
Jul 07 22:42:21.445741 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.445706    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-data\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.445934 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.445757    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-workload-certs\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.445934 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.445836    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-workload-socket\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.445934 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.445865    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-credential-socket\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.445934 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.445920    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-podinfo\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.446100 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.445939    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65hxv\" (UniqueName: \"kubernetes.io/projected/609ebe14-11f7-4403-8c78-87348a8c8a14-kube-api-access-65hxv\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.446100 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.445965    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-envoy\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.446100 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.446004    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-token\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.446100 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.446023    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/609ebe14-11f7-4403-8c78-87348a8c8a14-istiod-ca-cert\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.547008 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.546961    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65hxv\" (UniqueName: \"kubernetes.io/projected/609ebe14-11f7-4403-8c78-87348a8c8a14-kube-api-access-65hxv\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.547188 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.547018    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-envoy\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.547188 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.547047    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-token\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.547188 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.547071    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/609ebe14-11f7-4403-8c78-87348a8c8a14-istiod-ca-cert\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.547188 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.547120    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-data\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.547188 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.547180    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-workload-certs\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.547476 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.547238    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-workload-socket\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.547476 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.547266    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-credential-socket\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.547476 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.547293    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-podinfo\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.547630 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.547578    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-data\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.547684 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.547647    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-workload-certs\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.547736 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.547688    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-credential-socket\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.547736 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.547719    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-workload-socket\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.547956 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.547938    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/609ebe14-11f7-4403-8c78-87348a8c8a14-istiod-ca-cert\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.549573 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.549542    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-envoy\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.549690 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.549644    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-podinfo\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.554157 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.554135    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-token\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.554504 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.554481    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65hxv\" (UniqueName: \"kubernetes.io/projected/609ebe14-11f7-4403-8c78-87348a8c8a14-kube-api-access-65hxv\") pod \"e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") " pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.669366 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.669279    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:21.810638 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.810611    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"]
Jul 07 22:42:21.812994 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:42:21.812970    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod609ebe14_11f7_4403_8c78_87348a8c8a14.slice/crio-ac3b03fd2dfbb48f17c52d0ea1258b0dd796ec85d26a596f86db232000f75d85 WatchSource:0}: Error finding container ac3b03fd2dfbb48f17c52d0ea1258b0dd796ec85d26a596f86db232000f75d85: Status 404 returned error can't find the container with id ac3b03fd2dfbb48f17c52d0ea1258b0dd796ec85d26a596f86db232000f75d85
Jul 07 22:42:21.815029 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.814996    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:42:21.815115 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.815057    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:42:21.815115 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:21.815087    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:42:22.105608 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:22.105571    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb" event={"ID":"609ebe14-11f7-4403-8c78-87348a8c8a14","Type":"ContainerStarted","Data":"33abaad3cf3f6b3cc7f4d9ada2b9a9c8cad6d0c24498c5d5c26b7bebc906b278"}
Jul 07 22:42:22.105807 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:22.105630    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb" event={"ID":"609ebe14-11f7-4403-8c78-87348a8c8a14","Type":"ContainerStarted","Data":"ac3b03fd2dfbb48f17c52d0ea1258b0dd796ec85d26a596f86db232000f75d85"}
Jul 07 22:42:22.125341 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:22.125293    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb" podStartSLOduration=1.125277873 podStartE2EDuration="1.125277873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:42:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:42:22.123517306 +0000 UTC m=+1639.795736294" watchObservedRunningTime="2026-07-07 22:42:22.125277873 +0000 UTC m=+1639.797496919"
Jul 07 22:42:22.670358 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:22.670311    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:22.675097 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:22.675074    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:23.110117 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:23.110081    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:23.110939 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:23.110918    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:24.109159 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:24.109127    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t"]
Jul 07 22:42:24.111695 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:24.111671    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t"
Jul 07 22:42:24.114200 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:24.114176    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-r9rv4\""
Jul 07 22:42:24.114322 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:24.114220    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-740f61f7-serving-cert\""
Jul 07 22:42:24.121824 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:24.121795    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t"]
Jul 07 22:42:24.272412 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:24.272368    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1ada20a-e5d5-4ae0-bcca-e020d8ef3552-maas-api-tls\") pod \"maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t\" (UID: \"b1ada20a-e5d5-4ae0-bcca-e020d8ef3552\") " pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t"
Jul 07 22:42:24.272615 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:24.272452    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t2sm\" (UniqueName: \"kubernetes.io/projected/b1ada20a-e5d5-4ae0-bcca-e020d8ef3552-kube-api-access-5t2sm\") pod \"maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t\" (UID: \"b1ada20a-e5d5-4ae0-bcca-e020d8ef3552\") " pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t"
Jul 07 22:42:24.272677 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:24.272618    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1ada20a-e5d5-4ae0-bcca-e020d8ef3552-openshift-service-ca\") pod \"maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t\" (UID: \"b1ada20a-e5d5-4ae0-bcca-e020d8ef3552\") " pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t"
Jul 07 22:42:24.373810 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:24.373714    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t2sm\" (UniqueName: \"kubernetes.io/projected/b1ada20a-e5d5-4ae0-bcca-e020d8ef3552-kube-api-access-5t2sm\") pod \"maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t\" (UID: \"b1ada20a-e5d5-4ae0-bcca-e020d8ef3552\") " pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t"
Jul 07 22:42:24.373810 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:24.373771    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1ada20a-e5d5-4ae0-bcca-e020d8ef3552-openshift-service-ca\") pod \"maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t\" (UID: \"b1ada20a-e5d5-4ae0-bcca-e020d8ef3552\") " pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t"
Jul 07 22:42:24.374084 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:24.373825    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1ada20a-e5d5-4ae0-bcca-e020d8ef3552-maas-api-tls\") pod \"maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t\" (UID: \"b1ada20a-e5d5-4ae0-bcca-e020d8ef3552\") " pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t"
Jul 07 22:42:24.374483 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:24.374458    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1ada20a-e5d5-4ae0-bcca-e020d8ef3552-openshift-service-ca\") pod \"maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t\" (UID: \"b1ada20a-e5d5-4ae0-bcca-e020d8ef3552\") " pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t"
Jul 07 22:42:24.376239 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:24.376221    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1ada20a-e5d5-4ae0-bcca-e020d8ef3552-maas-api-tls\") pod \"maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t\" (UID: \"b1ada20a-e5d5-4ae0-bcca-e020d8ef3552\") " pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t"
Jul 07 22:42:24.381497 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:24.381473    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t2sm\" (UniqueName: \"kubernetes.io/projected/b1ada20a-e5d5-4ae0-bcca-e020d8ef3552-kube-api-access-5t2sm\") pod \"maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t\" (UID: \"b1ada20a-e5d5-4ae0-bcca-e020d8ef3552\") " pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t"
Jul 07 22:42:24.424298 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:24.424262    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t"
Jul 07 22:42:24.552901 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:24.552851    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t"]
Jul 07 22:42:24.555441 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:42:24.555411    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1ada20a_e5d5_4ae0_bcca_e020d8ef3552.slice/crio-e298899c7f032e2871bf8247a5da150c76861e53712d4e5d812ebfab9d336eef WatchSource:0}: Error finding container e298899c7f032e2871bf8247a5da150c76861e53712d4e5d812ebfab9d336eef: Status 404 returned error can't find the container with id e298899c7f032e2871bf8247a5da150c76861e53712d4e5d812ebfab9d336eef
Jul 07 22:42:25.120341 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:25.120301    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t" event={"ID":"b1ada20a-e5d5-4ae0-bcca-e020d8ef3552","Type":"ContainerStarted","Data":"08fb7388520dce42002def9d2ab8eae30c5240cdd3b43446aff6c591e527d8d0"}
Jul 07 22:42:25.120762 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:25.120361    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t" event={"ID":"b1ada20a-e5d5-4ae0-bcca-e020d8ef3552","Type":"ContainerStarted","Data":"e298899c7f032e2871bf8247a5da150c76861e53712d4e5d812ebfab9d336eef"}
Jul 07 22:42:25.120762 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:25.120481    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t"
Jul 07 22:42:25.155257 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:25.155209    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t" podStartSLOduration=0.718463353 podStartE2EDuration="1.155191101s" totalImagesPullingTime="436.727748ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:42:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:42:25.153433351 +0000 UTC m=+1642.825652351" watchObservedRunningTime="2026-07-07 22:42:25.155191101 +0000 UTC m=+1642.827410085"
Jul 07 22:42:31.129895 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:31.129839    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t"
Jul 07 22:42:36.092492 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:36.092446    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t"]
Jul 07 22:42:36.092977 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:36.092709    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t" podUID="b1ada20a-e5d5-4ae0-bcca-e020d8ef3552" containerName="maas-api" containerID="cri-o://08fb7388520dce42002def9d2ab8eae30c5240cdd3b43446aff6c591e527d8d0" gracePeriod=30
Jul 07 22:42:36.124632 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:36.124592    2555 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t" podUID="b1ada20a-e5d5-4ae0-bcca-e020d8ef3552" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.37:8443/health\": dial tcp 10.133.0.37:8443: connect: connection refused"
Jul 07 22:42:36.347330 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:36.347269    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t"
Jul 07 22:42:36.386130 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:36.386100    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5t2sm\" (UniqueName: \"kubernetes.io/projected/b1ada20a-e5d5-4ae0-bcca-e020d8ef3552-kube-api-access-5t2sm\") pod \"b1ada20a-e5d5-4ae0-bcca-e020d8ef3552\" (UID: \"b1ada20a-e5d5-4ae0-bcca-e020d8ef3552\") "
Jul 07 22:42:36.386316 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:36.386161    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1ada20a-e5d5-4ae0-bcca-e020d8ef3552-maas-api-tls\") pod \"b1ada20a-e5d5-4ae0-bcca-e020d8ef3552\" (UID: \"b1ada20a-e5d5-4ae0-bcca-e020d8ef3552\") "
Jul 07 22:42:36.386316 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:36.386186    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1ada20a-e5d5-4ae0-bcca-e020d8ef3552-openshift-service-ca\") pod \"b1ada20a-e5d5-4ae0-bcca-e020d8ef3552\" (UID: \"b1ada20a-e5d5-4ae0-bcca-e020d8ef3552\") "
Jul 07 22:42:36.386595 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:36.386569    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1ada20a-e5d5-4ae0-bcca-e020d8ef3552-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b1ada20a-e5d5-4ae0-bcca-e020d8ef3552" (UID: "b1ada20a-e5d5-4ae0-bcca-e020d8ef3552"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 22:42:36.388345 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:36.388317    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1ada20a-e5d5-4ae0-bcca-e020d8ef3552-kube-api-access-5t2sm" (OuterVolumeSpecName: "kube-api-access-5t2sm") pod "b1ada20a-e5d5-4ae0-bcca-e020d8ef3552" (UID: "b1ada20a-e5d5-4ae0-bcca-e020d8ef3552"). InnerVolumeSpecName "kube-api-access-5t2sm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:42:36.388345 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:36.388328    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1ada20a-e5d5-4ae0-bcca-e020d8ef3552-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b1ada20a-e5d5-4ae0-bcca-e020d8ef3552" (UID: "b1ada20a-e5d5-4ae0-bcca-e020d8ef3552"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 22:42:36.487041 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:36.487001    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1ada20a-e5d5-4ae0-bcca-e020d8ef3552-maas-api-tls\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:42:36.487041 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:36.487037    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1ada20a-e5d5-4ae0-bcca-e020d8ef3552-openshift-service-ca\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:42:36.487041 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:36.487047    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5t2sm\" (UniqueName: \"kubernetes.io/projected/b1ada20a-e5d5-4ae0-bcca-e020d8ef3552-kube-api-access-5t2sm\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:42:37.175567 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:37.175471    2555 generic.go:356] "Generic (PLEG): container finished" podID="b1ada20a-e5d5-4ae0-bcca-e020d8ef3552" containerID="08fb7388520dce42002def9d2ab8eae30c5240cdd3b43446aff6c591e527d8d0" exitCode=0
Jul 07 22:42:37.175567 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:37.175542    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t"
Jul 07 22:42:37.175567 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:37.175555    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t" event={"ID":"b1ada20a-e5d5-4ae0-bcca-e020d8ef3552","Type":"ContainerDied","Data":"08fb7388520dce42002def9d2ab8eae30c5240cdd3b43446aff6c591e527d8d0"}
Jul 07 22:42:37.176203 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:37.175612    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t" event={"ID":"b1ada20a-e5d5-4ae0-bcca-e020d8ef3552","Type":"ContainerDied","Data":"e298899c7f032e2871bf8247a5da150c76861e53712d4e5d812ebfab9d336eef"}
Jul 07 22:42:37.176203 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:37.175682    2555 scope.go:117] "RemoveContainer" containerID="08fb7388520dce42002def9d2ab8eae30c5240cdd3b43446aff6c591e527d8d0"
Jul 07 22:42:37.177259 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:37.177236    2555 generic.go:356] "Generic (PLEG): container finished" podID="609ebe14-11f7-4403-8c78-87348a8c8a14" containerID="33abaad3cf3f6b3cc7f4d9ada2b9a9c8cad6d0c24498c5d5c26b7bebc906b278" exitCode=0
Jul 07 22:42:37.177547 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:37.177294    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb" event={"ID":"609ebe14-11f7-4403-8c78-87348a8c8a14","Type":"ContainerDied","Data":"33abaad3cf3f6b3cc7f4d9ada2b9a9c8cad6d0c24498c5d5c26b7bebc906b278"}
Jul 07 22:42:37.177673 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:37.177651    2555 scope.go:117] "RemoveContainer" containerID="33abaad3cf3f6b3cc7f4d9ada2b9a9c8cad6d0c24498c5d5c26b7bebc906b278"
Jul 07 22:42:37.178713 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:37.178683    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:42:37.178806 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:37.178752    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:42:37.178806 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:37.178787    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:42:37.188077 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:37.188057    2555 scope.go:117] "RemoveContainer" containerID="08fb7388520dce42002def9d2ab8eae30c5240cdd3b43446aff6c591e527d8d0"
Jul 07 22:42:37.188382 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:42:37.188366    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08fb7388520dce42002def9d2ab8eae30c5240cdd3b43446aff6c591e527d8d0\": container with ID starting with 08fb7388520dce42002def9d2ab8eae30c5240cdd3b43446aff6c591e527d8d0 not found: ID does not exist" containerID="08fb7388520dce42002def9d2ab8eae30c5240cdd3b43446aff6c591e527d8d0"
Jul 07 22:42:37.188425 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:37.188389    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08fb7388520dce42002def9d2ab8eae30c5240cdd3b43446aff6c591e527d8d0"} err="failed to get container status \"08fb7388520dce42002def9d2ab8eae30c5240cdd3b43446aff6c591e527d8d0\": rpc error: code = NotFound desc = could not find container \"08fb7388520dce42002def9d2ab8eae30c5240cdd3b43446aff6c591e527d8d0\": container with ID starting with 08fb7388520dce42002def9d2ab8eae30c5240cdd3b43446aff6c591e527d8d0 not found: ID does not exist"
Jul 07 22:42:37.211799 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:37.211746    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t"]
Jul 07 22:42:37.220056 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:37.220029    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-740f61f7-686664c4f9-ccm2t"]
Jul 07 22:42:38.110011 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:38.109971    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:38.187427 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:38.187393    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb" event={"ID":"609ebe14-11f7-4403-8c78-87348a8c8a14","Type":"ContainerStarted","Data":"ddf3469717c37d761971459afff0fbe68c25e40d7639e7489669c5516f9ddfa5"}
Jul 07 22:42:38.670852 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:38.670809    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:38.671076 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:38.670985    2555 patch_prober.go:28] interesting pod/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 07 22:42:38.671076 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:38.671045    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb" podUID="609ebe14-11f7-4403-8c78-87348a8c8a14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 07 22:42:38.923143 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:38.923065    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1ada20a-e5d5-4ae0-bcca-e020d8ef3552" path="/var/lib/kubelet/pods/b1ada20a-e5d5-4ae0-bcca-e020d8ef3552/volumes"
Jul 07 22:42:39.670194 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:39.670157    2555 patch_prober.go:28] interesting pod/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 07 22:42:39.670763 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:39.670729    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb" podUID="609ebe14-11f7-4403-8c78-87348a8c8a14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 07 22:42:40.670714 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:40.670675    2555 patch_prober.go:28] interesting pod/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 07 22:42:40.671350 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:40.671314    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb" podUID="609ebe14-11f7-4403-8c78-87348a8c8a14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 07 22:42:41.670386 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:41.669922    2555 patch_prober.go:28] interesting pod/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 07 22:42:41.670386 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:41.670002    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb" podUID="609ebe14-11f7-4403-8c78-87348a8c8a14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 07 22:42:42.340438 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:42.340302    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"]
Jul 07 22:42:42.341096 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:42.341065    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb" podUID="609ebe14-11f7-4403-8c78-87348a8c8a14" containerName="istio-proxy" containerID="cri-o://ddf3469717c37d761971459afff0fbe68c25e40d7639e7489669c5516f9ddfa5" gracePeriod=30
Jul 07 22:42:47.588365 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.588340    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:47.679582 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.679491    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-envoy\") pod \"609ebe14-11f7-4403-8c78-87348a8c8a14\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") "
Jul 07 22:42:47.679582 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.679546    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-credential-socket\") pod \"609ebe14-11f7-4403-8c78-87348a8c8a14\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") "
Jul 07 22:42:47.679582 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.679569    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/609ebe14-11f7-4403-8c78-87348a8c8a14-istiod-ca-cert\") pod \"609ebe14-11f7-4403-8c78-87348a8c8a14\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") "
Jul 07 22:42:47.679856 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.679590    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-data\") pod \"609ebe14-11f7-4403-8c78-87348a8c8a14\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") "
Jul 07 22:42:47.679856 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.679606    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-token\") pod \"609ebe14-11f7-4403-8c78-87348a8c8a14\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") "
Jul 07 22:42:47.679856 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.679643    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65hxv\" (UniqueName: \"kubernetes.io/projected/609ebe14-11f7-4403-8c78-87348a8c8a14-kube-api-access-65hxv\") pod \"609ebe14-11f7-4403-8c78-87348a8c8a14\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") "
Jul 07 22:42:47.679856 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.679825    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "609ebe14-11f7-4403-8c78-87348a8c8a14" (UID: "609ebe14-11f7-4403-8c78-87348a8c8a14"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:42:47.680101 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.679948    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-workload-socket\") pod \"609ebe14-11f7-4403-8c78-87348a8c8a14\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") "
Jul 07 22:42:47.680101 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.679981    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-podinfo\") pod \"609ebe14-11f7-4403-8c78-87348a8c8a14\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") "
Jul 07 22:42:47.680101 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.680014    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-workload-certs\") pod \"609ebe14-11f7-4403-8c78-87348a8c8a14\" (UID: \"609ebe14-11f7-4403-8c78-87348a8c8a14\") "
Jul 07 22:42:47.680101 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.680046    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/609ebe14-11f7-4403-8c78-87348a8c8a14-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "609ebe14-11f7-4403-8c78-87348a8c8a14" (UID: "609ebe14-11f7-4403-8c78-87348a8c8a14"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 22:42:47.680304 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.680177    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "609ebe14-11f7-4403-8c78-87348a8c8a14" (UID: "609ebe14-11f7-4403-8c78-87348a8c8a14"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:42:47.680388 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.680366    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-credential-socket\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:42:47.680462 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.680398    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/609ebe14-11f7-4403-8c78-87348a8c8a14-istiod-ca-cert\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:42:47.680462 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.680413    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-workload-socket\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:42:47.680462 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.680371    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "609ebe14-11f7-4403-8c78-87348a8c8a14" (UID: "609ebe14-11f7-4403-8c78-87348a8c8a14"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:42:47.680617 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.680571    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-data" (OuterVolumeSpecName: "istio-data") pod "609ebe14-11f7-4403-8c78-87348a8c8a14" (UID: "609ebe14-11f7-4403-8c78-87348a8c8a14"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:42:47.681901 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.681859    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/609ebe14-11f7-4403-8c78-87348a8c8a14-kube-api-access-65hxv" (OuterVolumeSpecName: "kube-api-access-65hxv") pod "609ebe14-11f7-4403-8c78-87348a8c8a14" (UID: "609ebe14-11f7-4403-8c78-87348a8c8a14"). InnerVolumeSpecName "kube-api-access-65hxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:42:47.682012 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.681985    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "609ebe14-11f7-4403-8c78-87348a8c8a14" (UID: "609ebe14-11f7-4403-8c78-87348a8c8a14"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 22:42:47.682232 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.682206    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "609ebe14-11f7-4403-8c78-87348a8c8a14" (UID: "609ebe14-11f7-4403-8c78-87348a8c8a14"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:42:47.682298 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.682257    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-token" (OuterVolumeSpecName: "istio-token") pod "609ebe14-11f7-4403-8c78-87348a8c8a14" (UID: "609ebe14-11f7-4403-8c78-87348a8c8a14"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:42:47.781090 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.781030    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-data\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:42:47.781090 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.781078    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-token\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:42:47.781090 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.781088    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65hxv\" (UniqueName: \"kubernetes.io/projected/609ebe14-11f7-4403-8c78-87348a8c8a14-kube-api-access-65hxv\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:42:47.781090 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.781098    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-podinfo\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:42:47.781090 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.781108    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-workload-certs\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:42:47.781448 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:47.781117    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/609ebe14-11f7-4403-8c78-87348a8c8a14-istio-envoy\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:42:48.236504 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:48.236467    2555 generic.go:356] "Generic (PLEG): container finished" podID="609ebe14-11f7-4403-8c78-87348a8c8a14" containerID="ddf3469717c37d761971459afff0fbe68c25e40d7639e7489669c5516f9ddfa5" exitCode=0
Jul 07 22:42:48.236684 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:48.236532    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"
Jul 07 22:42:48.236684 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:48.236563    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb" event={"ID":"609ebe14-11f7-4403-8c78-87348a8c8a14","Type":"ContainerDied","Data":"ddf3469717c37d761971459afff0fbe68c25e40d7639e7489669c5516f9ddfa5"}
Jul 07 22:42:48.236684 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:48.236624    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb" event={"ID":"609ebe14-11f7-4403-8c78-87348a8c8a14","Type":"ContainerDied","Data":"ac3b03fd2dfbb48f17c52d0ea1258b0dd796ec85d26a596f86db232000f75d85"}
Jul 07 22:42:48.236684 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:48.236641    2555 scope.go:117] "RemoveContainer" containerID="ddf3469717c37d761971459afff0fbe68c25e40d7639e7489669c5516f9ddfa5"
Jul 07 22:42:48.245515 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:48.245498    2555 scope.go:117] "RemoveContainer" containerID="33abaad3cf3f6b3cc7f4d9ada2b9a9c8cad6d0c24498c5d5c26b7bebc906b278"
Jul 07 22:42:48.253287 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:48.253268    2555 scope.go:117] "RemoveContainer" containerID="ddf3469717c37d761971459afff0fbe68c25e40d7639e7489669c5516f9ddfa5"
Jul 07 22:42:48.253595 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:42:48.253572    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddf3469717c37d761971459afff0fbe68c25e40d7639e7489669c5516f9ddfa5\": container with ID starting with ddf3469717c37d761971459afff0fbe68c25e40d7639e7489669c5516f9ddfa5 not found: ID does not exist" containerID="ddf3469717c37d761971459afff0fbe68c25e40d7639e7489669c5516f9ddfa5"
Jul 07 22:42:48.253647 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:48.253604    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddf3469717c37d761971459afff0fbe68c25e40d7639e7489669c5516f9ddfa5"} err="failed to get container status \"ddf3469717c37d761971459afff0fbe68c25e40d7639e7489669c5516f9ddfa5\": rpc error: code = NotFound desc = could not find container \"ddf3469717c37d761971459afff0fbe68c25e40d7639e7489669c5516f9ddfa5\": container with ID starting with ddf3469717c37d761971459afff0fbe68c25e40d7639e7489669c5516f9ddfa5 not found: ID does not exist"
Jul 07 22:42:48.253647 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:48.253621    2555 scope.go:117] "RemoveContainer" containerID="33abaad3cf3f6b3cc7f4d9ada2b9a9c8cad6d0c24498c5d5c26b7bebc906b278"
Jul 07 22:42:48.253844 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:42:48.253827    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33abaad3cf3f6b3cc7f4d9ada2b9a9c8cad6d0c24498c5d5c26b7bebc906b278\": container with ID starting with 33abaad3cf3f6b3cc7f4d9ada2b9a9c8cad6d0c24498c5d5c26b7bebc906b278 not found: ID does not exist" containerID="33abaad3cf3f6b3cc7f4d9ada2b9a9c8cad6d0c24498c5d5c26b7bebc906b278"
Jul 07 22:42:48.253920 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:48.253854    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33abaad3cf3f6b3cc7f4d9ada2b9a9c8cad6d0c24498c5d5c26b7bebc906b278"} err="failed to get container status \"33abaad3cf3f6b3cc7f4d9ada2b9a9c8cad6d0c24498c5d5c26b7bebc906b278\": rpc error: code = NotFound desc = could not find container \"33abaad3cf3f6b3cc7f4d9ada2b9a9c8cad6d0c24498c5d5c26b7bebc906b278\": container with ID starting with 33abaad3cf3f6b3cc7f4d9ada2b9a9c8cad6d0c24498c5d5c26b7bebc906b278 not found: ID does not exist"
Jul 07 22:42:48.256549 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:48.256529    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"]
Jul 07 22:42:48.262027 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:48.262000    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-740f61f7-openshift-default-f574bf79-kjkkb"]
Jul 07 22:42:48.922499 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:42:48.922466    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="609ebe14-11f7-4403-8c78-87348a8c8a14" path="/var/lib/kubelet/pods/609ebe14-11f7-4403-8c78-87348a8c8a14/volumes"
Jul 07 22:43:01.734087 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.734044    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"]
Jul 07 22:43:01.734578 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.734496    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="609ebe14-11f7-4403-8c78-87348a8c8a14" containerName="istio-proxy"
Jul 07 22:43:01.734578 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.734512    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="609ebe14-11f7-4403-8c78-87348a8c8a14" containerName="istio-proxy"
Jul 07 22:43:01.734578 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.734550    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1ada20a-e5d5-4ae0-bcca-e020d8ef3552" containerName="maas-api"
Jul 07 22:43:01.734578 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.734558    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1ada20a-e5d5-4ae0-bcca-e020d8ef3552" containerName="maas-api"
Jul 07 22:43:01.734742 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.734637    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="609ebe14-11f7-4403-8c78-87348a8c8a14" containerName="istio-proxy"
Jul 07 22:43:01.734742 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.734650    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="609ebe14-11f7-4403-8c78-87348a8c8a14" containerName="istio-proxy"
Jul 07 22:43:01.734742 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.734660    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1ada20a-e5d5-4ae0-bcca-e020d8ef3552" containerName="maas-api"
Jul 07 22:43:01.737425 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.737407    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.740204 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.740177    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-928393a0-openshift-default-dockercfg-vcm6d\""
Jul 07 22:43:01.740317 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.740226    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 07 22:43:01.754781 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.754757    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"]
Jul 07 22:43:01.803157 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.803120    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-podinfo\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.803157 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.803158    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-workload-socket\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.803358 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.803186    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-credential-socket\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.803358 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.803247    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfg2x\" (UniqueName: \"kubernetes.io/projected/575be9b7-4dbf-4300-a7d6-c855483c8e40-kube-api-access-qfg2x\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.803358 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.803283    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-token\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.803358 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.803319    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-workload-certs\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.803502 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.803361    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575be9b7-4dbf-4300-a7d6-c855483c8e40-istiod-ca-cert\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.803502 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.803400    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-envoy\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.803502 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.803429    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-data\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.904457 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.904424    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-workload-certs\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.904457 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.904461    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575be9b7-4dbf-4300-a7d6-c855483c8e40-istiod-ca-cert\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.904743 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.904482    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-envoy\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.904743 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.904508    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-data\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.904743 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.904544    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-podinfo\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.904743 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.904590    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-workload-socket\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.904743 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.904628    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-credential-socket\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.904743 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.904678    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfg2x\" (UniqueName: \"kubernetes.io/projected/575be9b7-4dbf-4300-a7d6-c855483c8e40-kube-api-access-qfg2x\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.904743 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.904728    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-token\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.905109 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.905043    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-workload-certs\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.905167 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.905126    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-data\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.905289 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.905259    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575be9b7-4dbf-4300-a7d6-c855483c8e40-istiod-ca-cert\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.905438 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.905417    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-credential-socket\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.905504 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.905416    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-workload-socket\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.907208 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.907185    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-envoy\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.907303 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.907286    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-podinfo\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.912448 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.912420    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-token\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:01.912640 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:01.912621    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfg2x\" (UniqueName: \"kubernetes.io/projected/575be9b7-4dbf-4300-a7d6-c855483c8e40-kube-api-access-qfg2x\") pod \"e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") " pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:02.048150 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:02.048044    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:02.186648 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:02.186620    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"]
Jul 07 22:43:02.189369 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:43:02.189335    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod575be9b7_4dbf_4300_a7d6_c855483c8e40.slice/crio-71f787c1a3b0bffd5224e72b0e2ca8ea071b7175d98c1dbc5fd990be88c703b5 WatchSource:0}: Error finding container 71f787c1a3b0bffd5224e72b0e2ca8ea071b7175d98c1dbc5fd990be88c703b5: Status 404 returned error can't find the container with id 71f787c1a3b0bffd5224e72b0e2ca8ea071b7175d98c1dbc5fd990be88c703b5
Jul 07 22:43:02.191591 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:02.191557    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:43:02.191678 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:02.191624    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:43:02.191678 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:02.191653    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:43:02.302287 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:02.302194    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v" event={"ID":"575be9b7-4dbf-4300-a7d6-c855483c8e40","Type":"ContainerStarted","Data":"576a0a0d2b98c4a0c6d26ecdc39a17b380357d20aa517a61c4272eafcf040f8a"}
Jul 07 22:43:02.302287 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:02.302256    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v" event={"ID":"575be9b7-4dbf-4300-a7d6-c855483c8e40","Type":"ContainerStarted","Data":"71f787c1a3b0bffd5224e72b0e2ca8ea071b7175d98c1dbc5fd990be88c703b5"}
Jul 07 22:43:02.321423 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:02.321363    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v" podStartSLOduration=1.321344778 podStartE2EDuration="1.321344778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:43:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:43:02.320166383 +0000 UTC m=+1679.992385382" watchObservedRunningTime="2026-07-07 22:43:02.321344778 +0000 UTC m=+1679.993563769"
Jul 07 22:43:03.048348 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:03.048307    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:03.053000 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:03.052973    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:03.307600 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:03.307523    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:03.308597 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:03.308576    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:04.322622 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:04.322589    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"]
Jul 07 22:43:04.323146 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:04.323128    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="609ebe14-11f7-4403-8c78-87348a8c8a14" containerName="istio-proxy"
Jul 07 22:43:04.323221 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:04.323149    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="609ebe14-11f7-4403-8c78-87348a8c8a14" containerName="istio-proxy"
Jul 07 22:43:04.325124 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:04.325101    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"
Jul 07 22:43:04.332692 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:04.332663    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-928393a0-serving-cert\""
Jul 07 22:43:04.332859 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:04.332670    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-r9rv4\""
Jul 07 22:43:04.336470 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:04.336443    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"]
Jul 07 22:43:04.433805 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:04.433769    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e546a2b8-5867-4db6-b2db-fd7210e81f60-openshift-service-ca\") pod \"maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5\" (UID: \"e546a2b8-5867-4db6-b2db-fd7210e81f60\") " pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"
Jul 07 22:43:04.433805 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:04.433804    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e546a2b8-5867-4db6-b2db-fd7210e81f60-maas-api-tls\") pod \"maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5\" (UID: \"e546a2b8-5867-4db6-b2db-fd7210e81f60\") " pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"
Jul 07 22:43:04.434082 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:04.434033    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9khdg\" (UniqueName: \"kubernetes.io/projected/e546a2b8-5867-4db6-b2db-fd7210e81f60-kube-api-access-9khdg\") pod \"maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5\" (UID: \"e546a2b8-5867-4db6-b2db-fd7210e81f60\") " pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"
Jul 07 22:43:04.535130 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:04.535091    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e546a2b8-5867-4db6-b2db-fd7210e81f60-openshift-service-ca\") pod \"maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5\" (UID: \"e546a2b8-5867-4db6-b2db-fd7210e81f60\") " pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"
Jul 07 22:43:04.535130 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:04.535128    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e546a2b8-5867-4db6-b2db-fd7210e81f60-maas-api-tls\") pod \"maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5\" (UID: \"e546a2b8-5867-4db6-b2db-fd7210e81f60\") " pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"
Jul 07 22:43:04.535349 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:04.535184    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9khdg\" (UniqueName: \"kubernetes.io/projected/e546a2b8-5867-4db6-b2db-fd7210e81f60-kube-api-access-9khdg\") pod \"maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5\" (UID: \"e546a2b8-5867-4db6-b2db-fd7210e81f60\") " pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"
Jul 07 22:43:04.535349 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:43:04.535302    2555 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-928393a0-serving-cert: secret "maas-api-e2e-mt-928393a0-serving-cert" not found
Jul 07 22:43:04.535426 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:43:04.535370    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e546a2b8-5867-4db6-b2db-fd7210e81f60-maas-api-tls podName:e546a2b8-5867-4db6-b2db-fd7210e81f60 nodeName:}" failed. No retries permitted until 2026-07-07 22:43:05.035351472 +0000 UTC m=+1682.707570438 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e546a2b8-5867-4db6-b2db-fd7210e81f60-maas-api-tls") pod "maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5" (UID: "e546a2b8-5867-4db6-b2db-fd7210e81f60") : secret "maas-api-e2e-mt-928393a0-serving-cert" not found
Jul 07 22:43:04.535722 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:04.535702    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e546a2b8-5867-4db6-b2db-fd7210e81f60-openshift-service-ca\") pod \"maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5\" (UID: \"e546a2b8-5867-4db6-b2db-fd7210e81f60\") " pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"
Jul 07 22:43:04.561082 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:04.561052    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9khdg\" (UniqueName: \"kubernetes.io/projected/e546a2b8-5867-4db6-b2db-fd7210e81f60-kube-api-access-9khdg\") pod \"maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5\" (UID: \"e546a2b8-5867-4db6-b2db-fd7210e81f60\") " pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"
Jul 07 22:43:05.040917 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:05.040864    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e546a2b8-5867-4db6-b2db-fd7210e81f60-maas-api-tls\") pod \"maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5\" (UID: \"e546a2b8-5867-4db6-b2db-fd7210e81f60\") " pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"
Jul 07 22:43:05.043434 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:05.043408    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e546a2b8-5867-4db6-b2db-fd7210e81f60-maas-api-tls\") pod \"maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5\" (UID: \"e546a2b8-5867-4db6-b2db-fd7210e81f60\") " pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"
Jul 07 22:43:05.243233 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:05.243197    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"
Jul 07 22:43:05.372687 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:05.372639    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"]
Jul 07 22:43:05.374792 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:43:05.374767    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode546a2b8_5867_4db6_b2db_fd7210e81f60.slice/crio-ff45784857068bbf431faddd32d6d42de7339fcf5e4a05b1d9c58a2eef64f011 WatchSource:0}: Error finding container ff45784857068bbf431faddd32d6d42de7339fcf5e4a05b1d9c58a2eef64f011: Status 404 returned error can't find the container with id ff45784857068bbf431faddd32d6d42de7339fcf5e4a05b1d9c58a2eef64f011
Jul 07 22:43:06.322909 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:06.322857    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5" event={"ID":"e546a2b8-5867-4db6-b2db-fd7210e81f60","Type":"ContainerStarted","Data":"87fa85448de2b060875317c427c7a954593948f9cb3364de7139f2fd773ee09e"}
Jul 07 22:43:06.323106 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:06.322927    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5" event={"ID":"e546a2b8-5867-4db6-b2db-fd7210e81f60","Type":"ContainerStarted","Data":"ff45784857068bbf431faddd32d6d42de7339fcf5e4a05b1d9c58a2eef64f011"}
Jul 07 22:43:06.323106 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:06.322979    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"
Jul 07 22:43:06.340835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:06.340780    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5" podStartSLOduration=1.928543085 podStartE2EDuration="2.340763709s" totalImagesPullingTime="412.220624ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:43:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:43:06.33857182 +0000 UTC m=+1684.010790820" watchObservedRunningTime="2026-07-07 22:43:06.340763709 +0000 UTC m=+1684.012982697"
Jul 07 22:43:12.332952 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:12.332922    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"
Jul 07 22:43:15.972953 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:15.972918    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"]
Jul 07 22:43:15.975313 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:15.975292    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:15.977943 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:15.977918    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-928393a0\"/\"openshift-service-ca.crt\""
Jul 07 22:43:15.978899 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:15.978855    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-928393a0\"/\"kube-root-ca.crt\""
Jul 07 22:43:15.979007 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:15.978903    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-928393a0\"/\"e2e-lifecycle-model-928393a0-kserve-self-signed-certs\""
Jul 07 22:43:15.979076 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:15.979025    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-928393a0\"/\"default-dockercfg-mw2wx\""
Jul 07 22:43:15.986296 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:15.986270    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"]
Jul 07 22:43:16.148488 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:16.148455    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/625a1ee6-2adf-4896-b781-e35afca48ed0-tls-certs\") pod \"e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") " pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:16.148662 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:16.148496    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-dshm\") pod \"e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") " pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:16.148662 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:16.148612    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-tmp-dir\") pod \"e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") " pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:16.148740 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:16.148670    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l8qr\" (UniqueName: \"kubernetes.io/projected/625a1ee6-2adf-4896-b781-e35afca48ed0-kube-api-access-4l8qr\") pod \"e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") " pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:16.148774 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:16.148748    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-home\") pod \"e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") " pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:16.148810 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:16.148780    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-model-cache\") pod \"e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") " pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:16.250247 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:16.250163    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-home\") pod \"e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") " pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:16.250247 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:16.250198    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-model-cache\") pod \"e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") " pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:16.250247 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:16.250227    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/625a1ee6-2adf-4896-b781-e35afca48ed0-tls-certs\") pod \"e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") " pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:16.250247 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:16.250249    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-dshm\") pod \"e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") " pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:16.250579 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:16.250306    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-tmp-dir\") pod \"e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") " pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:16.250579 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:16.250332    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l8qr\" (UniqueName: \"kubernetes.io/projected/625a1ee6-2adf-4896-b781-e35afca48ed0-kube-api-access-4l8qr\") pod \"e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") " pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:16.250685 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:16.250627    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-model-cache\") pod \"e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") " pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:16.250685 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:16.250640    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-home\") pod \"e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") " pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:16.250774 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:16.250702    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-tmp-dir\") pod \"e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") " pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:16.252507 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:16.252487    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-dshm\") pod \"e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") " pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:16.252845 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:16.252824    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/625a1ee6-2adf-4896-b781-e35afca48ed0-tls-certs\") pod \"e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") " pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:16.257887 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:16.257838    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l8qr\" (UniqueName: \"kubernetes.io/projected/625a1ee6-2adf-4896-b781-e35afca48ed0-kube-api-access-4l8qr\") pod \"e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") " pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:16.284708 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:16.284674    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:16.426328 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:16.426236    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"]
Jul 07 22:43:16.428806 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:43:16.428775    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod625a1ee6_2adf_4896_b781_e35afca48ed0.slice/crio-23fbb0af487758488e26a4a98e674cbacffd0459e38b51444d07cbdb28d14edd WatchSource:0}: Error finding container 23fbb0af487758488e26a4a98e674cbacffd0459e38b51444d07cbdb28d14edd: Status 404 returned error can't find the container with id 23fbb0af487758488e26a4a98e674cbacffd0459e38b51444d07cbdb28d14edd
Jul 07 22:43:17.378935 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:17.378898    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6" event={"ID":"625a1ee6-2adf-4896-b781-e35afca48ed0","Type":"ContainerStarted","Data":"197018ee5aad0169b2a5cb3f968553612d9e96c584732a3b866bb0cd92c0efa2"}
Jul 07 22:43:17.379416 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:17.378945    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6" event={"ID":"625a1ee6-2adf-4896-b781-e35afca48ed0","Type":"ContainerStarted","Data":"23fbb0af487758488e26a4a98e674cbacffd0459e38b51444d07cbdb28d14edd"}
Jul 07 22:43:17.380458 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:17.380437    2555 generic.go:356] "Generic (PLEG): container finished" podID="575be9b7-4dbf-4300-a7d6-c855483c8e40" containerID="576a0a0d2b98c4a0c6d26ecdc39a17b380357d20aa517a61c4272eafcf040f8a" exitCode=0
Jul 07 22:43:17.380551 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:17.380505    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v" event={"ID":"575be9b7-4dbf-4300-a7d6-c855483c8e40","Type":"ContainerDied","Data":"576a0a0d2b98c4a0c6d26ecdc39a17b380357d20aa517a61c4272eafcf040f8a"}
Jul 07 22:43:17.380847 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:17.380834    2555 scope.go:117] "RemoveContainer" containerID="576a0a0d2b98c4a0c6d26ecdc39a17b380357d20aa517a61c4272eafcf040f8a"
Jul 07 22:43:17.381890 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:17.381842    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:43:17.381984 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:17.381930    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:43:17.382046 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:17.381982    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:43:17.397097 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:17.397045    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6" podStartSLOduration=2.397026485 podStartE2EDuration="2.397026485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:43:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:43:17.394762355 +0000 UTC m=+1695.066981345" watchObservedRunningTime="2026-07-07 22:43:17.397026485 +0000 UTC m=+1695.069245475"
Jul 07 22:43:18.307692 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:18.307652    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:18.388269 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:18.388232    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v" event={"ID":"575be9b7-4dbf-4300-a7d6-c855483c8e40","Type":"ContainerStarted","Data":"902259de2ac7db931173ce705a59df913b559cb94b806f12cd9f625530648b7c"}
Jul 07 22:43:19.048946 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:19.048909    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:19.050073 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:19.050049    2555 patch_prober.go:28] interesting pod/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 07 22:43:19.050184 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:19.050099    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v" podUID="575be9b7-4dbf-4300-a7d6-c855483c8e40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 07 22:43:20.048633 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:20.048594    2555 patch_prober.go:28] interesting pod/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 07 22:43:20.049082 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:20.048654    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v" podUID="575be9b7-4dbf-4300-a7d6-c855483c8e40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 07 22:43:21.049044 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:21.049002    2555 patch_prober.go:28] interesting pod/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 07 22:43:21.049458 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:21.049063    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v" podUID="575be9b7-4dbf-4300-a7d6-c855483c8e40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 07 22:43:22.049067 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:22.049029    2555 patch_prober.go:28] interesting pod/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 07 22:43:22.049481 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:22.049091    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v" podUID="575be9b7-4dbf-4300-a7d6-c855483c8e40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 07 22:43:23.052320 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:23.052288    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:23.052690 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:23.052629    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:23.053374 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:23.053357    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:43:26.285753 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:26.285663    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:26.285753 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:26.285718    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:26.298242 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:26.298215    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:26.438921 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:26.438893    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:43:41.810725 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:41.810689    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"]
Jul 07 22:43:41.811325 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:41.810902    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5" podUID="e546a2b8-5867-4db6-b2db-fd7210e81f60" containerName="maas-api" containerID="cri-o://87fa85448de2b060875317c427c7a954593948f9cb3364de7139f2fd773ee09e" gracePeriod=30
Jul 07 22:43:42.063008 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:42.062683    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"
Jul 07 22:43:42.187302 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:42.187275    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e546a2b8-5867-4db6-b2db-fd7210e81f60-maas-api-tls\") pod \"e546a2b8-5867-4db6-b2db-fd7210e81f60\" (UID: \"e546a2b8-5867-4db6-b2db-fd7210e81f60\") "
Jul 07 22:43:42.187460 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:42.187407    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e546a2b8-5867-4db6-b2db-fd7210e81f60-openshift-service-ca\") pod \"e546a2b8-5867-4db6-b2db-fd7210e81f60\" (UID: \"e546a2b8-5867-4db6-b2db-fd7210e81f60\") "
Jul 07 22:43:42.187460 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:42.187426    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9khdg\" (UniqueName: \"kubernetes.io/projected/e546a2b8-5867-4db6-b2db-fd7210e81f60-kube-api-access-9khdg\") pod \"e546a2b8-5867-4db6-b2db-fd7210e81f60\" (UID: \"e546a2b8-5867-4db6-b2db-fd7210e81f60\") "
Jul 07 22:43:42.187738 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:42.187711    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e546a2b8-5867-4db6-b2db-fd7210e81f60-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e546a2b8-5867-4db6-b2db-fd7210e81f60" (UID: "e546a2b8-5867-4db6-b2db-fd7210e81f60"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 22:43:42.189204 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:42.189183    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e546a2b8-5867-4db6-b2db-fd7210e81f60-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e546a2b8-5867-4db6-b2db-fd7210e81f60" (UID: "e546a2b8-5867-4db6-b2db-fd7210e81f60"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 22:43:42.189305 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:42.189280    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e546a2b8-5867-4db6-b2db-fd7210e81f60-kube-api-access-9khdg" (OuterVolumeSpecName: "kube-api-access-9khdg") pod "e546a2b8-5867-4db6-b2db-fd7210e81f60" (UID: "e546a2b8-5867-4db6-b2db-fd7210e81f60"). InnerVolumeSpecName "kube-api-access-9khdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:43:42.288914 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:42.288841    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e546a2b8-5867-4db6-b2db-fd7210e81f60-openshift-service-ca\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:43:42.288914 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:42.288908    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9khdg\" (UniqueName: \"kubernetes.io/projected/e546a2b8-5867-4db6-b2db-fd7210e81f60-kube-api-access-9khdg\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:43:42.288914 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:42.288920    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e546a2b8-5867-4db6-b2db-fd7210e81f60-maas-api-tls\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:43:42.507551 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:42.507515    2555 generic.go:356] "Generic (PLEG): container finished" podID="e546a2b8-5867-4db6-b2db-fd7210e81f60" containerID="87fa85448de2b060875317c427c7a954593948f9cb3364de7139f2fd773ee09e" exitCode=0
Jul 07 22:43:42.507719 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:42.507564    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"
Jul 07 22:43:42.507719 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:42.507601    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5" event={"ID":"e546a2b8-5867-4db6-b2db-fd7210e81f60","Type":"ContainerDied","Data":"87fa85448de2b060875317c427c7a954593948f9cb3364de7139f2fd773ee09e"}
Jul 07 22:43:42.507719 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:42.507654    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5" event={"ID":"e546a2b8-5867-4db6-b2db-fd7210e81f60","Type":"ContainerDied","Data":"ff45784857068bbf431faddd32d6d42de7339fcf5e4a05b1d9c58a2eef64f011"}
Jul 07 22:43:42.507719 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:42.507676    2555 scope.go:117] "RemoveContainer" containerID="87fa85448de2b060875317c427c7a954593948f9cb3364de7139f2fd773ee09e"
Jul 07 22:43:42.516636 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:42.516620    2555 scope.go:117] "RemoveContainer" containerID="87fa85448de2b060875317c427c7a954593948f9cb3364de7139f2fd773ee09e"
Jul 07 22:43:42.516907 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:43:42.516882    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87fa85448de2b060875317c427c7a954593948f9cb3364de7139f2fd773ee09e\": container with ID starting with 87fa85448de2b060875317c427c7a954593948f9cb3364de7139f2fd773ee09e not found: ID does not exist" containerID="87fa85448de2b060875317c427c7a954593948f9cb3364de7139f2fd773ee09e"
Jul 07 22:43:42.516956 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:42.516919    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87fa85448de2b060875317c427c7a954593948f9cb3364de7139f2fd773ee09e"} err="failed to get container status \"87fa85448de2b060875317c427c7a954593948f9cb3364de7139f2fd773ee09e\": rpc error: code = NotFound desc = could not find container \"87fa85448de2b060875317c427c7a954593948f9cb3364de7139f2fd773ee09e\": container with ID starting with 87fa85448de2b060875317c427c7a954593948f9cb3364de7139f2fd773ee09e not found: ID does not exist"
Jul 07 22:43:42.528032 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:42.528008    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"]
Jul 07 22:43:42.531946 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:42.531919    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-928393a0-c4c4bfb7-8xvg5"]
Jul 07 22:43:42.922843 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:42.922768    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e546a2b8-5867-4db6-b2db-fd7210e81f60" path="/var/lib/kubelet/pods/e546a2b8-5867-4db6-b2db-fd7210e81f60/volumes"
Jul 07 22:43:52.080742 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:43:52.080705    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"]
Jul 07 22:44:07.101283 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.101220    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6" podUID="625a1ee6-2adf-4896-b781-e35afca48ed0" containerName="main" containerID="cri-o://197018ee5aad0169b2a5cb3f968553612d9e96c584732a3b866bb0cd92c0efa2" gracePeriod=45
Jul 07 22:44:07.337659 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.337633    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:44:07.420470 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.420377    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4l8qr\" (UniqueName: \"kubernetes.io/projected/625a1ee6-2adf-4896-b781-e35afca48ed0-kube-api-access-4l8qr\") pod \"625a1ee6-2adf-4896-b781-e35afca48ed0\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") "
Jul 07 22:44:07.420470 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.420419    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-model-cache\") pod \"625a1ee6-2adf-4896-b781-e35afca48ed0\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") "
Jul 07 22:44:07.420470 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.420466    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/625a1ee6-2adf-4896-b781-e35afca48ed0-tls-certs\") pod \"625a1ee6-2adf-4896-b781-e35afca48ed0\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") "
Jul 07 22:44:07.420735 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.420493    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-dshm\") pod \"625a1ee6-2adf-4896-b781-e35afca48ed0\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") "
Jul 07 22:44:07.420735 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.420616    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-tmp-dir\") pod \"625a1ee6-2adf-4896-b781-e35afca48ed0\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") "
Jul 07 22:44:07.420735 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.420639    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-home\") pod \"625a1ee6-2adf-4896-b781-e35afca48ed0\" (UID: \"625a1ee6-2adf-4896-b781-e35afca48ed0\") "
Jul 07 22:44:07.420735 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.420703    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-model-cache" (OuterVolumeSpecName: "model-cache") pod "625a1ee6-2adf-4896-b781-e35afca48ed0" (UID: "625a1ee6-2adf-4896-b781-e35afca48ed0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:44:07.420985 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.420926    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "625a1ee6-2adf-4896-b781-e35afca48ed0" (UID: "625a1ee6-2adf-4896-b781-e35afca48ed0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:44:07.420985 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.420947    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-home" (OuterVolumeSpecName: "home") pod "625a1ee6-2adf-4896-b781-e35afca48ed0" (UID: "625a1ee6-2adf-4896-b781-e35afca48ed0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:44:07.421093 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.421047    2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-model-cache\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:44:07.421093 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.421064    2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-tmp-dir\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:44:07.421093 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.421077    2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-home\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:44:07.422629 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.422605    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/625a1ee6-2adf-4896-b781-e35afca48ed0-kube-api-access-4l8qr" (OuterVolumeSpecName: "kube-api-access-4l8qr") pod "625a1ee6-2adf-4896-b781-e35afca48ed0" (UID: "625a1ee6-2adf-4896-b781-e35afca48ed0"). InnerVolumeSpecName "kube-api-access-4l8qr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:44:07.422720 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.422636    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/625a1ee6-2adf-4896-b781-e35afca48ed0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "625a1ee6-2adf-4896-b781-e35afca48ed0" (UID: "625a1ee6-2adf-4896-b781-e35afca48ed0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 22:44:07.422720 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.422658    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-dshm" (OuterVolumeSpecName: "dshm") pod "625a1ee6-2adf-4896-b781-e35afca48ed0" (UID: "625a1ee6-2adf-4896-b781-e35afca48ed0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:44:07.522501 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.522466    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4l8qr\" (UniqueName: \"kubernetes.io/projected/625a1ee6-2adf-4896-b781-e35afca48ed0-kube-api-access-4l8qr\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:44:07.522501 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.522494    2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/625a1ee6-2adf-4896-b781-e35afca48ed0-tls-certs\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:44:07.522501 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.522504    2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/625a1ee6-2adf-4896-b781-e35afca48ed0-dshm\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:44:07.618857 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.618819    2555 generic.go:356] "Generic (PLEG): container finished" podID="625a1ee6-2adf-4896-b781-e35afca48ed0" containerID="197018ee5aad0169b2a5cb3f968553612d9e96c584732a3b866bb0cd92c0efa2" exitCode=0
Jul 07 22:44:07.619037 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.618860    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"
Jul 07 22:44:07.619037 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.618865    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6" event={"ID":"625a1ee6-2adf-4896-b781-e35afca48ed0","Type":"ContainerDied","Data":"197018ee5aad0169b2a5cb3f968553612d9e96c584732a3b866bb0cd92c0efa2"}
Jul 07 22:44:07.619037 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.618931    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6" event={"ID":"625a1ee6-2adf-4896-b781-e35afca48ed0","Type":"ContainerDied","Data":"23fbb0af487758488e26a4a98e674cbacffd0459e38b51444d07cbdb28d14edd"}
Jul 07 22:44:07.619037 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.618953    2555 scope.go:117] "RemoveContainer" containerID="197018ee5aad0169b2a5cb3f968553612d9e96c584732a3b866bb0cd92c0efa2"
Jul 07 22:44:07.627644 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.627625    2555 scope.go:117] "RemoveContainer" containerID="197018ee5aad0169b2a5cb3f968553612d9e96c584732a3b866bb0cd92c0efa2"
Jul 07 22:44:07.628089 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:44:07.628068    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"197018ee5aad0169b2a5cb3f968553612d9e96c584732a3b866bb0cd92c0efa2\": container with ID starting with 197018ee5aad0169b2a5cb3f968553612d9e96c584732a3b866bb0cd92c0efa2 not found: ID does not exist" containerID="197018ee5aad0169b2a5cb3f968553612d9e96c584732a3b866bb0cd92c0efa2"
Jul 07 22:44:07.628209 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.628096    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"197018ee5aad0169b2a5cb3f968553612d9e96c584732a3b866bb0cd92c0efa2"} err="failed to get container status \"197018ee5aad0169b2a5cb3f968553612d9e96c584732a3b866bb0cd92c0efa2\": rpc error: code = NotFound desc = could not find container \"197018ee5aad0169b2a5cb3f968553612d9e96c584732a3b866bb0cd92c0efa2\": container with ID starting with 197018ee5aad0169b2a5cb3f968553612d9e96c584732a3b866bb0cd92c0efa2 not found: ID does not exist"
Jul 07 22:44:07.639009 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.638988    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"]
Jul 07 22:44:07.642147 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:07.642126    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-928393a0/e2e-lifecycle-model-928393a0-kserve-85f78c9b75-8vvr6"]
Jul 07 22:44:08.921967 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:08.921938    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="625a1ee6-2adf-4896-b781-e35afca48ed0" path="/var/lib/kubelet/pods/625a1ee6-2adf-4896-b781-e35afca48ed0/volumes"
Jul 07 22:44:34.713703 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:34.713664    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"]
Jul 07 22:44:34.714266 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:34.713940    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v" podUID="575be9b7-4dbf-4300-a7d6-c855483c8e40" containerName="istio-proxy" containerID="cri-o://902259de2ac7db931173ce705a59df913b559cb94b806f12cd9f625530648b7c" gracePeriod=30
Jul 07 22:44:38.055392 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:38.055359    2555 patch_prober.go:28] interesting pod/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 07 22:44:38.055846 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:38.055426    2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v" podUID="575be9b7-4dbf-4300-a7d6-c855483c8e40" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 22:44:39.945433 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:39.945409    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:44:40.112477 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.112389    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-envoy\") pod \"575be9b7-4dbf-4300-a7d6-c855483c8e40\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") "
Jul 07 22:44:40.112477 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.112431    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575be9b7-4dbf-4300-a7d6-c855483c8e40-istiod-ca-cert\") pod \"575be9b7-4dbf-4300-a7d6-c855483c8e40\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") "
Jul 07 22:44:40.112477 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.112462    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-data\") pod \"575be9b7-4dbf-4300-a7d6-c855483c8e40\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") "
Jul 07 22:44:40.112477 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.112480    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-workload-certs\") pod \"575be9b7-4dbf-4300-a7d6-c855483c8e40\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") "
Jul 07 22:44:40.112814 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.112498    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfg2x\" (UniqueName: \"kubernetes.io/projected/575be9b7-4dbf-4300-a7d6-c855483c8e40-kube-api-access-qfg2x\") pod \"575be9b7-4dbf-4300-a7d6-c855483c8e40\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") "
Jul 07 22:44:40.112814 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.112550    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-workload-socket\") pod \"575be9b7-4dbf-4300-a7d6-c855483c8e40\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") "
Jul 07 22:44:40.112814 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.112571    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-credential-socket\") pod \"575be9b7-4dbf-4300-a7d6-c855483c8e40\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") "
Jul 07 22:44:40.112814 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.112727    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-token\") pod \"575be9b7-4dbf-4300-a7d6-c855483c8e40\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") "
Jul 07 22:44:40.112814 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.112770    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-podinfo\") pod \"575be9b7-4dbf-4300-a7d6-c855483c8e40\" (UID: \"575be9b7-4dbf-4300-a7d6-c855483c8e40\") "
Jul 07 22:44:40.112814 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.112802    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "575be9b7-4dbf-4300-a7d6-c855483c8e40" (UID: "575be9b7-4dbf-4300-a7d6-c855483c8e40"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:44:40.113161 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.112907    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "575be9b7-4dbf-4300-a7d6-c855483c8e40" (UID: "575be9b7-4dbf-4300-a7d6-c855483c8e40"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:44:40.113161 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.112951    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/575be9b7-4dbf-4300-a7d6-c855483c8e40-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "575be9b7-4dbf-4300-a7d6-c855483c8e40" (UID: "575be9b7-4dbf-4300-a7d6-c855483c8e40"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 22:44:40.113161 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.113115    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "575be9b7-4dbf-4300-a7d6-c855483c8e40" (UID: "575be9b7-4dbf-4300-a7d6-c855483c8e40"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:44:40.113330 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.113226    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-workload-certs\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:44:40.113330 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.113247    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-workload-socket\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:44:40.113330 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.113262    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-credential-socket\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:44:40.113330 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.113276    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575be9b7-4dbf-4300-a7d6-c855483c8e40-istiod-ca-cert\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:44:40.113537 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.113417    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-data" (OuterVolumeSpecName: "istio-data") pod "575be9b7-4dbf-4300-a7d6-c855483c8e40" (UID: "575be9b7-4dbf-4300-a7d6-c855483c8e40"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:44:40.114823 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.114800    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-token" (OuterVolumeSpecName: "istio-token") pod "575be9b7-4dbf-4300-a7d6-c855483c8e40" (UID: "575be9b7-4dbf-4300-a7d6-c855483c8e40"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:44:40.114943 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.114825    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "575be9b7-4dbf-4300-a7d6-c855483c8e40" (UID: "575be9b7-4dbf-4300-a7d6-c855483c8e40"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 22:44:40.114943 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.114828    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "575be9b7-4dbf-4300-a7d6-c855483c8e40" (UID: "575be9b7-4dbf-4300-a7d6-c855483c8e40"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:44:40.114943 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.114851    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/575be9b7-4dbf-4300-a7d6-c855483c8e40-kube-api-access-qfg2x" (OuterVolumeSpecName: "kube-api-access-qfg2x") pod "575be9b7-4dbf-4300-a7d6-c855483c8e40" (UID: "575be9b7-4dbf-4300-a7d6-c855483c8e40"). InnerVolumeSpecName "kube-api-access-qfg2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:44:40.214253 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.214218    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-token\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:44:40.214253 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.214248    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-podinfo\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:44:40.214253 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.214261    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-envoy\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:44:40.214494 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.214272    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575be9b7-4dbf-4300-a7d6-c855483c8e40-istio-data\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:44:40.214494 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.214284    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfg2x\" (UniqueName: \"kubernetes.io/projected/575be9b7-4dbf-4300-a7d6-c855483c8e40-kube-api-access-qfg2x\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:44:40.770344 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.770303    2555 generic.go:356] "Generic (PLEG): container finished" podID="575be9b7-4dbf-4300-a7d6-c855483c8e40" containerID="902259de2ac7db931173ce705a59df913b559cb94b806f12cd9f625530648b7c" exitCode=0
Jul 07 22:44:40.770550 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.770354    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v" event={"ID":"575be9b7-4dbf-4300-a7d6-c855483c8e40","Type":"ContainerDied","Data":"902259de2ac7db931173ce705a59df913b559cb94b806f12cd9f625530648b7c"}
Jul 07 22:44:40.770550 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.770417    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v" event={"ID":"575be9b7-4dbf-4300-a7d6-c855483c8e40","Type":"ContainerDied","Data":"71f787c1a3b0bffd5224e72b0e2ca8ea071b7175d98c1dbc5fd990be88c703b5"}
Jul 07 22:44:40.770550 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.770460    2555 scope.go:117] "RemoveContainer" containerID="902259de2ac7db931173ce705a59df913b559cb94b806f12cd9f625530648b7c"
Jul 07 22:44:40.770705 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.770648    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"
Jul 07 22:44:40.783502 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.783476    2555 scope.go:117] "RemoveContainer" containerID="576a0a0d2b98c4a0c6d26ecdc39a17b380357d20aa517a61c4272eafcf040f8a"
Jul 07 22:44:40.793202 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.793173    2555 scope.go:117] "RemoveContainer" containerID="902259de2ac7db931173ce705a59df913b559cb94b806f12cd9f625530648b7c"
Jul 07 22:44:40.793515 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:44:40.793492    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"902259de2ac7db931173ce705a59df913b559cb94b806f12cd9f625530648b7c\": container with ID starting with 902259de2ac7db931173ce705a59df913b559cb94b806f12cd9f625530648b7c not found: ID does not exist" containerID="902259de2ac7db931173ce705a59df913b559cb94b806f12cd9f625530648b7c"
Jul 07 22:44:40.793616 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.793527    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"902259de2ac7db931173ce705a59df913b559cb94b806f12cd9f625530648b7c"} err="failed to get container status \"902259de2ac7db931173ce705a59df913b559cb94b806f12cd9f625530648b7c\": rpc error: code = NotFound desc = could not find container \"902259de2ac7db931173ce705a59df913b559cb94b806f12cd9f625530648b7c\": container with ID starting with 902259de2ac7db931173ce705a59df913b559cb94b806f12cd9f625530648b7c not found: ID does not exist"
Jul 07 22:44:40.793616 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.793550    2555 scope.go:117] "RemoveContainer" containerID="576a0a0d2b98c4a0c6d26ecdc39a17b380357d20aa517a61c4272eafcf040f8a"
Jul 07 22:44:40.793936 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:44:40.793904    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"576a0a0d2b98c4a0c6d26ecdc39a17b380357d20aa517a61c4272eafcf040f8a\": container with ID starting with 576a0a0d2b98c4a0c6d26ecdc39a17b380357d20aa517a61c4272eafcf040f8a not found: ID does not exist" containerID="576a0a0d2b98c4a0c6d26ecdc39a17b380357d20aa517a61c4272eafcf040f8a"
Jul 07 22:44:40.794053 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.793944    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"576a0a0d2b98c4a0c6d26ecdc39a17b380357d20aa517a61c4272eafcf040f8a"} err="failed to get container status \"576a0a0d2b98c4a0c6d26ecdc39a17b380357d20aa517a61c4272eafcf040f8a\": rpc error: code = NotFound desc = could not find container \"576a0a0d2b98c4a0c6d26ecdc39a17b380357d20aa517a61c4272eafcf040f8a\": container with ID starting with 576a0a0d2b98c4a0c6d26ecdc39a17b380357d20aa517a61c4272eafcf040f8a not found: ID does not exist"
Jul 07 22:44:40.798612 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.798586    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"]
Jul 07 22:44:40.803091 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.803066    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-928393a0-openshift-default-76f6598f79-cmh9v"]
Jul 07 22:44:40.922606 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:44:40.922577    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="575be9b7-4dbf-4300-a7d6-c855483c8e40" path="/var/lib/kubelet/pods/575be9b7-4dbf-4300-a7d6-c855483c8e40/volumes"
Jul 07 22:45:02.907964 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:02.907934    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:45:02.909516 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:02.909494    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:45:29.173732 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.173697    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"]
Jul 07 22:45:29.174229 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.174024    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="575be9b7-4dbf-4300-a7d6-c855483c8e40" containerName="istio-proxy"
Jul 07 22:45:29.174229 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.174036    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="575be9b7-4dbf-4300-a7d6-c855483c8e40" containerName="istio-proxy"
Jul 07 22:45:29.174229 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.174052    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e546a2b8-5867-4db6-b2db-fd7210e81f60" containerName="maas-api"
Jul 07 22:45:29.174229 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.174057    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e546a2b8-5867-4db6-b2db-fd7210e81f60" containerName="maas-api"
Jul 07 22:45:29.174229 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.174073    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="625a1ee6-2adf-4896-b781-e35afca48ed0" containerName="main"
Jul 07 22:45:29.174229 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.174080    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="625a1ee6-2adf-4896-b781-e35afca48ed0" containerName="main"
Jul 07 22:45:29.174229 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.174126    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="575be9b7-4dbf-4300-a7d6-c855483c8e40" containerName="istio-proxy"
Jul 07 22:45:29.174229 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.174136    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e546a2b8-5867-4db6-b2db-fd7210e81f60" containerName="maas-api"
Jul 07 22:45:29.174229 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.174142    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="625a1ee6-2adf-4896-b781-e35afca48ed0" containerName="main"
Jul 07 22:45:29.176031 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.176014    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.180016 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.179994    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 07 22:45:29.180127 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.180070    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-2de05d-openshift-default-dockercfg-cwjkh\""
Jul 07 22:45:29.212634 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.212604    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"]
Jul 07 22:45:29.335728 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.335692    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-istio-envoy\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.335929 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.335758    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f784e50a-1aab-4584-a1d3-0453630685b0-istio-podinfo\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.335929 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.335804    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f6qz\" (UniqueName: \"kubernetes.io/projected/f784e50a-1aab-4584-a1d3-0453630685b0-kube-api-access-9f6qz\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.335929 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.335836    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f784e50a-1aab-4584-a1d3-0453630685b0-istiod-ca-cert\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.335929 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.335897    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-workload-socket\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.335929 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.335929    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-credential-socket\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.336137 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.335945    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f784e50a-1aab-4584-a1d3-0453630685b0-istio-token\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.336137 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.335983    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-workload-certs\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.336137 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.336020    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-istio-data\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.436519 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.436434    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-workload-socket\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.436519 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.436477    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-credential-socket\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.436519 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.436493    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f784e50a-1aab-4584-a1d3-0453630685b0-istio-token\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.436519 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.436516    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-workload-certs\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.436865 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.436632    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-istio-data\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.436865 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.436756    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-istio-envoy\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.436865 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.436787    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f784e50a-1aab-4584-a1d3-0453630685b0-istio-podinfo\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.436865 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.436818    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9f6qz\" (UniqueName: \"kubernetes.io/projected/f784e50a-1aab-4584-a1d3-0453630685b0-kube-api-access-9f6qz\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.436865 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.436853    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f784e50a-1aab-4584-a1d3-0453630685b0-istiod-ca-cert\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.437178 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.436923    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-workload-certs\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.437178 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.437085    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-credential-socket\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.437305 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.437259    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-workload-socket\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.437305 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.437276    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-istio-data\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.437629 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.437609    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f784e50a-1aab-4584-a1d3-0453630685b0-istiod-ca-cert\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.439180 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.439155    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-istio-envoy\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.439443 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.439423    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f784e50a-1aab-4584-a1d3-0453630685b0-istio-podinfo\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.444320 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.444294    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f6qz\" (UniqueName: \"kubernetes.io/projected/f784e50a-1aab-4584-a1d3-0453630685b0-kube-api-access-9f6qz\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.444941 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.444922    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f784e50a-1aab-4584-a1d3-0453630685b0-istio-token\") pod \"e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") " pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.485512 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.485484    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:29.617892 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.617793    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"]
Jul 07 22:45:29.620140 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:45:29.620111    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf784e50a_1aab_4584_a1d3_0453630685b0.slice/crio-daff47c9ee5836c873f66fb414d6fb296da892f515d8f5637b17596aee61376b WatchSource:0}: Error finding container daff47c9ee5836c873f66fb414d6fb296da892f515d8f5637b17596aee61376b: Status 404 returned error can't find the container with id daff47c9ee5836c873f66fb414d6fb296da892f515d8f5637b17596aee61376b
Jul 07 22:45:29.622318 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.622287    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:45:29.622384 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.622355    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:45:29.622420 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.622384    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:45:29.988134 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.988098    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj" event={"ID":"f784e50a-1aab-4584-a1d3-0453630685b0","Type":"ContainerStarted","Data":"425d6abb207b6675c292d7838e00eb80261e66176550473eb801f86e996edf49"}
Jul 07 22:45:29.988302 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:29.988165    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj" event={"ID":"f784e50a-1aab-4584-a1d3-0453630685b0","Type":"ContainerStarted","Data":"daff47c9ee5836c873f66fb414d6fb296da892f515d8f5637b17596aee61376b"}
Jul 07 22:45:30.007068 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:30.007018    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj" podStartSLOduration=1.007003718 podStartE2EDuration="1.007003718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:45:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:45:30.005387828 +0000 UTC m=+1827.677606843" watchObservedRunningTime="2026-07-07 22:45:30.007003718 +0000 UTC m=+1827.679222707"
Jul 07 22:45:30.485689 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:30.485654    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:30.490456 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:30.490434    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:30.991907 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:30.991850    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:30.992918 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:30.992891    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:31.948889 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:31.948846    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"]
Jul 07 22:45:31.949378 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:31.949360    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="575be9b7-4dbf-4300-a7d6-c855483c8e40" containerName="istio-proxy"
Jul 07 22:45:31.949454 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:31.949382    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="575be9b7-4dbf-4300-a7d6-c855483c8e40" containerName="istio-proxy"
Jul 07 22:45:31.949508 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:31.949490    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="575be9b7-4dbf-4300-a7d6-c855483c8e40" containerName="istio-proxy"
Jul 07 22:45:31.951691 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:31.951670    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"
Jul 07 22:45:31.954317 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:31.954299    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-2de05d-serving-cert\""
Jul 07 22:45:31.954424 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:31.954342    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-r9rv4\""
Jul 07 22:45:31.961674 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:31.961651    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"]
Jul 07 22:45:32.064564 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:32.064533    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f9f6\" (UniqueName: \"kubernetes.io/projected/3a74b336-3249-4ca3-a614-04b64b84c6da-kube-api-access-7f9f6\") pod \"maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd\" (UID: \"3a74b336-3249-4ca3-a614-04b64b84c6da\") " pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"
Jul 07 22:45:32.064564 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:32.064568    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a74b336-3249-4ca3-a614-04b64b84c6da-maas-api-tls\") pod \"maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd\" (UID: \"3a74b336-3249-4ca3-a614-04b64b84c6da\") " pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"
Jul 07 22:45:32.064788 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:32.064696    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a74b336-3249-4ca3-a614-04b64b84c6da-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd\" (UID: \"3a74b336-3249-4ca3-a614-04b64b84c6da\") " pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"
Jul 07 22:45:32.165813 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:32.165780    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a74b336-3249-4ca3-a614-04b64b84c6da-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd\" (UID: \"3a74b336-3249-4ca3-a614-04b64b84c6da\") " pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"
Jul 07 22:45:32.166006 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:32.165983    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f9f6\" (UniqueName: \"kubernetes.io/projected/3a74b336-3249-4ca3-a614-04b64b84c6da-kube-api-access-7f9f6\") pod \"maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd\" (UID: \"3a74b336-3249-4ca3-a614-04b64b84c6da\") " pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"
Jul 07 22:45:32.166104 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:32.166026    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a74b336-3249-4ca3-a614-04b64b84c6da-maas-api-tls\") pod \"maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd\" (UID: \"3a74b336-3249-4ca3-a614-04b64b84c6da\") " pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"
Jul 07 22:45:32.166230 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:45:32.166211    2555 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-2de05d-serving-cert: secret "maas-api-e2e-inf-a-2de05d-serving-cert" not found
Jul 07 22:45:32.166289 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:45:32.166279    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a74b336-3249-4ca3-a614-04b64b84c6da-maas-api-tls podName:3a74b336-3249-4ca3-a614-04b64b84c6da nodeName:}" failed. No retries permitted until 2026-07-07 22:45:32.666259412 +0000 UTC m=+1830.338478384 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3a74b336-3249-4ca3-a614-04b64b84c6da-maas-api-tls") pod "maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd" (UID: "3a74b336-3249-4ca3-a614-04b64b84c6da") : secret "maas-api-e2e-inf-a-2de05d-serving-cert" not found
Jul 07 22:45:32.166541 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:32.166523    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a74b336-3249-4ca3-a614-04b64b84c6da-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd\" (UID: \"3a74b336-3249-4ca3-a614-04b64b84c6da\") " pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"
Jul 07 22:45:32.176462 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:32.176435    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f9f6\" (UniqueName: \"kubernetes.io/projected/3a74b336-3249-4ca3-a614-04b64b84c6da-kube-api-access-7f9f6\") pod \"maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd\" (UID: \"3a74b336-3249-4ca3-a614-04b64b84c6da\") " pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"
Jul 07 22:45:32.670532 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:32.670494    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a74b336-3249-4ca3-a614-04b64b84c6da-maas-api-tls\") pod \"maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd\" (UID: \"3a74b336-3249-4ca3-a614-04b64b84c6da\") " pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"
Jul 07 22:45:32.673011 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:32.672987    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a74b336-3249-4ca3-a614-04b64b84c6da-maas-api-tls\") pod \"maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd\" (UID: \"3a74b336-3249-4ca3-a614-04b64b84c6da\") " pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"
Jul 07 22:45:32.864078 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:32.864041    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"
Jul 07 22:45:32.993270 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:32.993239    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"]
Jul 07 22:45:32.996764 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:45:32.996735    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a74b336_3249_4ca3_a614_04b64b84c6da.slice/crio-5d8792c59fcbd8dfe5edfa66a6dc00ea549fa0d4338f93a686e9f96d069e3134 WatchSource:0}: Error finding container 5d8792c59fcbd8dfe5edfa66a6dc00ea549fa0d4338f93a686e9f96d069e3134: Status 404 returned error can't find the container with id 5d8792c59fcbd8dfe5edfa66a6dc00ea549fa0d4338f93a686e9f96d069e3134
Jul 07 22:45:32.998374 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:32.998344    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 22:45:34.006604 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:34.006571    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd" event={"ID":"3a74b336-3249-4ca3-a614-04b64b84c6da","Type":"ContainerStarted","Data":"0b7e71dd9fc6fa8bebd1a496736f2872b4227acd58801538aa04b108a8cd6265"}
Jul 07 22:45:34.007001 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:34.006647    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd" event={"ID":"3a74b336-3249-4ca3-a614-04b64b84c6da","Type":"ContainerStarted","Data":"5d8792c59fcbd8dfe5edfa66a6dc00ea549fa0d4338f93a686e9f96d069e3134"}
Jul 07 22:45:34.007001 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:34.006685    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"
Jul 07 22:45:34.023566 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:34.023527    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd" podStartSLOduration=2.645638454 podStartE2EDuration="3.023512825s" totalImagesPullingTime="377.874371ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:45:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:45:34.021682759 +0000 UTC m=+1831.693901761" watchObservedRunningTime="2026-07-07 22:45:34.023512825 +0000 UTC m=+1831.695731812"
Jul 07 22:45:39.998463 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:39.998428    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"]
Jul 07 22:45:40.000817 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:40.000795    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"
Jul 07 22:45:40.003574 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:40.003546    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-346e61-serving-cert\""
Jul 07 22:45:40.011696 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:40.011668    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"]
Jul 07 22:45:40.017734 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:40.017713    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"
Jul 07 22:45:40.137954 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:40.137917    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7fa08bb-aa24-4882-b5b6-323ca6523ed0-openshift-service-ca\") pod \"maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89\" (UID: \"e7fa08bb-aa24-4882-b5b6-323ca6523ed0\") " pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"
Jul 07 22:45:40.138143 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:40.137986    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7fa08bb-aa24-4882-b5b6-323ca6523ed0-maas-api-tls\") pod \"maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89\" (UID: \"e7fa08bb-aa24-4882-b5b6-323ca6523ed0\") " pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"
Jul 07 22:45:40.138207 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:40.138179    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lt28\" (UniqueName: \"kubernetes.io/projected/e7fa08bb-aa24-4882-b5b6-323ca6523ed0-kube-api-access-7lt28\") pod \"maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89\" (UID: \"e7fa08bb-aa24-4882-b5b6-323ca6523ed0\") " pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"
Jul 07 22:45:40.239485 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:40.239430    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7fa08bb-aa24-4882-b5b6-323ca6523ed0-maas-api-tls\") pod \"maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89\" (UID: \"e7fa08bb-aa24-4882-b5b6-323ca6523ed0\") " pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"
Jul 07 22:45:40.239681 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:40.239551    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lt28\" (UniqueName: \"kubernetes.io/projected/e7fa08bb-aa24-4882-b5b6-323ca6523ed0-kube-api-access-7lt28\") pod \"maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89\" (UID: \"e7fa08bb-aa24-4882-b5b6-323ca6523ed0\") " pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"
Jul 07 22:45:40.239681 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:45:40.239582    2555 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-346e61-serving-cert: secret "maas-api-e2e-inf-b-346e61-serving-cert" not found
Jul 07 22:45:40.239681 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:40.239618    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7fa08bb-aa24-4882-b5b6-323ca6523ed0-openshift-service-ca\") pod \"maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89\" (UID: \"e7fa08bb-aa24-4882-b5b6-323ca6523ed0\") " pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"
Jul 07 22:45:40.239681 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:45:40.239647    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7fa08bb-aa24-4882-b5b6-323ca6523ed0-maas-api-tls podName:e7fa08bb-aa24-4882-b5b6-323ca6523ed0 nodeName:}" failed. No retries permitted until 2026-07-07 22:45:40.739630241 +0000 UTC m=+1838.411849207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e7fa08bb-aa24-4882-b5b6-323ca6523ed0-maas-api-tls") pod "maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89" (UID: "e7fa08bb-aa24-4882-b5b6-323ca6523ed0") : secret "maas-api-e2e-inf-b-346e61-serving-cert" not found
Jul 07 22:45:40.240356 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:40.240332    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7fa08bb-aa24-4882-b5b6-323ca6523ed0-openshift-service-ca\") pod \"maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89\" (UID: \"e7fa08bb-aa24-4882-b5b6-323ca6523ed0\") " pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"
Jul 07 22:45:40.249600 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:40.249527    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lt28\" (UniqueName: \"kubernetes.io/projected/e7fa08bb-aa24-4882-b5b6-323ca6523ed0-kube-api-access-7lt28\") pod \"maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89\" (UID: \"e7fa08bb-aa24-4882-b5b6-323ca6523ed0\") " pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"
Jul 07 22:45:40.743919 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:40.743850    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7fa08bb-aa24-4882-b5b6-323ca6523ed0-maas-api-tls\") pod \"maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89\" (UID: \"e7fa08bb-aa24-4882-b5b6-323ca6523ed0\") " pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"
Jul 07 22:45:40.746279 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:40.746255    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7fa08bb-aa24-4882-b5b6-323ca6523ed0-maas-api-tls\") pod \"maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89\" (UID: \"e7fa08bb-aa24-4882-b5b6-323ca6523ed0\") " pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"
Jul 07 22:45:40.911825 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:40.911789    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"
Jul 07 22:45:41.040105 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:41.040059    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"]
Jul 07 22:45:41.042434 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:45:41.042407    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7fa08bb_aa24_4882_b5b6_323ca6523ed0.slice/crio-a73d05a787bec2e7967b9cf1baeee553989ffcc02db7cba162f7ebfad4db8b21 WatchSource:0}: Error finding container a73d05a787bec2e7967b9cf1baeee553989ffcc02db7cba162f7ebfad4db8b21: Status 404 returned error can't find the container with id a73d05a787bec2e7967b9cf1baeee553989ffcc02db7cba162f7ebfad4db8b21
Jul 07 22:45:42.049430 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:42.049397    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89" event={"ID":"e7fa08bb-aa24-4882-b5b6-323ca6523ed0","Type":"ContainerStarted","Data":"36766c6d0135341d3f2439f863690471a44a4c94c301ef59c294a4539e9f5e8d"}
Jul 07 22:45:42.049839 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:42.049452    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89" event={"ID":"e7fa08bb-aa24-4882-b5b6-323ca6523ed0","Type":"ContainerStarted","Data":"a73d05a787bec2e7967b9cf1baeee553989ffcc02db7cba162f7ebfad4db8b21"}
Jul 07 22:45:42.049839 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:42.049553    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"
Jul 07 22:45:42.067716 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:42.067651    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89" podStartSLOduration=2.699270484 podStartE2EDuration="3.067628785s" totalImagesPullingTime="368.358301ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:45:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:45:42.064811459 +0000 UTC m=+1839.737030483" watchObservedRunningTime="2026-07-07 22:45:42.067628785 +0000 UTC m=+1839.739847773"
Jul 07 22:45:47.034083 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:47.034036    2555 patch_prober.go:28] interesting pod/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 07 22:45:47.034693 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:47.034127    2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj" podUID="f784e50a-1aab-4584-a1d3-0453630685b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 07 22:45:48.058979 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:48.058941    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"
Jul 07 22:45:49.083850 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:49.083816    2555 generic.go:356] "Generic (PLEG): container finished" podID="f784e50a-1aab-4584-a1d3-0453630685b0" containerID="425d6abb207b6675c292d7838e00eb80261e66176550473eb801f86e996edf49" exitCode=0
Jul 07 22:45:49.084241 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:49.083893    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj" event={"ID":"f784e50a-1aab-4584-a1d3-0453630685b0","Type":"ContainerDied","Data":"425d6abb207b6675c292d7838e00eb80261e66176550473eb801f86e996edf49"}
Jul 07 22:45:49.084284 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:49.084251    2555 scope.go:117] "RemoveContainer" containerID="425d6abb207b6675c292d7838e00eb80261e66176550473eb801f86e996edf49"
Jul 07 22:45:49.085205 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:49.085177    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:45:49.085260 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:49.085247    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:45:49.085310 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:49.085297    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:45:50.089849 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:50.089817    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj" event={"ID":"f784e50a-1aab-4584-a1d3-0453630685b0","Type":"ContainerStarted","Data":"1b52506b7ee29c1b8fe129e361ecbb96829de5eb0c9268b699f19032c5c2fe2a"}
Jul 07 22:45:50.486500 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:50.486457    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:50.486730 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:50.486708    2555 patch_prober.go:28] interesting pod/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 07 22:45:50.486786 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:50.486761    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj" podUID="f784e50a-1aab-4584-a1d3-0453630685b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 07 22:45:51.486456 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:51.486410    2555 patch_prober.go:28] interesting pod/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 07 22:45:51.486894 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:51.486498    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj" podUID="f784e50a-1aab-4584-a1d3-0453630685b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 07 22:45:52.486224 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:52.486180    2555 patch_prober.go:28] interesting pod/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 07 22:45:52.486432 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:52.486252    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj" podUID="f784e50a-1aab-4584-a1d3-0453630685b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 07 22:45:53.490253 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:53.490218    2555 patch_prober.go:28] interesting pod/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 07 22:45:53.490736 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:53.490539    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj" podUID="f784e50a-1aab-4584-a1d3-0453630685b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 07 22:45:54.489395 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:54.489360    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:54.489638 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:54.489616    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:45:54.490199 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:45:54.490184    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:46:07.983430 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:07.983396    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"]
Jul 07 22:46:07.986051 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:07.986028    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:07.988530 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:07.988507    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-346e61\"/\"openshift-service-ca.crt\""
Jul 07 22:46:07.989416 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:07.989395    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-346e61\"/\"test-model-346e61-kserve-self-signed-certs\""
Jul 07 22:46:07.989510 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:07.989382    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-346e61\"/\"kube-root-ca.crt\""
Jul 07 22:46:07.989510 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:07.989402    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-346e61\"/\"default-dockercfg-rmxx4\""
Jul 07 22:46:07.996993 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:07.996972    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"]
Jul 07 22:46:08.098499 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:08.098471    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-tmp-dir\") pod \"test-model-346e61-kserve-68b748c4cf-cjfpj\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") " pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:08.098499 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:08.098502    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thkpn\" (UniqueName: \"kubernetes.io/projected/4c09c0cd-b979-4272-9c2f-19b7965b7fab-kube-api-access-thkpn\") pod \"test-model-346e61-kserve-68b748c4cf-cjfpj\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") " pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:08.098712 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:08.098524    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-home\") pod \"test-model-346e61-kserve-68b748c4cf-cjfpj\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") " pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:08.098712 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:08.098541    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-dshm\") pod \"test-model-346e61-kserve-68b748c4cf-cjfpj\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") " pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:08.098712 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:08.098613    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-model-cache\") pod \"test-model-346e61-kserve-68b748c4cf-cjfpj\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") " pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:08.098712 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:08.098661    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c09c0cd-b979-4272-9c2f-19b7965b7fab-tls-certs\") pod \"test-model-346e61-kserve-68b748c4cf-cjfpj\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") " pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:08.199313 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:08.199278    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-tmp-dir\") pod \"test-model-346e61-kserve-68b748c4cf-cjfpj\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") " pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:08.199313 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:08.199313    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thkpn\" (UniqueName: \"kubernetes.io/projected/4c09c0cd-b979-4272-9c2f-19b7965b7fab-kube-api-access-thkpn\") pod \"test-model-346e61-kserve-68b748c4cf-cjfpj\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") " pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:08.199538 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:08.199333    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-home\") pod \"test-model-346e61-kserve-68b748c4cf-cjfpj\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") " pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:08.199538 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:08.199346    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-dshm\") pod \"test-model-346e61-kserve-68b748c4cf-cjfpj\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") " pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:08.199538 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:08.199380    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-model-cache\") pod \"test-model-346e61-kserve-68b748c4cf-cjfpj\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") " pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:08.199538 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:08.199419    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c09c0cd-b979-4272-9c2f-19b7965b7fab-tls-certs\") pod \"test-model-346e61-kserve-68b748c4cf-cjfpj\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") " pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:08.199717 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:08.199664    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-tmp-dir\") pod \"test-model-346e61-kserve-68b748c4cf-cjfpj\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") " pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:08.199827 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:08.199804    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-home\") pod \"test-model-346e61-kserve-68b748c4cf-cjfpj\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") " pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:08.199928 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:08.199845    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-model-cache\") pod \"test-model-346e61-kserve-68b748c4cf-cjfpj\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") " pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:08.201501 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:08.201480    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-dshm\") pod \"test-model-346e61-kserve-68b748c4cf-cjfpj\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") " pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:08.201798 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:08.201783    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c09c0cd-b979-4272-9c2f-19b7965b7fab-tls-certs\") pod \"test-model-346e61-kserve-68b748c4cf-cjfpj\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") " pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:08.206444 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:08.206424    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thkpn\" (UniqueName: \"kubernetes.io/projected/4c09c0cd-b979-4272-9c2f-19b7965b7fab-kube-api-access-thkpn\") pod \"test-model-346e61-kserve-68b748c4cf-cjfpj\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") " pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:08.295345 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:08.295254    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:08.421236 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:08.421201    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"]
Jul 07 22:46:08.421897 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:46:08.421860    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c09c0cd_b979_4272_9c2f_19b7965b7fab.slice/crio-f69d174f5a295eb40e366c1b410589b59a424c3ebbe2919752a6452b41be2a0a WatchSource:0}: Error finding container f69d174f5a295eb40e366c1b410589b59a424c3ebbe2919752a6452b41be2a0a: Status 404 returned error can't find the container with id f69d174f5a295eb40e366c1b410589b59a424c3ebbe2919752a6452b41be2a0a
Jul 07 22:46:09.177907 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:09.177856    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj" event={"ID":"4c09c0cd-b979-4272-9c2f-19b7965b7fab","Type":"ContainerStarted","Data":"e22991907c0cfb813cb37f35794be7fa36a349d7ea0fe5d178e6477d3f5dfc04"}
Jul 07 22:46:09.177907 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:09.177914    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj" event={"ID":"4c09c0cd-b979-4272-9c2f-19b7965b7fab","Type":"ContainerStarted","Data":"f69d174f5a295eb40e366c1b410589b59a424c3ebbe2919752a6452b41be2a0a"}
Jul 07 22:46:09.193842 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:09.193769    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj" podStartSLOduration=2.193753897 podStartE2EDuration="2.193753897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:46:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:46:09.193538386 +0000 UTC m=+1866.865757397" watchObservedRunningTime="2026-07-07 22:46:09.193753897 +0000 UTC m=+1866.865972884"
Jul 07 22:46:18.296234 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:18.296197    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:18.296234 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:18.296239    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:18.308507 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:18.308480    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:46:19.230340 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:46:19.230314    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:47:00.560544 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:00.560511    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"]
Jul 07 22:47:01.350286 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:01.350251    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"]
Jul 07 22:47:01.350574 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:01.350533    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd" podUID="3a74b336-3249-4ca3-a614-04b64b84c6da" containerName="maas-api" containerID="cri-o://0b7e71dd9fc6fa8bebd1a496736f2872b4227acd58801538aa04b108a8cd6265" gracePeriod=30
Jul 07 22:47:01.592384 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:01.592355    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"
Jul 07 22:47:01.694208 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:01.694124    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a74b336-3249-4ca3-a614-04b64b84c6da-maas-api-tls\") pod \"3a74b336-3249-4ca3-a614-04b64b84c6da\" (UID: \"3a74b336-3249-4ca3-a614-04b64b84c6da\") "
Jul 07 22:47:01.694208 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:01.694162    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a74b336-3249-4ca3-a614-04b64b84c6da-openshift-service-ca\") pod \"3a74b336-3249-4ca3-a614-04b64b84c6da\" (UID: \"3a74b336-3249-4ca3-a614-04b64b84c6da\") "
Jul 07 22:47:01.694208 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:01.694198    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7f9f6\" (UniqueName: \"kubernetes.io/projected/3a74b336-3249-4ca3-a614-04b64b84c6da-kube-api-access-7f9f6\") pod \"3a74b336-3249-4ca3-a614-04b64b84c6da\" (UID: \"3a74b336-3249-4ca3-a614-04b64b84c6da\") "
Jul 07 22:47:01.694605 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:01.694575    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a74b336-3249-4ca3-a614-04b64b84c6da-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3a74b336-3249-4ca3-a614-04b64b84c6da" (UID: "3a74b336-3249-4ca3-a614-04b64b84c6da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 22:47:01.696273 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:01.696251    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a74b336-3249-4ca3-a614-04b64b84c6da-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3a74b336-3249-4ca3-a614-04b64b84c6da" (UID: "3a74b336-3249-4ca3-a614-04b64b84c6da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 22:47:01.696375 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:01.696289    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a74b336-3249-4ca3-a614-04b64b84c6da-kube-api-access-7f9f6" (OuterVolumeSpecName: "kube-api-access-7f9f6") pod "3a74b336-3249-4ca3-a614-04b64b84c6da" (UID: "3a74b336-3249-4ca3-a614-04b64b84c6da"). InnerVolumeSpecName "kube-api-access-7f9f6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:47:01.795805 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:01.795758    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a74b336-3249-4ca3-a614-04b64b84c6da-maas-api-tls\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:47:01.795805 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:01.795798    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a74b336-3249-4ca3-a614-04b64b84c6da-openshift-service-ca\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:47:01.795805 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:01.795807    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7f9f6\" (UniqueName: \"kubernetes.io/projected/3a74b336-3249-4ca3-a614-04b64b84c6da-kube-api-access-7f9f6\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:47:02.417602 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:02.417563    2555 generic.go:356] "Generic (PLEG): container finished" podID="3a74b336-3249-4ca3-a614-04b64b84c6da" containerID="0b7e71dd9fc6fa8bebd1a496736f2872b4227acd58801538aa04b108a8cd6265" exitCode=0
Jul 07 22:47:02.417602 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:02.417606    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"
Jul 07 22:47:02.417830 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:02.417649    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd" event={"ID":"3a74b336-3249-4ca3-a614-04b64b84c6da","Type":"ContainerDied","Data":"0b7e71dd9fc6fa8bebd1a496736f2872b4227acd58801538aa04b108a8cd6265"}
Jul 07 22:47:02.417830 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:02.417699    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd" event={"ID":"3a74b336-3249-4ca3-a614-04b64b84c6da","Type":"ContainerDied","Data":"5d8792c59fcbd8dfe5edfa66a6dc00ea549fa0d4338f93a686e9f96d069e3134"}
Jul 07 22:47:02.417830 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:02.417721    2555 scope.go:117] "RemoveContainer" containerID="0b7e71dd9fc6fa8bebd1a496736f2872b4227acd58801538aa04b108a8cd6265"
Jul 07 22:47:02.426603 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:02.426586    2555 scope.go:117] "RemoveContainer" containerID="0b7e71dd9fc6fa8bebd1a496736f2872b4227acd58801538aa04b108a8cd6265"
Jul 07 22:47:02.426856 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:47:02.426838    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b7e71dd9fc6fa8bebd1a496736f2872b4227acd58801538aa04b108a8cd6265\": container with ID starting with 0b7e71dd9fc6fa8bebd1a496736f2872b4227acd58801538aa04b108a8cd6265 not found: ID does not exist" containerID="0b7e71dd9fc6fa8bebd1a496736f2872b4227acd58801538aa04b108a8cd6265"
Jul 07 22:47:02.426943 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:02.426883    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b7e71dd9fc6fa8bebd1a496736f2872b4227acd58801538aa04b108a8cd6265"} err="failed to get container status \"0b7e71dd9fc6fa8bebd1a496736f2872b4227acd58801538aa04b108a8cd6265\": rpc error: code = NotFound desc = could not find container \"0b7e71dd9fc6fa8bebd1a496736f2872b4227acd58801538aa04b108a8cd6265\": container with ID starting with 0b7e71dd9fc6fa8bebd1a496736f2872b4227acd58801538aa04b108a8cd6265 not found: ID does not exist"
Jul 07 22:47:02.437763 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:02.437733    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"]
Jul 07 22:47:02.441344 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:02.441321    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-2de05d-787c74fdb8-4nfzd"]
Jul 07 22:47:02.922169 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:02.922134    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a74b336-3249-4ca3-a614-04b64b84c6da" path="/var/lib/kubelet/pods/3a74b336-3249-4ca3-a614-04b64b84c6da/volumes"
Jul 07 22:47:15.582986 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:15.582944    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj" podUID="4c09c0cd-b979-4272-9c2f-19b7965b7fab" containerName="main" containerID="cri-o://e22991907c0cfb813cb37f35794be7fa36a349d7ea0fe5d178e6477d3f5dfc04" gracePeriod=45
Jul 07 22:47:15.819030 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:15.819005    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:47:15.923248 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:15.923160    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thkpn\" (UniqueName: \"kubernetes.io/projected/4c09c0cd-b979-4272-9c2f-19b7965b7fab-kube-api-access-thkpn\") pod \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") "
Jul 07 22:47:15.923248 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:15.923214    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-model-cache\") pod \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") "
Jul 07 22:47:15.923455 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:15.923274    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-tmp-dir\") pod \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") "
Jul 07 22:47:15.923455 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:15.923290    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c09c0cd-b979-4272-9c2f-19b7965b7fab-tls-certs\") pod \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") "
Jul 07 22:47:15.923455 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:15.923321    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-home\") pod \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") "
Jul 07 22:47:15.923455 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:15.923380    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-dshm\") pod \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\" (UID: \"4c09c0cd-b979-4272-9c2f-19b7965b7fab\") "
Jul 07 22:47:15.923665 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:15.923546    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4c09c0cd-b979-4272-9c2f-19b7965b7fab" (UID: "4c09c0cd-b979-4272-9c2f-19b7965b7fab"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:47:15.923665 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:15.923558    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-model-cache" (OuterVolumeSpecName: "model-cache") pod "4c09c0cd-b979-4272-9c2f-19b7965b7fab" (UID: "4c09c0cd-b979-4272-9c2f-19b7965b7fab"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:47:15.923665 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:15.923644    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-home" (OuterVolumeSpecName: "home") pod "4c09c0cd-b979-4272-9c2f-19b7965b7fab" (UID: "4c09c0cd-b979-4272-9c2f-19b7965b7fab"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:47:15.923831 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:15.923694    2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-tmp-dir\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:47:15.923831 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:15.923708    2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-model-cache\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:47:15.925454 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:15.925430    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-dshm" (OuterVolumeSpecName: "dshm") pod "4c09c0cd-b979-4272-9c2f-19b7965b7fab" (UID: "4c09c0cd-b979-4272-9c2f-19b7965b7fab"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:47:15.925537 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:15.925442    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c09c0cd-b979-4272-9c2f-19b7965b7fab-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4c09c0cd-b979-4272-9c2f-19b7965b7fab" (UID: "4c09c0cd-b979-4272-9c2f-19b7965b7fab"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 22:47:15.925537 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:15.925469    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c09c0cd-b979-4272-9c2f-19b7965b7fab-kube-api-access-thkpn" (OuterVolumeSpecName: "kube-api-access-thkpn") pod "4c09c0cd-b979-4272-9c2f-19b7965b7fab" (UID: "4c09c0cd-b979-4272-9c2f-19b7965b7fab"). InnerVolumeSpecName "kube-api-access-thkpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:47:16.024752 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:16.024720    2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c09c0cd-b979-4272-9c2f-19b7965b7fab-tls-certs\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:47:16.024752 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:16.024747    2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-home\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:47:16.024752 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:16.024755    2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c09c0cd-b979-4272-9c2f-19b7965b7fab-dshm\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:47:16.025020 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:16.024765    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thkpn\" (UniqueName: \"kubernetes.io/projected/4c09c0cd-b979-4272-9c2f-19b7965b7fab-kube-api-access-thkpn\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:47:16.479798 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:16.479757    2555 generic.go:356] "Generic (PLEG): container finished" podID="4c09c0cd-b979-4272-9c2f-19b7965b7fab" containerID="e22991907c0cfb813cb37f35794be7fa36a349d7ea0fe5d178e6477d3f5dfc04" exitCode=0
Jul 07 22:47:16.480021 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:16.479805    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"
Jul 07 22:47:16.480021 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:16.479847    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj" event={"ID":"4c09c0cd-b979-4272-9c2f-19b7965b7fab","Type":"ContainerDied","Data":"e22991907c0cfb813cb37f35794be7fa36a349d7ea0fe5d178e6477d3f5dfc04"}
Jul 07 22:47:16.480021 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:16.479913    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj" event={"ID":"4c09c0cd-b979-4272-9c2f-19b7965b7fab","Type":"ContainerDied","Data":"f69d174f5a295eb40e366c1b410589b59a424c3ebbe2919752a6452b41be2a0a"}
Jul 07 22:47:16.480021 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:16.479937    2555 scope.go:117] "RemoveContainer" containerID="e22991907c0cfb813cb37f35794be7fa36a349d7ea0fe5d178e6477d3f5dfc04"
Jul 07 22:47:16.488673 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:16.488655    2555 scope.go:117] "RemoveContainer" containerID="e22991907c0cfb813cb37f35794be7fa36a349d7ea0fe5d178e6477d3f5dfc04"
Jul 07 22:47:16.488925 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:47:16.488907    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e22991907c0cfb813cb37f35794be7fa36a349d7ea0fe5d178e6477d3f5dfc04\": container with ID starting with e22991907c0cfb813cb37f35794be7fa36a349d7ea0fe5d178e6477d3f5dfc04 not found: ID does not exist" containerID="e22991907c0cfb813cb37f35794be7fa36a349d7ea0fe5d178e6477d3f5dfc04"
Jul 07 22:47:16.488976 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:16.488935    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e22991907c0cfb813cb37f35794be7fa36a349d7ea0fe5d178e6477d3f5dfc04"} err="failed to get container status \"e22991907c0cfb813cb37f35794be7fa36a349d7ea0fe5d178e6477d3f5dfc04\": rpc error: code = NotFound desc = could not find container \"e22991907c0cfb813cb37f35794be7fa36a349d7ea0fe5d178e6477d3f5dfc04\": container with ID starting with e22991907c0cfb813cb37f35794be7fa36a349d7ea0fe5d178e6477d3f5dfc04 not found: ID does not exist"
Jul 07 22:47:16.500089 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:16.500070    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"]
Jul 07 22:47:16.503672 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:16.503655    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-346e61/test-model-346e61-kserve-68b748c4cf-cjfpj"]
Jul 07 22:47:16.922945 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:16.922837    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c09c0cd-b979-4272-9c2f-19b7965b7fab" path="/var/lib/kubelet/pods/4c09c0cd-b979-4272-9c2f-19b7965b7fab/volumes"
Jul 07 22:47:43.064962 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:43.064928    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"]
Jul 07 22:47:43.065438 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:43.065215    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj" podUID="f784e50a-1aab-4584-a1d3-0453630685b0" containerName="istio-proxy" containerID="cri-o://1b52506b7ee29c1b8fe129e361ecbb96829de5eb0c9268b699f19032c5c2fe2a" gracePeriod=30
Jul 07 22:47:44.267780 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:44.267740    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"]
Jul 07 22:47:44.268222 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:44.268018    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89" podUID="e7fa08bb-aa24-4882-b5b6-323ca6523ed0" containerName="maas-api" containerID="cri-o://36766c6d0135341d3f2439f863690471a44a4c94c301ef59c294a4539e9f5e8d" gracePeriod=30
Jul 07 22:47:44.520779 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:44.520700    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"
Jul 07 22:47:44.604758 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:44.604726    2555 generic.go:356] "Generic (PLEG): container finished" podID="e7fa08bb-aa24-4882-b5b6-323ca6523ed0" containerID="36766c6d0135341d3f2439f863690471a44a4c94c301ef59c294a4539e9f5e8d" exitCode=0
Jul 07 22:47:44.604954 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:44.604780    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"
Jul 07 22:47:44.604954 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:44.604826    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89" event={"ID":"e7fa08bb-aa24-4882-b5b6-323ca6523ed0","Type":"ContainerDied","Data":"36766c6d0135341d3f2439f863690471a44a4c94c301ef59c294a4539e9f5e8d"}
Jul 07 22:47:44.604954 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:44.604914    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89" event={"ID":"e7fa08bb-aa24-4882-b5b6-323ca6523ed0","Type":"ContainerDied","Data":"a73d05a787bec2e7967b9cf1baeee553989ffcc02db7cba162f7ebfad4db8b21"}
Jul 07 22:47:44.604954 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:44.604945    2555 scope.go:117] "RemoveContainer" containerID="36766c6d0135341d3f2439f863690471a44a4c94c301ef59c294a4539e9f5e8d"
Jul 07 22:47:44.613841 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:44.613825    2555 scope.go:117] "RemoveContainer" containerID="36766c6d0135341d3f2439f863690471a44a4c94c301ef59c294a4539e9f5e8d"
Jul 07 22:47:44.614143 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:47:44.614126    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36766c6d0135341d3f2439f863690471a44a4c94c301ef59c294a4539e9f5e8d\": container with ID starting with 36766c6d0135341d3f2439f863690471a44a4c94c301ef59c294a4539e9f5e8d not found: ID does not exist" containerID="36766c6d0135341d3f2439f863690471a44a4c94c301ef59c294a4539e9f5e8d"
Jul 07 22:47:44.614193 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:44.614153    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36766c6d0135341d3f2439f863690471a44a4c94c301ef59c294a4539e9f5e8d"} err="failed to get container status \"36766c6d0135341d3f2439f863690471a44a4c94c301ef59c294a4539e9f5e8d\": rpc error: code = NotFound desc = could not find container \"36766c6d0135341d3f2439f863690471a44a4c94c301ef59c294a4539e9f5e8d\": container with ID starting with 36766c6d0135341d3f2439f863690471a44a4c94c301ef59c294a4539e9f5e8d not found: ID does not exist"
Jul 07 22:47:44.674605 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:44.674566    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7fa08bb-aa24-4882-b5b6-323ca6523ed0-openshift-service-ca\") pod \"e7fa08bb-aa24-4882-b5b6-323ca6523ed0\" (UID: \"e7fa08bb-aa24-4882-b5b6-323ca6523ed0\") "
Jul 07 22:47:44.674807 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:44.674614    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lt28\" (UniqueName: \"kubernetes.io/projected/e7fa08bb-aa24-4882-b5b6-323ca6523ed0-kube-api-access-7lt28\") pod \"e7fa08bb-aa24-4882-b5b6-323ca6523ed0\" (UID: \"e7fa08bb-aa24-4882-b5b6-323ca6523ed0\") "
Jul 07 22:47:44.674894 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:44.674809    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7fa08bb-aa24-4882-b5b6-323ca6523ed0-maas-api-tls\") pod \"e7fa08bb-aa24-4882-b5b6-323ca6523ed0\" (UID: \"e7fa08bb-aa24-4882-b5b6-323ca6523ed0\") "
Jul 07 22:47:44.675045 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:44.675016    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7fa08bb-aa24-4882-b5b6-323ca6523ed0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e7fa08bb-aa24-4882-b5b6-323ca6523ed0" (UID: "e7fa08bb-aa24-4882-b5b6-323ca6523ed0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 22:47:44.675206 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:44.675187    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7fa08bb-aa24-4882-b5b6-323ca6523ed0-openshift-service-ca\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:47:44.676789 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:44.676765    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7fa08bb-aa24-4882-b5b6-323ca6523ed0-kube-api-access-7lt28" (OuterVolumeSpecName: "kube-api-access-7lt28") pod "e7fa08bb-aa24-4882-b5b6-323ca6523ed0" (UID: "e7fa08bb-aa24-4882-b5b6-323ca6523ed0"). InnerVolumeSpecName "kube-api-access-7lt28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:47:44.676912 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:44.676818    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7fa08bb-aa24-4882-b5b6-323ca6523ed0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e7fa08bb-aa24-4882-b5b6-323ca6523ed0" (UID: "e7fa08bb-aa24-4882-b5b6-323ca6523ed0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 22:47:44.775724 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:44.775637    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7fa08bb-aa24-4882-b5b6-323ca6523ed0-maas-api-tls\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:47:44.775724 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:44.775667    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lt28\" (UniqueName: \"kubernetes.io/projected/e7fa08bb-aa24-4882-b5b6-323ca6523ed0-kube-api-access-7lt28\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:47:44.926363 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:44.926329    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"]
Jul 07 22:47:44.931886 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:44.931841    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-346e61-c5bbfdb66-dww89"]
Jul 07 22:47:46.923212 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:46.923179    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7fa08bb-aa24-4882-b5b6-323ca6523ed0" path="/var/lib/kubelet/pods/e7fa08bb-aa24-4882-b5b6-323ca6523ed0/volumes"
Jul 07 22:47:48.313631 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.313603    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:47:48.407948 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.407850    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-credential-socket\") pod \"f784e50a-1aab-4584-a1d3-0453630685b0\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") "
Jul 07 22:47:48.408126 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.407950    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-istio-envoy\") pod \"f784e50a-1aab-4584-a1d3-0453630685b0\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") "
Jul 07 22:47:48.408126 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.407977    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f784e50a-1aab-4584-a1d3-0453630685b0-istio-podinfo\") pod \"f784e50a-1aab-4584-a1d3-0453630685b0\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") "
Jul 07 22:47:48.408126 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.407998    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9f6qz\" (UniqueName: \"kubernetes.io/projected/f784e50a-1aab-4584-a1d3-0453630685b0-kube-api-access-9f6qz\") pod \"f784e50a-1aab-4584-a1d3-0453630685b0\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") "
Jul 07 22:47:48.408126 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.408020    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f784e50a-1aab-4584-a1d3-0453630685b0-istiod-ca-cert\") pod \"f784e50a-1aab-4584-a1d3-0453630685b0\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") "
Jul 07 22:47:48.408126 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.408049    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-istio-data\") pod \"f784e50a-1aab-4584-a1d3-0453630685b0\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") "
Jul 07 22:47:48.408126 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.408076    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-workload-socket\") pod \"f784e50a-1aab-4584-a1d3-0453630685b0\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") "
Jul 07 22:47:48.408430 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.408138    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-workload-certs\") pod \"f784e50a-1aab-4584-a1d3-0453630685b0\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") "
Jul 07 22:47:48.408430 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.408153    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f784e50a-1aab-4584-a1d3-0453630685b0" (UID: "f784e50a-1aab-4584-a1d3-0453630685b0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:47:48.408430 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.408255    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f784e50a-1aab-4584-a1d3-0453630685b0-istio-token\") pod \"f784e50a-1aab-4584-a1d3-0453630685b0\" (UID: \"f784e50a-1aab-4584-a1d3-0453630685b0\") "
Jul 07 22:47:48.408430 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.408373    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f784e50a-1aab-4584-a1d3-0453630685b0" (UID: "f784e50a-1aab-4584-a1d3-0453630685b0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:47:48.408645 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.408531    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f784e50a-1aab-4584-a1d3-0453630685b0" (UID: "f784e50a-1aab-4584-a1d3-0453630685b0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:47:48.408645 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.408546    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f784e50a-1aab-4584-a1d3-0453630685b0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f784e50a-1aab-4584-a1d3-0453630685b0" (UID: "f784e50a-1aab-4584-a1d3-0453630685b0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 22:47:48.408645 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.408600    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-credential-socket\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:47:48.408645 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.408620    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-workload-socket\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:47:48.409300 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.409268    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-istio-data" (OuterVolumeSpecName: "istio-data") pod "f784e50a-1aab-4584-a1d3-0453630685b0" (UID: "f784e50a-1aab-4584-a1d3-0453630685b0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:47:48.410248 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.410223    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f784e50a-1aab-4584-a1d3-0453630685b0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f784e50a-1aab-4584-a1d3-0453630685b0" (UID: "f784e50a-1aab-4584-a1d3-0453630685b0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 22:47:48.410248 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.410234    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f784e50a-1aab-4584-a1d3-0453630685b0" (UID: "f784e50a-1aab-4584-a1d3-0453630685b0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:47:48.410404 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.410286    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f784e50a-1aab-4584-a1d3-0453630685b0-kube-api-access-9f6qz" (OuterVolumeSpecName: "kube-api-access-9f6qz") pod "f784e50a-1aab-4584-a1d3-0453630685b0" (UID: "f784e50a-1aab-4584-a1d3-0453630685b0"). InnerVolumeSpecName "kube-api-access-9f6qz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:47:48.410404 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.410306    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f784e50a-1aab-4584-a1d3-0453630685b0-istio-token" (OuterVolumeSpecName: "istio-token") pod "f784e50a-1aab-4584-a1d3-0453630685b0" (UID: "f784e50a-1aab-4584-a1d3-0453630685b0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:47:48.509584 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.509551    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f784e50a-1aab-4584-a1d3-0453630685b0-istio-token\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:47:48.509584 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.509580    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-istio-envoy\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:47:48.509584 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.509590    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f784e50a-1aab-4584-a1d3-0453630685b0-istio-podinfo\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:47:48.509910 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.509599    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9f6qz\" (UniqueName: \"kubernetes.io/projected/f784e50a-1aab-4584-a1d3-0453630685b0-kube-api-access-9f6qz\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:47:48.509910 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.509607    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f784e50a-1aab-4584-a1d3-0453630685b0-istiod-ca-cert\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:47:48.509910 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.509616    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-istio-data\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:47:48.509910 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.509626    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f784e50a-1aab-4584-a1d3-0453630685b0-workload-certs\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:47:48.625211 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.625177    2555 generic.go:356] "Generic (PLEG): container finished" podID="f784e50a-1aab-4584-a1d3-0453630685b0" containerID="1b52506b7ee29c1b8fe129e361ecbb96829de5eb0c9268b699f19032c5c2fe2a" exitCode=0
Jul 07 22:47:48.625374 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.625228    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"
Jul 07 22:47:48.625374 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.625250    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj" event={"ID":"f784e50a-1aab-4584-a1d3-0453630685b0","Type":"ContainerDied","Data":"1b52506b7ee29c1b8fe129e361ecbb96829de5eb0c9268b699f19032c5c2fe2a"}
Jul 07 22:47:48.625374 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.625310    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj" event={"ID":"f784e50a-1aab-4584-a1d3-0453630685b0","Type":"ContainerDied","Data":"daff47c9ee5836c873f66fb414d6fb296da892f515d8f5637b17596aee61376b"}
Jul 07 22:47:48.625374 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.625331    2555 scope.go:117] "RemoveContainer" containerID="1b52506b7ee29c1b8fe129e361ecbb96829de5eb0c9268b699f19032c5c2fe2a"
Jul 07 22:47:48.639726 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.639705    2555 scope.go:117] "RemoveContainer" containerID="425d6abb207b6675c292d7838e00eb80261e66176550473eb801f86e996edf49"
Jul 07 22:47:48.647690 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.647669    2555 scope.go:117] "RemoveContainer" containerID="1b52506b7ee29c1b8fe129e361ecbb96829de5eb0c9268b699f19032c5c2fe2a"
Jul 07 22:47:48.647995 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:47:48.647973    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b52506b7ee29c1b8fe129e361ecbb96829de5eb0c9268b699f19032c5c2fe2a\": container with ID starting with 1b52506b7ee29c1b8fe129e361ecbb96829de5eb0c9268b699f19032c5c2fe2a not found: ID does not exist" containerID="1b52506b7ee29c1b8fe129e361ecbb96829de5eb0c9268b699f19032c5c2fe2a"
Jul 07 22:47:48.648096 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.648001    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b52506b7ee29c1b8fe129e361ecbb96829de5eb0c9268b699f19032c5c2fe2a"} err="failed to get container status \"1b52506b7ee29c1b8fe129e361ecbb96829de5eb0c9268b699f19032c5c2fe2a\": rpc error: code = NotFound desc = could not find container \"1b52506b7ee29c1b8fe129e361ecbb96829de5eb0c9268b699f19032c5c2fe2a\": container with ID starting with 1b52506b7ee29c1b8fe129e361ecbb96829de5eb0c9268b699f19032c5c2fe2a not found: ID does not exist"
Jul 07 22:47:48.648096 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.648018    2555 scope.go:117] "RemoveContainer" containerID="425d6abb207b6675c292d7838e00eb80261e66176550473eb801f86e996edf49"
Jul 07 22:47:48.648301 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:47:48.648280    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"425d6abb207b6675c292d7838e00eb80261e66176550473eb801f86e996edf49\": container with ID starting with 425d6abb207b6675c292d7838e00eb80261e66176550473eb801f86e996edf49 not found: ID does not exist" containerID="425d6abb207b6675c292d7838e00eb80261e66176550473eb801f86e996edf49"
Jul 07 22:47:48.648342 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.648307    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"425d6abb207b6675c292d7838e00eb80261e66176550473eb801f86e996edf49"} err="failed to get container status \"425d6abb207b6675c292d7838e00eb80261e66176550473eb801f86e996edf49\": rpc error: code = NotFound desc = could not find container \"425d6abb207b6675c292d7838e00eb80261e66176550473eb801f86e996edf49\": container with ID starting with 425d6abb207b6675c292d7838e00eb80261e66176550473eb801f86e996edf49 not found: ID does not exist"
Jul 07 22:47:48.652624 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.652602    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"]
Jul 07 22:47:48.655556 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.655537    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-2de05d-openshift-default-6f54669769-qlhxj"]
Jul 07 22:47:48.922252 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:48.922217    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f784e50a-1aab-4584-a1d3-0453630685b0" path="/var/lib/kubelet/pods/f784e50a-1aab-4584-a1d3-0453630685b0/volumes"
Jul 07 22:47:51.405202 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.405175    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"]
Jul 07 22:47:51.405632 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.405614    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a74b336-3249-4ca3-a614-04b64b84c6da" containerName="maas-api"
Jul 07 22:47:51.405679 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.405637    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a74b336-3249-4ca3-a614-04b64b84c6da" containerName="maas-api"
Jul 07 22:47:51.405679 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.405668    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7fa08bb-aa24-4882-b5b6-323ca6523ed0" containerName="maas-api"
Jul 07 22:47:51.405679 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.405679    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7fa08bb-aa24-4882-b5b6-323ca6523ed0" containerName="maas-api"
Jul 07 22:47:51.405770 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.405701    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f784e50a-1aab-4584-a1d3-0453630685b0" containerName="istio-proxy"
Jul 07 22:47:51.405770 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.405710    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f784e50a-1aab-4584-a1d3-0453630685b0" containerName="istio-proxy"
Jul 07 22:47:51.405770 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.405723    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f784e50a-1aab-4584-a1d3-0453630685b0" containerName="istio-proxy"
Jul 07 22:47:51.405770 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.405728    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f784e50a-1aab-4584-a1d3-0453630685b0" containerName="istio-proxy"
Jul 07 22:47:51.405770 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.405734    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c09c0cd-b979-4272-9c2f-19b7965b7fab" containerName="main"
Jul 07 22:47:51.405770 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.405739    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c09c0cd-b979-4272-9c2f-19b7965b7fab" containerName="main"
Jul 07 22:47:51.405975 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.405792    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f784e50a-1aab-4584-a1d3-0453630685b0" containerName="istio-proxy"
Jul 07 22:47:51.405975 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.405800    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7fa08bb-aa24-4882-b5b6-323ca6523ed0" containerName="maas-api"
Jul 07 22:47:51.405975 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.405807    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a74b336-3249-4ca3-a614-04b64b84c6da" containerName="maas-api"
Jul 07 22:47:51.405975 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.405814    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c09c0cd-b979-4272-9c2f-19b7965b7fab" containerName="main"
Jul 07 22:47:51.410259 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.410239    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.412765 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.412747    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-fb6979-openshift-default-dockercfg-nrbb2\""
Jul 07 22:47:51.412999 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.412986    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 07 22:47:51.420429 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.420406    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"]
Jul 07 22:47:51.537994 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.537957    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-workload-socket\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.537994 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.537994    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-credential-socket\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.538248 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.538074    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a94c7289-eba2-4cfe-ab11-09b32703a947-istiod-ca-cert\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.538248 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.538101    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-data\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.538248 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.538156    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-token\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.538248 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.538195    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzljw\" (UniqueName: \"kubernetes.io/projected/a94c7289-eba2-4cfe-ab11-09b32703a947-kube-api-access-xzljw\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.538248 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.538240    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-envoy\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.538463 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.538265    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-podinfo\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.538463 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.538363    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-workload-certs\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.639276 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.639243    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-envoy\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.639276 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.639276    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-podinfo\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.639538 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.639310    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-workload-certs\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.639538 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.639340    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-workload-socket\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.639538 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.639364    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-credential-socket\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.639538 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.639445    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a94c7289-eba2-4cfe-ab11-09b32703a947-istiod-ca-cert\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.639538 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.639469    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-data\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.639812 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.639575    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-token\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.639812 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.639600    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzljw\" (UniqueName: \"kubernetes.io/projected/a94c7289-eba2-4cfe-ab11-09b32703a947-kube-api-access-xzljw\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.639812 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.639778    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-workload-certs\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.639982 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.639831    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-workload-socket\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.640076 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.640051    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-data\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.640233 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.640210    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-credential-socket\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.640339 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.640321    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a94c7289-eba2-4cfe-ab11-09b32703a947-istiod-ca-cert\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.641626 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.641597    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-envoy\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.641830 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.641812    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-podinfo\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.649720 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.649700    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzljw\" (UniqueName: \"kubernetes.io/projected/a94c7289-eba2-4cfe-ab11-09b32703a947-kube-api-access-xzljw\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.668640 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.668586    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-token\") pod \"e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") " pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.721291 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.721265    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:51.857915 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.857886    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"]
Jul 07 22:47:51.860063 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:47:51.860037    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda94c7289_eba2_4cfe_ab11_09b32703a947.slice/crio-ae934a084c09b6b6d800ca7c5c1fbf1996867c81f5a00c6064bf04215b7b8c0b WatchSource:0}: Error finding container ae934a084c09b6b6d800ca7c5c1fbf1996867c81f5a00c6064bf04215b7b8c0b: Status 404 returned error can't find the container with id ae934a084c09b6b6d800ca7c5c1fbf1996867c81f5a00c6064bf04215b7b8c0b
Jul 07 22:47:51.862112 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.862083    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:47:51.862170 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.862142    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:47:51.862211 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:51.862175    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:47:52.648146 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:52.648109    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4" event={"ID":"a94c7289-eba2-4cfe-ab11-09b32703a947","Type":"ContainerStarted","Data":"c5ec4efff08cae1f5b89ecddbebeb8326798b8da17e7636f9964027948118348"}
Jul 07 22:47:52.648597 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:52.648168    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4" event={"ID":"a94c7289-eba2-4cfe-ab11-09b32703a947","Type":"ContainerStarted","Data":"ae934a084c09b6b6d800ca7c5c1fbf1996867c81f5a00c6064bf04215b7b8c0b"}
Jul 07 22:47:52.667290 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:52.667243    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4" podStartSLOduration=1.6672279190000001 podStartE2EDuration="1.667227919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:47:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:47:52.666117882 +0000 UTC m=+1970.338336883" watchObservedRunningTime="2026-07-07 22:47:52.667227919 +0000 UTC m=+1970.339446908"
Jul 07 22:47:52.722163 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:52.722131    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:52.727127 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:52.727102    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:53.652839 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:53.652806    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:53.653679 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:53.653662    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:47:54.692271 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:54.692194    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"]
Jul 07 22:47:54.692808 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:54.692788    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f784e50a-1aab-4584-a1d3-0453630685b0" containerName="istio-proxy"
Jul 07 22:47:54.696330 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:54.696306    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"
Jul 07 22:47:54.698612 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:54.698593    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-fb6979-serving-cert\""
Jul 07 22:47:54.698790 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:54.698770    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-r9rv4\""
Jul 07 22:47:54.705617 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:54.705593    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"]
Jul 07 22:47:54.868347 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:54.868317    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd44df46-2143-44ec-9ae5-1d157cebea3a-maas-api-tls\") pod \"maas-api-e2e-api-a-fb6979-7989444994-ctckv\" (UID: \"bd44df46-2143-44ec-9ae5-1d157cebea3a\") " pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"
Jul 07 22:47:54.868347 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:54.868350    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd44df46-2143-44ec-9ae5-1d157cebea3a-openshift-service-ca\") pod \"maas-api-e2e-api-a-fb6979-7989444994-ctckv\" (UID: \"bd44df46-2143-44ec-9ae5-1d157cebea3a\") " pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"
Jul 07 22:47:54.868543 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:54.868370    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r7gx\" (UniqueName: \"kubernetes.io/projected/bd44df46-2143-44ec-9ae5-1d157cebea3a-kube-api-access-8r7gx\") pod \"maas-api-e2e-api-a-fb6979-7989444994-ctckv\" (UID: \"bd44df46-2143-44ec-9ae5-1d157cebea3a\") " pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"
Jul 07 22:47:54.969496 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:54.969454    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd44df46-2143-44ec-9ae5-1d157cebea3a-maas-api-tls\") pod \"maas-api-e2e-api-a-fb6979-7989444994-ctckv\" (UID: \"bd44df46-2143-44ec-9ae5-1d157cebea3a\") " pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"
Jul 07 22:47:54.969496 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:54.969500    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd44df46-2143-44ec-9ae5-1d157cebea3a-openshift-service-ca\") pod \"maas-api-e2e-api-a-fb6979-7989444994-ctckv\" (UID: \"bd44df46-2143-44ec-9ae5-1d157cebea3a\") " pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"
Jul 07 22:47:54.969712 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:54.969517    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r7gx\" (UniqueName: \"kubernetes.io/projected/bd44df46-2143-44ec-9ae5-1d157cebea3a-kube-api-access-8r7gx\") pod \"maas-api-e2e-api-a-fb6979-7989444994-ctckv\" (UID: \"bd44df46-2143-44ec-9ae5-1d157cebea3a\") " pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"
Jul 07 22:47:54.969712 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:47:54.969683    2555 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-fb6979-serving-cert: secret "maas-api-e2e-api-a-fb6979-serving-cert" not found
Jul 07 22:47:54.969836 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:47:54.969762    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd44df46-2143-44ec-9ae5-1d157cebea3a-maas-api-tls podName:bd44df46-2143-44ec-9ae5-1d157cebea3a nodeName:}" failed. No retries permitted until 2026-07-07 22:47:55.469739291 +0000 UTC m=+1973.141958277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bd44df46-2143-44ec-9ae5-1d157cebea3a-maas-api-tls") pod "maas-api-e2e-api-a-fb6979-7989444994-ctckv" (UID: "bd44df46-2143-44ec-9ae5-1d157cebea3a") : secret "maas-api-e2e-api-a-fb6979-serving-cert" not found
Jul 07 22:47:54.970161 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:54.970142    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd44df46-2143-44ec-9ae5-1d157cebea3a-openshift-service-ca\") pod \"maas-api-e2e-api-a-fb6979-7989444994-ctckv\" (UID: \"bd44df46-2143-44ec-9ae5-1d157cebea3a\") " pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"
Jul 07 22:47:54.978502 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:54.978470    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r7gx\" (UniqueName: \"kubernetes.io/projected/bd44df46-2143-44ec-9ae5-1d157cebea3a-kube-api-access-8r7gx\") pod \"maas-api-e2e-api-a-fb6979-7989444994-ctckv\" (UID: \"bd44df46-2143-44ec-9ae5-1d157cebea3a\") " pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"
Jul 07 22:47:55.474361 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:55.474329    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd44df46-2143-44ec-9ae5-1d157cebea3a-maas-api-tls\") pod \"maas-api-e2e-api-a-fb6979-7989444994-ctckv\" (UID: \"bd44df46-2143-44ec-9ae5-1d157cebea3a\") " pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"
Jul 07 22:47:55.476643 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:55.476621    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd44df46-2143-44ec-9ae5-1d157cebea3a-maas-api-tls\") pod \"maas-api-e2e-api-a-fb6979-7989444994-ctckv\" (UID: \"bd44df46-2143-44ec-9ae5-1d157cebea3a\") " pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"
Jul 07 22:47:55.607735 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:55.607700    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"
Jul 07 22:47:55.733640 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:55.733615    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"]
Jul 07 22:47:55.735739 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:47:55.735714    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd44df46_2143_44ec_9ae5_1d157cebea3a.slice/crio-129a9f3ffc20ca798dd55f261ff4e43e0226e5b9ed353292665be515030668d0 WatchSource:0}: Error finding container 129a9f3ffc20ca798dd55f261ff4e43e0226e5b9ed353292665be515030668d0: Status 404 returned error can't find the container with id 129a9f3ffc20ca798dd55f261ff4e43e0226e5b9ed353292665be515030668d0
Jul 07 22:47:56.669454 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:56.669424    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv" event={"ID":"bd44df46-2143-44ec-9ae5-1d157cebea3a","Type":"ContainerStarted","Data":"8844069544d301e5b29b31490bc07053fb23c48c981500e98a7008a1af368e7f"}
Jul 07 22:47:56.669629 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:56.669472    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv" event={"ID":"bd44df46-2143-44ec-9ae5-1d157cebea3a","Type":"ContainerStarted","Data":"129a9f3ffc20ca798dd55f261ff4e43e0226e5b9ed353292665be515030668d0"}
Jul 07 22:47:56.669629 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:56.669562    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"
Jul 07 22:47:56.686735 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:47:56.686690    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv" podStartSLOduration=2.335915372 podStartE2EDuration="2.686670837s" totalImagesPullingTime="350.755465ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:47:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:47:56.685259173 +0000 UTC m=+1974.357478165" watchObservedRunningTime="2026-07-07 22:47:56.686670837 +0000 UTC m=+1974.358889824"
Jul 07 22:48:02.688310 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:02.688275    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"
Jul 07 22:48:02.805138 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:02.805100    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"]
Jul 07 22:48:02.822650 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:02.822609    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"]
Jul 07 22:48:02.822824 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:02.822706    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"
Jul 07 22:48:02.825504 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:02.825480    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-17949f-serving-cert\""
Jul 07 22:48:02.849340 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:02.849304    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/798de976-4991-4768-99c9-636b6968e4c0-maas-api-tls\") pod \"maas-api-e2e-api-b-17949f-99d78c86b-c8jc7\" (UID: \"798de976-4991-4768-99c9-636b6968e4c0\") " pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"
Jul 07 22:48:02.849465 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:02.849345    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/798de976-4991-4768-99c9-636b6968e4c0-openshift-service-ca\") pod \"maas-api-e2e-api-b-17949f-99d78c86b-c8jc7\" (UID: \"798de976-4991-4768-99c9-636b6968e4c0\") " pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"
Jul 07 22:48:02.849465 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:02.849428    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwnhf\" (UniqueName: \"kubernetes.io/projected/798de976-4991-4768-99c9-636b6968e4c0-kube-api-access-zwnhf\") pod \"maas-api-e2e-api-b-17949f-99d78c86b-c8jc7\" (UID: \"798de976-4991-4768-99c9-636b6968e4c0\") " pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"
Jul 07 22:48:02.950264 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:02.950179    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwnhf\" (UniqueName: \"kubernetes.io/projected/798de976-4991-4768-99c9-636b6968e4c0-kube-api-access-zwnhf\") pod \"maas-api-e2e-api-b-17949f-99d78c86b-c8jc7\" (UID: \"798de976-4991-4768-99c9-636b6968e4c0\") " pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"
Jul 07 22:48:02.950264 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:02.950245    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/798de976-4991-4768-99c9-636b6968e4c0-maas-api-tls\") pod \"maas-api-e2e-api-b-17949f-99d78c86b-c8jc7\" (UID: \"798de976-4991-4768-99c9-636b6968e4c0\") " pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"
Jul 07 22:48:02.950264 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:02.950264    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/798de976-4991-4768-99c9-636b6968e4c0-openshift-service-ca\") pod \"maas-api-e2e-api-b-17949f-99d78c86b-c8jc7\" (UID: \"798de976-4991-4768-99c9-636b6968e4c0\") " pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"
Jul 07 22:48:02.950498 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:48:02.950384    2555 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-17949f-serving-cert: secret "maas-api-e2e-api-b-17949f-serving-cert" not found
Jul 07 22:48:02.950498 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:48:02.950476    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/798de976-4991-4768-99c9-636b6968e4c0-maas-api-tls podName:798de976-4991-4768-99c9-636b6968e4c0 nodeName:}" failed. No retries permitted until 2026-07-07 22:48:03.450456643 +0000 UTC m=+1981.122675610 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/798de976-4991-4768-99c9-636b6968e4c0-maas-api-tls") pod "maas-api-e2e-api-b-17949f-99d78c86b-c8jc7" (UID: "798de976-4991-4768-99c9-636b6968e4c0") : secret "maas-api-e2e-api-b-17949f-serving-cert" not found
Jul 07 22:48:02.950809 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:02.950792    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/798de976-4991-4768-99c9-636b6968e4c0-openshift-service-ca\") pod \"maas-api-e2e-api-b-17949f-99d78c86b-c8jc7\" (UID: \"798de976-4991-4768-99c9-636b6968e4c0\") " pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"
Jul 07 22:48:02.958568 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:02.958534    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwnhf\" (UniqueName: \"kubernetes.io/projected/798de976-4991-4768-99c9-636b6968e4c0-kube-api-access-zwnhf\") pod \"maas-api-e2e-api-b-17949f-99d78c86b-c8jc7\" (UID: \"798de976-4991-4768-99c9-636b6968e4c0\") " pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"
Jul 07 22:48:03.455288 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:03.455256    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/798de976-4991-4768-99c9-636b6968e4c0-maas-api-tls\") pod \"maas-api-e2e-api-b-17949f-99d78c86b-c8jc7\" (UID: \"798de976-4991-4768-99c9-636b6968e4c0\") " pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"
Jul 07 22:48:03.457631 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:03.457612    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/798de976-4991-4768-99c9-636b6968e4c0-maas-api-tls\") pod \"maas-api-e2e-api-b-17949f-99d78c86b-c8jc7\" (UID: \"798de976-4991-4768-99c9-636b6968e4c0\") " pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"
Jul 07 22:48:03.732675 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:03.732582    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"
Jul 07 22:48:03.924297 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:03.924270    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"]
Jul 07 22:48:03.925905 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:48:03.925864    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod798de976_4991_4768_99c9_636b6968e4c0.slice/crio-0386d18f509acaf201d0e62671a360e14c12a0430f053a29b08c6a71b9ffe314 WatchSource:0}: Error finding container 0386d18f509acaf201d0e62671a360e14c12a0430f053a29b08c6a71b9ffe314: Status 404 returned error can't find the container with id 0386d18f509acaf201d0e62671a360e14c12a0430f053a29b08c6a71b9ffe314
Jul 07 22:48:04.708377 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:04.708341    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7" event={"ID":"798de976-4991-4768-99c9-636b6968e4c0","Type":"ContainerStarted","Data":"d8be31bec3803a0cd02d34095821bc52678cc26ee49bc1461c4580c4a3e2634a"}
Jul 07 22:48:04.708577 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:04.708399    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7" event={"ID":"798de976-4991-4768-99c9-636b6968e4c0","Type":"ContainerStarted","Data":"0386d18f509acaf201d0e62671a360e14c12a0430f053a29b08c6a71b9ffe314"}
Jul 07 22:48:04.708577 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:04.708493    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"
Jul 07 22:48:04.723932 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:04.723859    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7" podStartSLOduration=2.298285641 podStartE2EDuration="2.723840806s" totalImagesPullingTime="425.555165ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:48:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:48:04.723180022 +0000 UTC m=+1982.395399031" watchObservedRunningTime="2026-07-07 22:48:04.723840806 +0000 UTC m=+1982.396059796"
Jul 07 22:48:08.654166 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:08.654069    2555 patch_prober.go:28] interesting pod/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 07 22:48:08.654166 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:08.654135    2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4" podUID="a94c7289-eba2-4cfe-ab11-09b32703a947" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 07 22:48:08.734187 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:08.734146    2555 generic.go:356] "Generic (PLEG): container finished" podID="a94c7289-eba2-4cfe-ab11-09b32703a947" containerID="c5ec4efff08cae1f5b89ecddbebeb8326798b8da17e7636f9964027948118348" exitCode=0
Jul 07 22:48:08.734355 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:08.734217    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4" event={"ID":"a94c7289-eba2-4cfe-ab11-09b32703a947","Type":"ContainerDied","Data":"c5ec4efff08cae1f5b89ecddbebeb8326798b8da17e7636f9964027948118348"}
Jul 07 22:48:08.734595 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:08.734582    2555 scope.go:117] "RemoveContainer" containerID="c5ec4efff08cae1f5b89ecddbebeb8326798b8da17e7636f9964027948118348"
Jul 07 22:48:08.735490 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:08.735463    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:48:08.735580 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:08.735526    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:48:08.735580 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:08.735559    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:48:09.740808 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:09.740773    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4" event={"ID":"a94c7289-eba2-4cfe-ab11-09b32703a947","Type":"ContainerStarted","Data":"898c28fb19c7215325e7cbba35a37b7c90539bb277da9c9f2e7521bed5cb0acc"}
Jul 07 22:48:10.719202 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:10.719175    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"
Jul 07 22:48:10.721650 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:10.721633    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:48:10.721789 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:10.721769    2555 patch_prober.go:28] interesting pod/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 07 22:48:10.721835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:10.721817    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4" podUID="a94c7289-eba2-4cfe-ab11-09b32703a947" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 07 22:48:11.722542 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:11.722504    2555 patch_prober.go:28] interesting pod/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 07 22:48:11.722952 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:11.722559    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4" podUID="a94c7289-eba2-4cfe-ab11-09b32703a947" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 07 22:48:12.722356 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:12.722318    2555 patch_prober.go:28] interesting pod/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 07 22:48:12.722537 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:12.722384    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4" podUID="a94c7289-eba2-4cfe-ab11-09b32703a947" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 07 22:48:13.721777 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:13.721736    2555 patch_prober.go:28] interesting pod/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 07 22:48:13.722181 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:13.721789    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4" podUID="a94c7289-eba2-4cfe-ab11-09b32703a947" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 07 22:48:14.726112 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:14.726080    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:48:14.726565 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:14.726543    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:48:14.727074 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:14.727058    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:48:16.064923 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.064861    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"]
Jul 07 22:48:16.065408 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.065134    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv" podUID="bd44df46-2143-44ec-9ae5-1d157cebea3a" containerName="maas-api" containerID="cri-o://8844069544d301e5b29b31490bc07053fb23c48c981500e98a7008a1af368e7f" gracePeriod=30
Jul 07 22:48:16.300744 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.300719    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"
Jul 07 22:48:16.416993 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.416907    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8r7gx\" (UniqueName: \"kubernetes.io/projected/bd44df46-2143-44ec-9ae5-1d157cebea3a-kube-api-access-8r7gx\") pod \"bd44df46-2143-44ec-9ae5-1d157cebea3a\" (UID: \"bd44df46-2143-44ec-9ae5-1d157cebea3a\") "
Jul 07 22:48:16.416993 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.416968    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd44df46-2143-44ec-9ae5-1d157cebea3a-maas-api-tls\") pod \"bd44df46-2143-44ec-9ae5-1d157cebea3a\" (UID: \"bd44df46-2143-44ec-9ae5-1d157cebea3a\") "
Jul 07 22:48:16.417184 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.417030    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd44df46-2143-44ec-9ae5-1d157cebea3a-openshift-service-ca\") pod \"bd44df46-2143-44ec-9ae5-1d157cebea3a\" (UID: \"bd44df46-2143-44ec-9ae5-1d157cebea3a\") "
Jul 07 22:48:16.417411 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.417389    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd44df46-2143-44ec-9ae5-1d157cebea3a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bd44df46-2143-44ec-9ae5-1d157cebea3a" (UID: "bd44df46-2143-44ec-9ae5-1d157cebea3a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 22:48:16.419054 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.419032    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd44df46-2143-44ec-9ae5-1d157cebea3a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bd44df46-2143-44ec-9ae5-1d157cebea3a" (UID: "bd44df46-2143-44ec-9ae5-1d157cebea3a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 22:48:16.419161 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.419073    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd44df46-2143-44ec-9ae5-1d157cebea3a-kube-api-access-8r7gx" (OuterVolumeSpecName: "kube-api-access-8r7gx") pod "bd44df46-2143-44ec-9ae5-1d157cebea3a" (UID: "bd44df46-2143-44ec-9ae5-1d157cebea3a"). InnerVolumeSpecName "kube-api-access-8r7gx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:48:16.518316 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.518276    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd44df46-2143-44ec-9ae5-1d157cebea3a-openshift-service-ca\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:48:16.518316 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.518310    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8r7gx\" (UniqueName: \"kubernetes.io/projected/bd44df46-2143-44ec-9ae5-1d157cebea3a-kube-api-access-8r7gx\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:48:16.518316 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.518322    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd44df46-2143-44ec-9ae5-1d157cebea3a-maas-api-tls\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:48:16.774989 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.774956    2555 generic.go:356] "Generic (PLEG): container finished" podID="bd44df46-2143-44ec-9ae5-1d157cebea3a" containerID="8844069544d301e5b29b31490bc07053fb23c48c981500e98a7008a1af368e7f" exitCode=0
Jul 07 22:48:16.775163 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.775001    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"
Jul 07 22:48:16.775163 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.775015    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv" event={"ID":"bd44df46-2143-44ec-9ae5-1d157cebea3a","Type":"ContainerDied","Data":"8844069544d301e5b29b31490bc07053fb23c48c981500e98a7008a1af368e7f"}
Jul 07 22:48:16.775163 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.775057    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv" event={"ID":"bd44df46-2143-44ec-9ae5-1d157cebea3a","Type":"ContainerDied","Data":"129a9f3ffc20ca798dd55f261ff4e43e0226e5b9ed353292665be515030668d0"}
Jul 07 22:48:16.775163 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.775080    2555 scope.go:117] "RemoveContainer" containerID="8844069544d301e5b29b31490bc07053fb23c48c981500e98a7008a1af368e7f"
Jul 07 22:48:16.783601 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.783581    2555 scope.go:117] "RemoveContainer" containerID="8844069544d301e5b29b31490bc07053fb23c48c981500e98a7008a1af368e7f"
Jul 07 22:48:16.783891 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:48:16.783850    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8844069544d301e5b29b31490bc07053fb23c48c981500e98a7008a1af368e7f\": container with ID starting with 8844069544d301e5b29b31490bc07053fb23c48c981500e98a7008a1af368e7f not found: ID does not exist" containerID="8844069544d301e5b29b31490bc07053fb23c48c981500e98a7008a1af368e7f"
Jul 07 22:48:16.783957 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.783903    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8844069544d301e5b29b31490bc07053fb23c48c981500e98a7008a1af368e7f"} err="failed to get container status \"8844069544d301e5b29b31490bc07053fb23c48c981500e98a7008a1af368e7f\": rpc error: code = NotFound desc = could not find container \"8844069544d301e5b29b31490bc07053fb23c48c981500e98a7008a1af368e7f\": container with ID starting with 8844069544d301e5b29b31490bc07053fb23c48c981500e98a7008a1af368e7f not found: ID does not exist"
Jul 07 22:48:16.795728 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.795703    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"]
Jul 07 22:48:16.798891 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.798858    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-fb6979-7989444994-ctckv"]
Jul 07 22:48:16.922408 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:16.922374    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd44df46-2143-44ec-9ae5-1d157cebea3a" path="/var/lib/kubelet/pods/bd44df46-2143-44ec-9ae5-1d157cebea3a/volumes"
Jul 07 22:48:21.626381 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:21.626346    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"]
Jul 07 22:48:21.626743 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:21.626561    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4" podUID="a94c7289-eba2-4cfe-ab11-09b32703a947" containerName="istio-proxy" containerID="cri-o://898c28fb19c7215325e7cbba35a37b7c90539bb277da9c9f2e7521bed5cb0acc" gracePeriod=30
Jul 07 22:48:22.719583 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:22.719546    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"]
Jul 07 22:48:22.720034 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:22.719807    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7" podUID="798de976-4991-4768-99c9-636b6968e4c0" containerName="maas-api" containerID="cri-o://d8be31bec3803a0cd02d34095821bc52678cc26ee49bc1461c4580c4a3e2634a" gracePeriod=30
Jul 07 22:48:22.959814 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:22.959788    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"
Jul 07 22:48:23.077701 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:23.077606    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/798de976-4991-4768-99c9-636b6968e4c0-openshift-service-ca\") pod \"798de976-4991-4768-99c9-636b6968e4c0\" (UID: \"798de976-4991-4768-99c9-636b6968e4c0\") "
Jul 07 22:48:23.077701 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:23.077644    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/798de976-4991-4768-99c9-636b6968e4c0-maas-api-tls\") pod \"798de976-4991-4768-99c9-636b6968e4c0\" (UID: \"798de976-4991-4768-99c9-636b6968e4c0\") "
Jul 07 22:48:23.077981 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:23.077750    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwnhf\" (UniqueName: \"kubernetes.io/projected/798de976-4991-4768-99c9-636b6968e4c0-kube-api-access-zwnhf\") pod \"798de976-4991-4768-99c9-636b6968e4c0\" (UID: \"798de976-4991-4768-99c9-636b6968e4c0\") "
Jul 07 22:48:23.078082 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:23.078058    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/798de976-4991-4768-99c9-636b6968e4c0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "798de976-4991-4768-99c9-636b6968e4c0" (UID: "798de976-4991-4768-99c9-636b6968e4c0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 22:48:23.079789 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:23.079760    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/798de976-4991-4768-99c9-636b6968e4c0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "798de976-4991-4768-99c9-636b6968e4c0" (UID: "798de976-4991-4768-99c9-636b6968e4c0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 22:48:23.079845 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:23.079813    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/798de976-4991-4768-99c9-636b6968e4c0-kube-api-access-zwnhf" (OuterVolumeSpecName: "kube-api-access-zwnhf") pod "798de976-4991-4768-99c9-636b6968e4c0" (UID: "798de976-4991-4768-99c9-636b6968e4c0"). InnerVolumeSpecName "kube-api-access-zwnhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:48:23.179137 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:23.179091    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwnhf\" (UniqueName: \"kubernetes.io/projected/798de976-4991-4768-99c9-636b6968e4c0-kube-api-access-zwnhf\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:48:23.179137 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:23.179131    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/798de976-4991-4768-99c9-636b6968e4c0-openshift-service-ca\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:48:23.179137 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:23.179145    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/798de976-4991-4768-99c9-636b6968e4c0-maas-api-tls\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:48:23.809227 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:23.809192    2555 generic.go:356] "Generic (PLEG): container finished" podID="798de976-4991-4768-99c9-636b6968e4c0" containerID="d8be31bec3803a0cd02d34095821bc52678cc26ee49bc1461c4580c4a3e2634a" exitCode=0
Jul 07 22:48:23.809645 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:23.809237    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"
Jul 07 22:48:23.809645 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:23.809237    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7" event={"ID":"798de976-4991-4768-99c9-636b6968e4c0","Type":"ContainerDied","Data":"d8be31bec3803a0cd02d34095821bc52678cc26ee49bc1461c4580c4a3e2634a"}
Jul 07 22:48:23.809645 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:23.809369    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7" event={"ID":"798de976-4991-4768-99c9-636b6968e4c0","Type":"ContainerDied","Data":"0386d18f509acaf201d0e62671a360e14c12a0430f053a29b08c6a71b9ffe314"}
Jul 07 22:48:23.809645 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:23.809399    2555 scope.go:117] "RemoveContainer" containerID="d8be31bec3803a0cd02d34095821bc52678cc26ee49bc1461c4580c4a3e2634a"
Jul 07 22:48:23.818153 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:23.818137    2555 scope.go:117] "RemoveContainer" containerID="d8be31bec3803a0cd02d34095821bc52678cc26ee49bc1461c4580c4a3e2634a"
Jul 07 22:48:23.818431 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:48:23.818413    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8be31bec3803a0cd02d34095821bc52678cc26ee49bc1461c4580c4a3e2634a\": container with ID starting with d8be31bec3803a0cd02d34095821bc52678cc26ee49bc1461c4580c4a3e2634a not found: ID does not exist" containerID="d8be31bec3803a0cd02d34095821bc52678cc26ee49bc1461c4580c4a3e2634a"
Jul 07 22:48:23.818474 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:23.818440    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8be31bec3803a0cd02d34095821bc52678cc26ee49bc1461c4580c4a3e2634a"} err="failed to get container status \"d8be31bec3803a0cd02d34095821bc52678cc26ee49bc1461c4580c4a3e2634a\": rpc error: code = NotFound desc = could not find container \"d8be31bec3803a0cd02d34095821bc52678cc26ee49bc1461c4580c4a3e2634a\": container with ID starting with d8be31bec3803a0cd02d34095821bc52678cc26ee49bc1461c4580c4a3e2634a not found: ID does not exist"
Jul 07 22:48:23.829577 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:23.829550    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"]
Jul 07 22:48:23.833154 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:23.833128    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-17949f-99d78c86b-c8jc7"]
Jul 07 22:48:24.922320 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:24.922287    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="798de976-4991-4768-99c9-636b6968e4c0" path="/var/lib/kubelet/pods/798de976-4991-4768-99c9-636b6968e4c0/volumes"
Jul 07 22:48:26.825029 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:26.824998    2555 generic.go:356] "Generic (PLEG): container finished" podID="a94c7289-eba2-4cfe-ab11-09b32703a947" containerID="898c28fb19c7215325e7cbba35a37b7c90539bb277da9c9f2e7521bed5cb0acc" exitCode=0
Jul 07 22:48:26.825422 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:26.825071    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4" event={"ID":"a94c7289-eba2-4cfe-ab11-09b32703a947","Type":"ContainerDied","Data":"898c28fb19c7215325e7cbba35a37b7c90539bb277da9c9f2e7521bed5cb0acc"}
Jul 07 22:48:26.825422 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:26.825150    2555 scope.go:117] "RemoveContainer" containerID="c5ec4efff08cae1f5b89ecddbebeb8326798b8da17e7636f9964027948118348"
Jul 07 22:48:26.882148 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:26.882122    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:48:27.027963 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.027923    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-envoy\") pod \"a94c7289-eba2-4cfe-ab11-09b32703a947\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") "
Jul 07 22:48:27.028143 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.028008    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzljw\" (UniqueName: \"kubernetes.io/projected/a94c7289-eba2-4cfe-ab11-09b32703a947-kube-api-access-xzljw\") pod \"a94c7289-eba2-4cfe-ab11-09b32703a947\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") "
Jul 07 22:48:27.028143 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.028038    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-workload-certs\") pod \"a94c7289-eba2-4cfe-ab11-09b32703a947\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") "
Jul 07 22:48:27.028143 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.028068    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-podinfo\") pod \"a94c7289-eba2-4cfe-ab11-09b32703a947\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") "
Jul 07 22:48:27.028143 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.028086    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a94c7289-eba2-4cfe-ab11-09b32703a947-istiod-ca-cert\") pod \"a94c7289-eba2-4cfe-ab11-09b32703a947\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") "
Jul 07 22:48:27.028143 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.028107    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-workload-socket\") pod \"a94c7289-eba2-4cfe-ab11-09b32703a947\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") "
Jul 07 22:48:27.028143 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.028131    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-data\") pod \"a94c7289-eba2-4cfe-ab11-09b32703a947\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") "
Jul 07 22:48:27.028498 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.028168    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-credential-socket\") pod \"a94c7289-eba2-4cfe-ab11-09b32703a947\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") "
Jul 07 22:48:27.028498 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.028190    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-token\") pod \"a94c7289-eba2-4cfe-ab11-09b32703a947\" (UID: \"a94c7289-eba2-4cfe-ab11-09b32703a947\") "
Jul 07 22:48:27.028498 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.028356    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a94c7289-eba2-4cfe-ab11-09b32703a947" (UID: "a94c7289-eba2-4cfe-ab11-09b32703a947"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:48:27.028498 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.028466    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-workload-socket\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:48:27.028710 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.028509    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a94c7289-eba2-4cfe-ab11-09b32703a947" (UID: "a94c7289-eba2-4cfe-ab11-09b32703a947"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:48:27.028710 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.028574    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a94c7289-eba2-4cfe-ab11-09b32703a947-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a94c7289-eba2-4cfe-ab11-09b32703a947" (UID: "a94c7289-eba2-4cfe-ab11-09b32703a947"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 22:48:27.028710 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.028639    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a94c7289-eba2-4cfe-ab11-09b32703a947" (UID: "a94c7289-eba2-4cfe-ab11-09b32703a947"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:48:27.029075 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.029049    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-data" (OuterVolumeSpecName: "istio-data") pod "a94c7289-eba2-4cfe-ab11-09b32703a947" (UID: "a94c7289-eba2-4cfe-ab11-09b32703a947"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:48:27.030281 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.030260    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a94c7289-eba2-4cfe-ab11-09b32703a947-kube-api-access-xzljw" (OuterVolumeSpecName: "kube-api-access-xzljw") pod "a94c7289-eba2-4cfe-ab11-09b32703a947" (UID: "a94c7289-eba2-4cfe-ab11-09b32703a947"). InnerVolumeSpecName "kube-api-access-xzljw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:48:27.030495 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.030467    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-token" (OuterVolumeSpecName: "istio-token") pod "a94c7289-eba2-4cfe-ab11-09b32703a947" (UID: "a94c7289-eba2-4cfe-ab11-09b32703a947"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:48:27.030653 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.030636    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a94c7289-eba2-4cfe-ab11-09b32703a947" (UID: "a94c7289-eba2-4cfe-ab11-09b32703a947"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 22:48:27.030793 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.030774    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a94c7289-eba2-4cfe-ab11-09b32703a947" (UID: "a94c7289-eba2-4cfe-ab11-09b32703a947"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:48:27.129337 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.129297    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-envoy\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:48:27.129337 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.129327    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzljw\" (UniqueName: \"kubernetes.io/projected/a94c7289-eba2-4cfe-ab11-09b32703a947-kube-api-access-xzljw\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:48:27.129337 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.129339    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-workload-certs\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:48:27.129337 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.129350    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-podinfo\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:48:27.129614 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.129358    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a94c7289-eba2-4cfe-ab11-09b32703a947-istiod-ca-cert\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:48:27.129614 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.129366    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-data\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:48:27.129614 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.129377    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94c7289-eba2-4cfe-ab11-09b32703a947-credential-socket\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:48:27.129614 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.129385    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a94c7289-eba2-4cfe-ab11-09b32703a947-istio-token\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:48:27.833213 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.833178    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4" event={"ID":"a94c7289-eba2-4cfe-ab11-09b32703a947","Type":"ContainerDied","Data":"ae934a084c09b6b6d800ca7c5c1fbf1996867c81f5a00c6064bf04215b7b8c0b"}
Jul 07 22:48:27.833650 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.833269    2555 scope.go:117] "RemoveContainer" containerID="898c28fb19c7215325e7cbba35a37b7c90539bb277da9c9f2e7521bed5cb0acc"
Jul 07 22:48:27.833650 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.833188    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"
Jul 07 22:48:27.854090 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.854068    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"]
Jul 07 22:48:27.860084 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:27.860059    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-fb6979-openshift-default-5fcbd544cc-66vs4"]
Jul 07 22:48:28.922012 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:28.921982    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a94c7289-eba2-4cfe-ab11-09b32703a947" path="/var/lib/kubelet/pods/a94c7289-eba2-4cfe-ab11-09b32703a947/volumes"
Jul 07 22:48:29.547199 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.547157    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"]
Jul 07 22:48:29.547607 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.547578    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd44df46-2143-44ec-9ae5-1d157cebea3a" containerName="maas-api"
Jul 07 22:48:29.547607 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.547605    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd44df46-2143-44ec-9ae5-1d157cebea3a" containerName="maas-api"
Jul 07 22:48:29.547760 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.547623    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="798de976-4991-4768-99c9-636b6968e4c0" containerName="maas-api"
Jul 07 22:48:29.547760 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.547631    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="798de976-4991-4768-99c9-636b6968e4c0" containerName="maas-api"
Jul 07 22:48:29.547760 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.547651    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a94c7289-eba2-4cfe-ab11-09b32703a947" containerName="istio-proxy"
Jul 07 22:48:29.547760 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.547659    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a94c7289-eba2-4cfe-ab11-09b32703a947" containerName="istio-proxy"
Jul 07 22:48:29.547760 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.547671    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a94c7289-eba2-4cfe-ab11-09b32703a947" containerName="istio-proxy"
Jul 07 22:48:29.547760 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.547679    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a94c7289-eba2-4cfe-ab11-09b32703a947" containerName="istio-proxy"
Jul 07 22:48:29.547760 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.547752    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd44df46-2143-44ec-9ae5-1d157cebea3a" containerName="maas-api"
Jul 07 22:48:29.548013 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.547771    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a94c7289-eba2-4cfe-ab11-09b32703a947" containerName="istio-proxy"
Jul 07 22:48:29.548013 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.547783    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a94c7289-eba2-4cfe-ab11-09b32703a947" containerName="istio-proxy"
Jul 07 22:48:29.548013 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.547791    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="798de976-4991-4768-99c9-636b6968e4c0" containerName="maas-api"
Jul 07 22:48:29.550469 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.550435    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.553264 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.553243    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 07 22:48:29.553264 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.553245    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-af2c94-openshift-default-dockercfg-hpgtm\""
Jul 07 22:48:29.562918 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.562893    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"]
Jul 07 22:48:29.652160 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.652124    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-data\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.652160 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.652167    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df21bb96-43a4-4a70-a23a-11b2c8630e64-istiod-ca-cert\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.652429 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.652195    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-credential-socket\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.652429 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.652284    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-envoy\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.652429 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.652318    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-podinfo\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.652429 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.652347    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-workload-socket\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.652429 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.652387    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thfxr\" (UniqueName: \"kubernetes.io/projected/df21bb96-43a4-4a70-a23a-11b2c8630e64-kube-api-access-thfxr\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.652640 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.652449    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-token\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.652640 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.652480    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-workload-certs\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.752971 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.752932    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-workload-socket\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.752971 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.752977    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thfxr\" (UniqueName: \"kubernetes.io/projected/df21bb96-43a4-4a70-a23a-11b2c8630e64-kube-api-access-thfxr\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.753214 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.753033    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-token\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.753214 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.753065    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-workload-certs\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.753214 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.753106    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-data\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.753214 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.753152    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df21bb96-43a4-4a70-a23a-11b2c8630e64-istiod-ca-cert\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.753424 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.753215    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-credential-socket\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.753424 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.753389    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-workload-socket\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.753531 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.753462    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-data\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.753531 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.753490    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-envoy\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.753531 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.753518    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-podinfo\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.753695 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.753515    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-workload-certs\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.753695 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.753596    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-credential-socket\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.753885 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.753855    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df21bb96-43a4-4a70-a23a-11b2c8630e64-istiod-ca-cert\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.755689 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.755668    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-envoy\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.755851 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.755832    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-podinfo\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.760613 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.760589    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-token\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.760707 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.760688    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thfxr\" (UniqueName: \"kubernetes.io/projected/df21bb96-43a4-4a70-a23a-11b2c8630e64-kube-api-access-thfxr\") pod \"e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") " pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.860730 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.860642    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:29.997519 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:29.997496    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"]
Jul 07 22:48:29.999327 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:48:29.999294    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf21bb96_43a4_4a70_a23a_11b2c8630e64.slice/crio-a74008b622b53f59c708094a0f18613c6469979407cc4a344b5c4c3b8793d721 WatchSource:0}: Error finding container a74008b622b53f59c708094a0f18613c6469979407cc4a344b5c4c3b8793d721: Status 404 returned error can't find the container with id a74008b622b53f59c708094a0f18613c6469979407cc4a344b5c4c3b8793d721
Jul 07 22:48:30.001507 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:30.001469    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:48:30.001581 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:30.001542    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:48:30.001581 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:30.001571    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:48:30.849545 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:30.849510    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv" event={"ID":"df21bb96-43a4-4a70-a23a-11b2c8630e64","Type":"ContainerStarted","Data":"755f1fe663bea616eab65f8a20cbc64dc2025325e35e8afc25761277937adf04"}
Jul 07 22:48:30.849763 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:30.849561    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv" event={"ID":"df21bb96-43a4-4a70-a23a-11b2c8630e64","Type":"ContainerStarted","Data":"a74008b622b53f59c708094a0f18613c6469979407cc4a344b5c4c3b8793d721"}
Jul 07 22:48:30.861339 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:30.861310    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:30.870117 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:30.870066    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv" podStartSLOduration=1.870050107 podStartE2EDuration="1.870050107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:48:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:48:30.867749955 +0000 UTC m=+2008.539968947" watchObservedRunningTime="2026-07-07 22:48:30.870050107 +0000 UTC m=+2008.542269095"
Jul 07 22:48:31.866112 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:31.866076    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:32.858804 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:32.858778    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:32.859748 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:32.859731    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:32.938438 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:32.938409    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg"]
Jul 07 22:48:32.940532 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:32.940514    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg"
Jul 07 22:48:32.947800 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:32.947778    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-af2c94-serving-cert\""
Jul 07 22:48:32.947939 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:32.947916    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-r9rv4\""
Jul 07 22:48:32.956525 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:32.956500    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg"]
Jul 07 22:48:32.982449 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:32.982192    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd9e3607-a143-4704-a017-9533cd98c775-openshift-service-ca\") pod \"maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg\" (UID: \"cd9e3607-a143-4704-a017-9533cd98c775\") " pod="opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg"
Jul 07 22:48:32.982627 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:32.982556    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25dfr\" (UniqueName: \"kubernetes.io/projected/cd9e3607-a143-4704-a017-9533cd98c775-kube-api-access-25dfr\") pod \"maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg\" (UID: \"cd9e3607-a143-4704-a017-9533cd98c775\") " pod="opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg"
Jul 07 22:48:32.982627 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:32.982610    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd9e3607-a143-4704-a017-9533cd98c775-maas-api-tls\") pod \"maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg\" (UID: \"cd9e3607-a143-4704-a017-9533cd98c775\") " pod="opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg"
Jul 07 22:48:33.083528 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:33.083495    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd9e3607-a143-4704-a017-9533cd98c775-openshift-service-ca\") pod \"maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg\" (UID: \"cd9e3607-a143-4704-a017-9533cd98c775\") " pod="opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg"
Jul 07 22:48:33.083703 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:33.083547    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25dfr\" (UniqueName: \"kubernetes.io/projected/cd9e3607-a143-4704-a017-9533cd98c775-kube-api-access-25dfr\") pod \"maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg\" (UID: \"cd9e3607-a143-4704-a017-9533cd98c775\") " pod="opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg"
Jul 07 22:48:33.083703 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:33.083569    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd9e3607-a143-4704-a017-9533cd98c775-maas-api-tls\") pod \"maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg\" (UID: \"cd9e3607-a143-4704-a017-9533cd98c775\") " pod="opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg"
Jul 07 22:48:33.084166 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:33.084146    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd9e3607-a143-4704-a017-9533cd98c775-openshift-service-ca\") pod \"maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg\" (UID: \"cd9e3607-a143-4704-a017-9533cd98c775\") " pod="opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg"
Jul 07 22:48:33.085896 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:33.085862    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd9e3607-a143-4704-a017-9533cd98c775-maas-api-tls\") pod \"maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg\" (UID: \"cd9e3607-a143-4704-a017-9533cd98c775\") " pod="opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg"
Jul 07 22:48:33.091971 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:33.091939    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25dfr\" (UniqueName: \"kubernetes.io/projected/cd9e3607-a143-4704-a017-9533cd98c775-kube-api-access-25dfr\") pod \"maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg\" (UID: \"cd9e3607-a143-4704-a017-9533cd98c775\") " pod="opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg"
Jul 07 22:48:33.250843 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:33.250805    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg"
Jul 07 22:48:33.387484 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:33.387460    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg"]
Jul 07 22:48:33.389690 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:48:33.389668    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd9e3607_a143_4704_a017_9533cd98c775.slice/crio-293f857ac18b7b2e2021f3a133d021a0aebcbcdc378484ad820ca7f6caaaa8de WatchSource:0}: Error finding container 293f857ac18b7b2e2021f3a133d021a0aebcbcdc378484ad820ca7f6caaaa8de: Status 404 returned error can't find the container with id 293f857ac18b7b2e2021f3a133d021a0aebcbcdc378484ad820ca7f6caaaa8de
Jul 07 22:48:33.866572 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:33.866538    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg" event={"ID":"cd9e3607-a143-4704-a017-9533cd98c775","Type":"ContainerStarted","Data":"d4673f21c9bfed8ac9a2c6236955a9a631af17620e9584c470235cfd1283689e"}
Jul 07 22:48:33.866803 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:33.866595    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg" event={"ID":"cd9e3607-a143-4704-a017-9533cd98c775","Type":"ContainerStarted","Data":"293f857ac18b7b2e2021f3a133d021a0aebcbcdc378484ad820ca7f6caaaa8de"}
Jul 07 22:48:33.866803 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:33.866736    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg"
Jul 07 22:48:33.883577 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:33.883525    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg" podStartSLOduration=1.5335599100000001 podStartE2EDuration="1.883510068s" totalImagesPullingTime="349.950158ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:48:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:48:33.881410172 +0000 UTC m=+2011.553629161" watchObservedRunningTime="2026-07-07 22:48:33.883510068 +0000 UTC m=+2011.555729055"
Jul 07 22:48:39.887320 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:39.887293    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg"
Jul 07 22:48:41.076222 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:41.075852    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"]
Jul 07 22:48:41.079941 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:41.079915    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"
Jul 07 22:48:41.082541 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:41.082518    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-986e04-serving-cert\""
Jul 07 22:48:41.091332 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:41.091234    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"]
Jul 07 22:48:41.155365 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:41.155333    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkhsh\" (UniqueName: \"kubernetes.io/projected/3196425d-00b5-40b1-84f6-7477c60b105d-kube-api-access-vkhsh\") pod \"maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w\" (UID: \"3196425d-00b5-40b1-84f6-7477c60b105d\") " pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"
Jul 07 22:48:41.155543 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:41.155374    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3196425d-00b5-40b1-84f6-7477c60b105d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w\" (UID: \"3196425d-00b5-40b1-84f6-7477c60b105d\") " pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"
Jul 07 22:48:41.155543 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:41.155461    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3196425d-00b5-40b1-84f6-7477c60b105d-maas-api-tls\") pod \"maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w\" (UID: \"3196425d-00b5-40b1-84f6-7477c60b105d\") " pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"
Jul 07 22:48:41.256618 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:41.256584    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3196425d-00b5-40b1-84f6-7477c60b105d-maas-api-tls\") pod \"maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w\" (UID: \"3196425d-00b5-40b1-84f6-7477c60b105d\") " pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"
Jul 07 22:48:41.256800 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:41.256651    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkhsh\" (UniqueName: \"kubernetes.io/projected/3196425d-00b5-40b1-84f6-7477c60b105d-kube-api-access-vkhsh\") pod \"maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w\" (UID: \"3196425d-00b5-40b1-84f6-7477c60b105d\") " pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"
Jul 07 22:48:41.256800 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:48:41.256739    2555 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-986e04-serving-cert: secret "maas-api-e2e-shared-b-986e04-serving-cert" not found
Jul 07 22:48:41.256948 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:48:41.256803    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3196425d-00b5-40b1-84f6-7477c60b105d-maas-api-tls podName:3196425d-00b5-40b1-84f6-7477c60b105d nodeName:}" failed. No retries permitted until 2026-07-07 22:48:41.756786018 +0000 UTC m=+2019.429004983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3196425d-00b5-40b1-84f6-7477c60b105d-maas-api-tls") pod "maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w" (UID: "3196425d-00b5-40b1-84f6-7477c60b105d") : secret "maas-api-e2e-shared-b-986e04-serving-cert" not found
Jul 07 22:48:41.256948 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:41.256820    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3196425d-00b5-40b1-84f6-7477c60b105d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w\" (UID: \"3196425d-00b5-40b1-84f6-7477c60b105d\") " pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"
Jul 07 22:48:41.257408 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:41.257392    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3196425d-00b5-40b1-84f6-7477c60b105d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w\" (UID: \"3196425d-00b5-40b1-84f6-7477c60b105d\") " pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"
Jul 07 22:48:41.266486 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:41.266449    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkhsh\" (UniqueName: \"kubernetes.io/projected/3196425d-00b5-40b1-84f6-7477c60b105d-kube-api-access-vkhsh\") pod \"maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w\" (UID: \"3196425d-00b5-40b1-84f6-7477c60b105d\") " pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"
Jul 07 22:48:41.762541 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:41.762501    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3196425d-00b5-40b1-84f6-7477c60b105d-maas-api-tls\") pod \"maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w\" (UID: \"3196425d-00b5-40b1-84f6-7477c60b105d\") " pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"
Jul 07 22:48:41.765776 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:41.765716    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3196425d-00b5-40b1-84f6-7477c60b105d-maas-api-tls\") pod \"maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w\" (UID: \"3196425d-00b5-40b1-84f6-7477c60b105d\") " pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"
Jul 07 22:48:42.001621 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:42.001580    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"
Jul 07 22:48:42.132165 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:42.132139    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"]
Jul 07 22:48:42.134388 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:48:42.134361    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3196425d_00b5_40b1_84f6_7477c60b105d.slice/crio-5d0e695938fc5598028a0f139038e14892b6c420543f6913463ff44a7d3e13ed WatchSource:0}: Error finding container 5d0e695938fc5598028a0f139038e14892b6c420543f6913463ff44a7d3e13ed: Status 404 returned error can't find the container with id 5d0e695938fc5598028a0f139038e14892b6c420543f6913463ff44a7d3e13ed
Jul 07 22:48:42.923029 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:42.922995    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w" event={"ID":"3196425d-00b5-40b1-84f6-7477c60b105d","Type":"ContainerStarted","Data":"b36c98b5a7e309858115d86e943058186c8d4172d493dfc2467a3f33f99cf5bc"}
Jul 07 22:48:42.923229 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:42.923044    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"
Jul 07 22:48:42.923229 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:42.923062    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w" event={"ID":"3196425d-00b5-40b1-84f6-7477c60b105d","Type":"ContainerStarted","Data":"5d0e695938fc5598028a0f139038e14892b6c420543f6913463ff44a7d3e13ed"}
Jul 07 22:48:42.938660 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:42.938597    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w" podStartSLOduration=1.6338171209999999 podStartE2EDuration="1.938576048s" totalImagesPullingTime="304.758927ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:48:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:48:42.937024834 +0000 UTC m=+2020.609243840" watchObservedRunningTime="2026-07-07 22:48:42.938576048 +0000 UTC m=+2020.610795037"
Jul 07 22:48:44.931644 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:44.931556    2555 generic.go:356] "Generic (PLEG): container finished" podID="df21bb96-43a4-4a70-a23a-11b2c8630e64" containerID="755f1fe663bea616eab65f8a20cbc64dc2025325e35e8afc25761277937adf04" exitCode=0
Jul 07 22:48:44.931644 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:44.931618    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv" event={"ID":"df21bb96-43a4-4a70-a23a-11b2c8630e64","Type":"ContainerDied","Data":"755f1fe663bea616eab65f8a20cbc64dc2025325e35e8afc25761277937adf04"}
Jul 07 22:48:44.932137 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:44.931992    2555 scope.go:117] "RemoveContainer" containerID="755f1fe663bea616eab65f8a20cbc64dc2025325e35e8afc25761277937adf04"
Jul 07 22:48:44.932962 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:44.932932    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:48:44.933026 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:44.932998    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:48:44.933068 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:44.933029    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 22:48:45.938489 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:45.938457    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv" event={"ID":"df21bb96-43a4-4a70-a23a-11b2c8630e64","Type":"ContainerStarted","Data":"ddcd7e532b450754113abe1613bbe93a47aeb948f2e115c973b7b84b7a3c9f9d"}
Jul 07 22:48:46.862895 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:46.862839    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:46.863110 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:46.862992    2555 patch_prober.go:28] interesting pod/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 07 22:48:46.863110 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:46.863039    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv" podUID="df21bb96-43a4-4a70-a23a-11b2c8630e64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 07 22:48:47.479893 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.475954    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"]
Jul 07 22:48:47.479893 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.478929    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:47.488841 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.488806    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-af2c94\"/\"default-dockercfg-l6txc\""
Jul 07 22:48:47.489170 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.489145    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-af2c94\"/\"openshift-service-ca.crt\""
Jul 07 22:48:47.489409 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.489390    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-af2c94\"/\"auth-test-model-af2c94-kserve-self-signed-certs\""
Jul 07 22:48:47.492891 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.489723    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-af2c94\"/\"kube-root-ca.crt\""
Jul 07 22:48:47.494326 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.494298    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"]
Jul 07 22:48:47.617458 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.617412    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-tmp-dir\") pod \"auth-test-model-af2c94-kserve-7746768975-8pd5s\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") " pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:47.617675 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.617484    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hplqq\" (UniqueName: \"kubernetes.io/projected/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-kube-api-access-hplqq\") pod \"auth-test-model-af2c94-kserve-7746768975-8pd5s\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") " pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:47.617675 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.617560    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-tls-certs\") pod \"auth-test-model-af2c94-kserve-7746768975-8pd5s\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") " pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:47.617675 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.617620    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-home\") pod \"auth-test-model-af2c94-kserve-7746768975-8pd5s\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") " pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:47.617675 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.617650    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-model-cache\") pod \"auth-test-model-af2c94-kserve-7746768975-8pd5s\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") " pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:47.617855 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.617741    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-dshm\") pod \"auth-test-model-af2c94-kserve-7746768975-8pd5s\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") " pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:47.719046 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.719005    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-dshm\") pod \"auth-test-model-af2c94-kserve-7746768975-8pd5s\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") " pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:47.719046 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.719054    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-tmp-dir\") pod \"auth-test-model-af2c94-kserve-7746768975-8pd5s\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") " pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:47.719314 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.719087    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hplqq\" (UniqueName: \"kubernetes.io/projected/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-kube-api-access-hplqq\") pod \"auth-test-model-af2c94-kserve-7746768975-8pd5s\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") " pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:47.719314 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.719107    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-tls-certs\") pod \"auth-test-model-af2c94-kserve-7746768975-8pd5s\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") " pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:47.719314 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.719136    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-home\") pod \"auth-test-model-af2c94-kserve-7746768975-8pd5s\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") " pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:47.719314 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.719293    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-model-cache\") pod \"auth-test-model-af2c94-kserve-7746768975-8pd5s\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") " pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:47.719537 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.719519    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-tmp-dir\") pod \"auth-test-model-af2c94-kserve-7746768975-8pd5s\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") " pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:47.719598 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.719564    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-home\") pod \"auth-test-model-af2c94-kserve-7746768975-8pd5s\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") " pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:47.719666 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.719592    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-model-cache\") pod \"auth-test-model-af2c94-kserve-7746768975-8pd5s\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") " pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:47.721408 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.721389    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-dshm\") pod \"auth-test-model-af2c94-kserve-7746768975-8pd5s\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") " pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:47.721696 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.721671    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-tls-certs\") pod \"auth-test-model-af2c94-kserve-7746768975-8pd5s\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") " pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:47.728078 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.728053    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hplqq\" (UniqueName: \"kubernetes.io/projected/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-kube-api-access-hplqq\") pod \"auth-test-model-af2c94-kserve-7746768975-8pd5s\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") " pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:47.800132 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.800034    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:47.868894 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.866070    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:47.868894 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.866230    2555 patch_prober.go:28] interesting pod/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 07 22:48:47.868894 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.866274    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv" podUID="df21bb96-43a4-4a70-a23a-11b2c8630e64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 07 22:48:47.948983 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:47.948961    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"]
Jul 07 22:48:47.950986 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:48:47.950958    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c387b4c_c9b5_4a3d_97c0_f7bf0ed513ac.slice/crio-36d85d90e7ce22032af16ab8e80752c5ff2d4896c2af1e1181f1aebf105c108c WatchSource:0}: Error finding container 36d85d90e7ce22032af16ab8e80752c5ff2d4896c2af1e1181f1aebf105c108c: Status 404 returned error can't find the container with id 36d85d90e7ce22032af16ab8e80752c5ff2d4896c2af1e1181f1aebf105c108c
Jul 07 22:48:48.861746 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:48.861707    2555 patch_prober.go:28] interesting pod/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 07 22:48:48.862357 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:48.862324    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv" podUID="df21bb96-43a4-4a70-a23a-11b2c8630e64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 07 22:48:48.931439 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:48.931411    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"
Jul 07 22:48:48.952732 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:48.952697    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s" event={"ID":"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac","Type":"ContainerStarted","Data":"c9dbd2bb6975a03e78a206ebea47bfcf747c31385d027acc8d1907e65649b958"}
Jul 07 22:48:48.952732 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:48.952736    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s" event={"ID":"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac","Type":"ContainerStarted","Data":"36d85d90e7ce22032af16ab8e80752c5ff2d4896c2af1e1181f1aebf105c108c"}
Jul 07 22:48:48.968302 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:48.968241    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s" podStartSLOduration=1.968220869 podStartE2EDuration="1.968220869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:48:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:48:48.967740005 +0000 UTC m=+2026.639958995" watchObservedRunningTime="2026-07-07 22:48:48.968220869 +0000 UTC m=+2026.640439859"
Jul 07 22:48:49.861636 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:49.861598    2555 patch_prober.go:28] interesting pod/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 07 22:48:49.861819 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:49.861664    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv" podUID="df21bb96-43a4-4a70-a23a-11b2c8630e64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 07 22:48:50.864987 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:50.864956    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:50.865846 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:50.865827    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:48:57.800284 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:57.800249    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:57.800720 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:57.800388    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:57.812724 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:57.812693    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:48:58.004883 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:48:58.004839    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:50:02.935684 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:50:02.935655    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:50:02.942212 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:50:02.942189    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:51:13.513975 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:13.513894    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"]
Jul 07 22:51:28.533502 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.533444    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s" podUID="5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac" containerName="main" containerID="cri-o://c9dbd2bb6975a03e78a206ebea47bfcf747c31385d027acc8d1907e65649b958" gracePeriod=45
Jul 07 22:51:28.688686 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.688656    2555 generic.go:356] "Generic (PLEG): container finished" podID="5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac" containerID="c9dbd2bb6975a03e78a206ebea47bfcf747c31385d027acc8d1907e65649b958" exitCode=0
Jul 07 22:51:28.688850 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.688721    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s" event={"ID":"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac","Type":"ContainerDied","Data":"c9dbd2bb6975a03e78a206ebea47bfcf747c31385d027acc8d1907e65649b958"}
Jul 07 22:51:28.782727 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.782701    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:51:28.817041 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.816961    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hplqq\" (UniqueName: \"kubernetes.io/projected/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-kube-api-access-hplqq\") pod \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") "
Jul 07 22:51:28.817041 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.817013    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-home\") pod \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") "
Jul 07 22:51:28.817256 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.817041    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-tmp-dir\") pod \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") "
Jul 07 22:51:28.817256 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.817086    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-tls-certs\") pod \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") "
Jul 07 22:51:28.817256 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.817133    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-model-cache\") pod \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") "
Jul 07 22:51:28.817256 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.817226    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-dshm\") pod \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\" (UID: \"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac\") "
Jul 07 22:51:28.817462 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.817333    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-home" (OuterVolumeSpecName: "home") pod "5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac" (UID: "5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:51:28.817534 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.817512    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac" (UID: "5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:51:28.817618 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.817576    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-model-cache" (OuterVolumeSpecName: "model-cache") pod "5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac" (UID: "5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:51:28.817618 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.817607    2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-home\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:51:28.817714 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.817652    2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-tmp-dir\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:51:28.819307 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.819275    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-kube-api-access-hplqq" (OuterVolumeSpecName: "kube-api-access-hplqq") pod "5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac" (UID: "5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac"). InnerVolumeSpecName "kube-api-access-hplqq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:51:28.819625 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.819601    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac" (UID: "5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 22:51:28.819743 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.819627    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-dshm" (OuterVolumeSpecName: "dshm") pod "5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac" (UID: "5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:51:28.918915 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.918863    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hplqq\" (UniqueName: \"kubernetes.io/projected/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-kube-api-access-hplqq\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:51:28.918915 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.918910    2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-tls-certs\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:51:28.918915 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.918921    2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-model-cache\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:51:28.919146 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:28.918932    2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac-dshm\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:51:29.694485 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:29.694446    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"
Jul 07 22:51:29.694923 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:29.694446    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s" event={"ID":"5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac","Type":"ContainerDied","Data":"36d85d90e7ce22032af16ab8e80752c5ff2d4896c2af1e1181f1aebf105c108c"}
Jul 07 22:51:29.694923 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:29.694575    2555 scope.go:117] "RemoveContainer" containerID="c9dbd2bb6975a03e78a206ebea47bfcf747c31385d027acc8d1907e65649b958"
Jul 07 22:51:29.727780 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:29.727743    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"]
Jul 07 22:51:29.736562 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:29.736531    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-af2c94/auth-test-model-af2c94-kserve-7746768975-8pd5s"]
Jul 07 22:51:30.923024 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:30.922995    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac" path="/var/lib/kubelet/pods/5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac/volumes"
Jul 07 22:51:54.172449 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.172415    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"]
Jul 07 22:51:54.172953 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.172747    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac" containerName="main"
Jul 07 22:51:54.172953 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.172757    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac" containerName="main"
Jul 07 22:51:54.172953 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.172817    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c387b4c-c9b5-4a3d-97c0-f7bf0ed513ac" containerName="main"
Jul 07 22:51:54.174666 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.174650    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:51:54.178051 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.178023    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-986e04\"/\"sub-test-model-986e04-kserve-self-signed-certs\""
Jul 07 22:51:54.178163 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.178026    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-986e04\"/\"openshift-service-ca.crt\""
Jul 07 22:51:54.178163 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.178039    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-986e04\"/\"kube-root-ca.crt\""
Jul 07 22:51:54.178163 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.178113    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-986e04\"/\"default-dockercfg-8wcrj\""
Jul 07 22:51:54.184697 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.184673    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"]
Jul 07 22:51:54.356968 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.356927    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ldtz\" (UniqueName: \"kubernetes.io/projected/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-kube-api-access-6ldtz\") pod \"sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") " pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:51:54.356968 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.356970    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-tls-certs\") pod \"sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") " pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:51:54.357218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.357057    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-dshm\") pod \"sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") " pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:51:54.357218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.357117    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-home\") pod \"sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") " pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:51:54.357218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.357144    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-model-cache\") pod \"sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") " pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:51:54.357218 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.357165    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-tmp-dir\") pod \"sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") " pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:51:54.458045 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.458014    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ldtz\" (UniqueName: \"kubernetes.io/projected/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-kube-api-access-6ldtz\") pod \"sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") " pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:51:54.458215 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.458058    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-tls-certs\") pod \"sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") " pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:51:54.458215 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.458129    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-dshm\") pod \"sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") " pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:51:54.458215 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.458153    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-home\") pod \"sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") " pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:51:54.458215 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.458175    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-model-cache\") pod \"sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") " pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:51:54.458215 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.458196    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-tmp-dir\") pod \"sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") " pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:51:54.458622 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.458595    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-home\") pod \"sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") " pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:51:54.458754 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.458652    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-tmp-dir\") pod \"sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") " pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:51:54.458754 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.458696    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-model-cache\") pod \"sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") " pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:51:54.460481 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.460460    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-dshm\") pod \"sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") " pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:51:54.460565 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.460542    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-tls-certs\") pod \"sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") " pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:51:54.466133 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.466112    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ldtz\" (UniqueName: \"kubernetes.io/projected/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-kube-api-access-6ldtz\") pod \"sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") " pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:51:54.483985 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.483961    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:51:54.621777 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.621751    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"]
Jul 07 22:51:54.623843 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:51:54.623816    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5868da3_7c3c_4401_b71b_a1aa09f82eb4.slice/crio-ce460844bc12277d945d48ce8d692b91f997a5aa80e2a48459054bfe0d42c504 WatchSource:0}: Error finding container ce460844bc12277d945d48ce8d692b91f997a5aa80e2a48459054bfe0d42c504: Status 404 returned error can't find the container with id ce460844bc12277d945d48ce8d692b91f997a5aa80e2a48459054bfe0d42c504
Jul 07 22:51:54.810252 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.810160    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r" event={"ID":"f5868da3-7c3c-4401-b71b-a1aa09f82eb4","Type":"ContainerStarted","Data":"de5cc55fa1bfe1dcaafa15c75e20117aee1c8ea0b0abce9a4715f66ebe7ecbff"}
Jul 07 22:51:54.810252 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.810200    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r" event={"ID":"f5868da3-7c3c-4401-b71b-a1aa09f82eb4","Type":"ContainerStarted","Data":"ce460844bc12277d945d48ce8d692b91f997a5aa80e2a48459054bfe0d42c504"}
Jul 07 22:51:54.827680 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:51:54.827628    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r" podStartSLOduration=0.827611557 podStartE2EDuration="827.611557ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:51:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:51:54.825582185 +0000 UTC m=+2212.497801173" watchObservedRunningTime="2026-07-07 22:51:54.827611557 +0000 UTC m=+2212.499830566"
Jul 07 22:52:04.484325 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:04.484285    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:52:04.484325 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:04.484334    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:52:04.497182 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:04.497154    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:52:04.868558 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:04.868485    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:52:36.263179 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:36.263148    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"]
Jul 07 22:52:39.190141 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.190109    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"]
Jul 07 22:52:39.193094 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.193078    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:39.196579 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.196555    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-af2c94\"/\"default-dockercfg-l6txc\""
Jul 07 22:52:39.196579 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.196560    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-af2c94\"/\"kube-root-ca.crt\""
Jul 07 22:52:39.196579 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.196559    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-af2c94\"/\"rate-test-model-af2c94-kserve-self-signed-certs\""
Jul 07 22:52:39.196832 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.196560    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-af2c94\"/\"openshift-service-ca.crt\""
Jul 07 22:52:39.204572 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.204552    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"]
Jul 07 22:52:39.268633 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.268602    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-home\") pod \"rate-test-model-af2c94-kserve-58f746dcdc-x8vq2\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") " pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:39.268633 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.268634    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/021795c5-a9f6-4308-b919-fa7cb735e961-tls-certs\") pod \"rate-test-model-af2c94-kserve-58f746dcdc-x8vq2\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") " pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:39.268828 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.268671    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-model-cache\") pod \"rate-test-model-af2c94-kserve-58f746dcdc-x8vq2\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") " pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:39.268828 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.268794    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtstq\" (UniqueName: \"kubernetes.io/projected/021795c5-a9f6-4308-b919-fa7cb735e961-kube-api-access-qtstq\") pod \"rate-test-model-af2c94-kserve-58f746dcdc-x8vq2\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") " pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:39.268828 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.268822    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-dshm\") pod \"rate-test-model-af2c94-kserve-58f746dcdc-x8vq2\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") " pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:39.268952 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.268845    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-tmp-dir\") pod \"rate-test-model-af2c94-kserve-58f746dcdc-x8vq2\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") " pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:39.369484 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.369453    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-home\") pod \"rate-test-model-af2c94-kserve-58f746dcdc-x8vq2\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") " pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:39.369484 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.369485    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/021795c5-a9f6-4308-b919-fa7cb735e961-tls-certs\") pod \"rate-test-model-af2c94-kserve-58f746dcdc-x8vq2\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") " pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:39.369705 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.369517    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-model-cache\") pod \"rate-test-model-af2c94-kserve-58f746dcdc-x8vq2\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") " pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:39.369705 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.369556    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtstq\" (UniqueName: \"kubernetes.io/projected/021795c5-a9f6-4308-b919-fa7cb735e961-kube-api-access-qtstq\") pod \"rate-test-model-af2c94-kserve-58f746dcdc-x8vq2\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") " pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:39.369705 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.369572    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-dshm\") pod \"rate-test-model-af2c94-kserve-58f746dcdc-x8vq2\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") " pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:39.369705 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.369588    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-tmp-dir\") pod \"rate-test-model-af2c94-kserve-58f746dcdc-x8vq2\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") " pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:39.369943 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.369904    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-home\") pod \"rate-test-model-af2c94-kserve-58f746dcdc-x8vq2\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") " pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:39.370002 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.369942    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-tmp-dir\") pod \"rate-test-model-af2c94-kserve-58f746dcdc-x8vq2\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") " pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:39.370002 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.369986    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-model-cache\") pod \"rate-test-model-af2c94-kserve-58f746dcdc-x8vq2\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") " pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:39.371930 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.371901    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-dshm\") pod \"rate-test-model-af2c94-kserve-58f746dcdc-x8vq2\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") " pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:39.372195 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.372176    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/021795c5-a9f6-4308-b919-fa7cb735e961-tls-certs\") pod \"rate-test-model-af2c94-kserve-58f746dcdc-x8vq2\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") " pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:39.377749 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.377731    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtstq\" (UniqueName: \"kubernetes.io/projected/021795c5-a9f6-4308-b919-fa7cb735e961-kube-api-access-qtstq\") pod \"rate-test-model-af2c94-kserve-58f746dcdc-x8vq2\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") " pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:39.504607 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.504576    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:39.630581 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:39.630557    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"]
Jul 07 22:52:39.632274 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:52:39.632245    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod021795c5_a9f6_4308_b919_fa7cb735e961.slice/crio-f743d772e68825e524436b9ffdb0d6134e1a50dbf8168b55d0a468f7c80a603f WatchSource:0}: Error finding container f743d772e68825e524436b9ffdb0d6134e1a50dbf8168b55d0a468f7c80a603f: Status 404 returned error can't find the container with id f743d772e68825e524436b9ffdb0d6134e1a50dbf8168b55d0a468f7c80a603f
Jul 07 22:52:40.012532 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:40.012488    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2" event={"ID":"021795c5-a9f6-4308-b919-fa7cb735e961","Type":"ContainerStarted","Data":"ea1e86264bfdd8a3c524b53b5089f93e27e4daa4a69f594768aac8b19d576f86"}
Jul 07 22:52:40.012709 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:40.012538    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2" event={"ID":"021795c5-a9f6-4308-b919-fa7cb735e961","Type":"ContainerStarted","Data":"f743d772e68825e524436b9ffdb0d6134e1a50dbf8168b55d0a468f7c80a603f"}
Jul 07 22:52:40.028578 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:40.028532    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2" podStartSLOduration=1.028518152 podStartE2EDuration="1.028518152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:52:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:52:40.027021513 +0000 UTC m=+2257.699240501" watchObservedRunningTime="2026-07-07 22:52:40.028518152 +0000 UTC m=+2257.700737139"
Jul 07 22:52:49.505499 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:49.505462    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:49.505499 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:49.505506    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:49.518355 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:49.518329    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:50.066890 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:50.066844    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:52:51.285488 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:51.285445    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r" podUID="f5868da3-7c3c-4401-b71b-a1aa09f82eb4" containerName="main" containerID="cri-o://de5cc55fa1bfe1dcaafa15c75e20117aee1c8ea0b0abce9a4715f66ebe7ecbff" gracePeriod=45
Jul 07 22:52:51.534010 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:51.533984    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:52:51.581252 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:51.581179    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ldtz\" (UniqueName: \"kubernetes.io/projected/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-kube-api-access-6ldtz\") pod \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") "
Jul 07 22:52:51.581402 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:51.581252    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-tmp-dir\") pod \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") "
Jul 07 22:52:51.581402 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:51.581337    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-model-cache\") pod \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") "
Jul 07 22:52:51.581402 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:51.581372    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-tls-certs\") pod \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") "
Jul 07 22:52:51.581573 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:51.581402    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-home\") pod \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") "
Jul 07 22:52:51.581573 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:51.581445    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-dshm\") pod \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\" (UID: \"f5868da3-7c3c-4401-b71b-a1aa09f82eb4\") "
Jul 07 22:52:51.581573 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:51.581513    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f5868da3-7c3c-4401-b71b-a1aa09f82eb4" (UID: "f5868da3-7c3c-4401-b71b-a1aa09f82eb4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:52:51.581736 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:51.581624    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-model-cache" (OuterVolumeSpecName: "model-cache") pod "f5868da3-7c3c-4401-b71b-a1aa09f82eb4" (UID: "f5868da3-7c3c-4401-b71b-a1aa09f82eb4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:52:51.581840 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:51.581816    2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-tmp-dir\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:52:51.581840 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:51.581843    2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-model-cache\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:52:51.582058 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:51.581921    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-home" (OuterVolumeSpecName: "home") pod "f5868da3-7c3c-4401-b71b-a1aa09f82eb4" (UID: "f5868da3-7c3c-4401-b71b-a1aa09f82eb4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:52:51.583720 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:51.583690    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-dshm" (OuterVolumeSpecName: "dshm") pod "f5868da3-7c3c-4401-b71b-a1aa09f82eb4" (UID: "f5868da3-7c3c-4401-b71b-a1aa09f82eb4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:52:51.583836 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:51.583821    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f5868da3-7c3c-4401-b71b-a1aa09f82eb4" (UID: "f5868da3-7c3c-4401-b71b-a1aa09f82eb4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 22:52:51.583951 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:51.583832    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-kube-api-access-6ldtz" (OuterVolumeSpecName: "kube-api-access-6ldtz") pod "f5868da3-7c3c-4401-b71b-a1aa09f82eb4" (UID: "f5868da3-7c3c-4401-b71b-a1aa09f82eb4"). InnerVolumeSpecName "kube-api-access-6ldtz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:52:51.683008 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:51.682976    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ldtz\" (UniqueName: \"kubernetes.io/projected/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-kube-api-access-6ldtz\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:52:51.683008 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:51.683002    2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-tls-certs\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:52:51.683008 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:51.683011    2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-home\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:52:51.683008 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:51.683018    2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5868da3-7c3c-4401-b71b-a1aa09f82eb4-dshm\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:52:52.065728 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:52.065696    2555 generic.go:356] "Generic (PLEG): container finished" podID="f5868da3-7c3c-4401-b71b-a1aa09f82eb4" containerID="de5cc55fa1bfe1dcaafa15c75e20117aee1c8ea0b0abce9a4715f66ebe7ecbff" exitCode=0
Jul 07 22:52:52.065929 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:52.065736    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"
Jul 07 22:52:52.065929 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:52.065740    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r" event={"ID":"f5868da3-7c3c-4401-b71b-a1aa09f82eb4","Type":"ContainerDied","Data":"de5cc55fa1bfe1dcaafa15c75e20117aee1c8ea0b0abce9a4715f66ebe7ecbff"}
Jul 07 22:52:52.065929 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:52.065830    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r" event={"ID":"f5868da3-7c3c-4401-b71b-a1aa09f82eb4","Type":"ContainerDied","Data":"ce460844bc12277d945d48ce8d692b91f997a5aa80e2a48459054bfe0d42c504"}
Jul 07 22:52:52.065929 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:52.065846    2555 scope.go:117] "RemoveContainer" containerID="de5cc55fa1bfe1dcaafa15c75e20117aee1c8ea0b0abce9a4715f66ebe7ecbff"
Jul 07 22:52:52.076826 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:52.076798    2555 scope.go:117] "RemoveContainer" containerID="de5cc55fa1bfe1dcaafa15c75e20117aee1c8ea0b0abce9a4715f66ebe7ecbff"
Jul 07 22:52:52.077229 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:52:52.077195    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de5cc55fa1bfe1dcaafa15c75e20117aee1c8ea0b0abce9a4715f66ebe7ecbff\": container with ID starting with de5cc55fa1bfe1dcaafa15c75e20117aee1c8ea0b0abce9a4715f66ebe7ecbff not found: ID does not exist" containerID="de5cc55fa1bfe1dcaafa15c75e20117aee1c8ea0b0abce9a4715f66ebe7ecbff"
Jul 07 22:52:52.077348 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:52.077239    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de5cc55fa1bfe1dcaafa15c75e20117aee1c8ea0b0abce9a4715f66ebe7ecbff"} err="failed to get container status \"de5cc55fa1bfe1dcaafa15c75e20117aee1c8ea0b0abce9a4715f66ebe7ecbff\": rpc error: code = NotFound desc = could not find container \"de5cc55fa1bfe1dcaafa15c75e20117aee1c8ea0b0abce9a4715f66ebe7ecbff\": container with ID starting with de5cc55fa1bfe1dcaafa15c75e20117aee1c8ea0b0abce9a4715f66ebe7ecbff not found: ID does not exist"
Jul 07 22:52:52.093215 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:52.093165    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"]
Jul 07 22:52:52.095126 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:52.095104    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-986e04/sub-test-model-986e04-kserve-75bb7b7bf9-gvz4r"]
Jul 07 22:52:52.922457 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:52:52.922421    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5868da3-7c3c-4401-b71b-a1aa09f82eb4" path="/var/lib/kubelet/pods/f5868da3-7c3c-4401-b71b-a1aa09f82eb4/volumes"
Jul 07 22:53:39.779577 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:39.779543    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"]
Jul 07 22:53:42.706899 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:42.706852    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg"]
Jul 07 22:53:42.707284 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:42.707132    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg" podUID="cd9e3607-a143-4704-a017-9533cd98c775" containerName="maas-api" containerID="cri-o://d4673f21c9bfed8ac9a2c6236955a9a631af17620e9584c470235cfd1283689e" gracePeriod=30
Jul 07 22:53:43.150313 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:43.150284    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg"
Jul 07 22:53:43.262291 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:43.262193    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd9e3607-a143-4704-a017-9533cd98c775-openshift-service-ca\") pod \"cd9e3607-a143-4704-a017-9533cd98c775\" (UID: \"cd9e3607-a143-4704-a017-9533cd98c775\") "
Jul 07 22:53:43.262450 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:43.262291    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25dfr\" (UniqueName: \"kubernetes.io/projected/cd9e3607-a143-4704-a017-9533cd98c775-kube-api-access-25dfr\") pod \"cd9e3607-a143-4704-a017-9533cd98c775\" (UID: \"cd9e3607-a143-4704-a017-9533cd98c775\") "
Jul 07 22:53:43.262450 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:43.262349    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd9e3607-a143-4704-a017-9533cd98c775-maas-api-tls\") pod \"cd9e3607-a143-4704-a017-9533cd98c775\" (UID: \"cd9e3607-a143-4704-a017-9533cd98c775\") "
Jul 07 22:53:43.262621 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:43.262590    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd9e3607-a143-4704-a017-9533cd98c775-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cd9e3607-a143-4704-a017-9533cd98c775" (UID: "cd9e3607-a143-4704-a017-9533cd98c775"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 22:53:43.264445 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:43.264419    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd9e3607-a143-4704-a017-9533cd98c775-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cd9e3607-a143-4704-a017-9533cd98c775" (UID: "cd9e3607-a143-4704-a017-9533cd98c775"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 22:53:43.264445 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:43.264425    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd9e3607-a143-4704-a017-9533cd98c775-kube-api-access-25dfr" (OuterVolumeSpecName: "kube-api-access-25dfr") pod "cd9e3607-a143-4704-a017-9533cd98c775" (UID: "cd9e3607-a143-4704-a017-9533cd98c775"). InnerVolumeSpecName "kube-api-access-25dfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:53:43.311100 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:43.311071    2555 generic.go:356] "Generic (PLEG): container finished" podID="cd9e3607-a143-4704-a017-9533cd98c775" containerID="d4673f21c9bfed8ac9a2c6236955a9a631af17620e9584c470235cfd1283689e" exitCode=0
Jul 07 22:53:43.311219 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:43.311111    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg"
Jul 07 22:53:43.311219 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:43.311152    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg" event={"ID":"cd9e3607-a143-4704-a017-9533cd98c775","Type":"ContainerDied","Data":"d4673f21c9bfed8ac9a2c6236955a9a631af17620e9584c470235cfd1283689e"}
Jul 07 22:53:43.311219 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:43.311203    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg" event={"ID":"cd9e3607-a143-4704-a017-9533cd98c775","Type":"ContainerDied","Data":"293f857ac18b7b2e2021f3a133d021a0aebcbcdc378484ad820ca7f6caaaa8de"}
Jul 07 22:53:43.311341 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:43.311227    2555 scope.go:117] "RemoveContainer" containerID="d4673f21c9bfed8ac9a2c6236955a9a631af17620e9584c470235cfd1283689e"
Jul 07 22:53:43.320066 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:43.320048    2555 scope.go:117] "RemoveContainer" containerID="d4673f21c9bfed8ac9a2c6236955a9a631af17620e9584c470235cfd1283689e"
Jul 07 22:53:43.320298 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:53:43.320281    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4673f21c9bfed8ac9a2c6236955a9a631af17620e9584c470235cfd1283689e\": container with ID starting with d4673f21c9bfed8ac9a2c6236955a9a631af17620e9584c470235cfd1283689e not found: ID does not exist" containerID="d4673f21c9bfed8ac9a2c6236955a9a631af17620e9584c470235cfd1283689e"
Jul 07 22:53:43.320340 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:43.320306    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4673f21c9bfed8ac9a2c6236955a9a631af17620e9584c470235cfd1283689e"} err="failed to get container status \"d4673f21c9bfed8ac9a2c6236955a9a631af17620e9584c470235cfd1283689e\": rpc error: code = NotFound desc = could not find container \"d4673f21c9bfed8ac9a2c6236955a9a631af17620e9584c470235cfd1283689e\": container with ID starting with d4673f21c9bfed8ac9a2c6236955a9a631af17620e9584c470235cfd1283689e not found: ID does not exist"
Jul 07 22:53:43.342279 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:43.342252    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg"]
Jul 07 22:53:43.354952 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:43.354934    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-af2c94-6dbc794f59-289cg"]
Jul 07 22:53:43.363764 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:43.363739    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd9e3607-a143-4704-a017-9533cd98c775-openshift-service-ca\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:53:43.363835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:43.363764    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25dfr\" (UniqueName: \"kubernetes.io/projected/cd9e3607-a143-4704-a017-9533cd98c775-kube-api-access-25dfr\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:53:43.363835 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:43.363777    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd9e3607-a143-4704-a017-9533cd98c775-maas-api-tls\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:53:44.922635 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:44.922534    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd9e3607-a143-4704-a017-9533cd98c775" path="/var/lib/kubelet/pods/cd9e3607-a143-4704-a017-9533cd98c775/volumes"
Jul 07 22:53:54.799210 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:54.799104    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2" podUID="021795c5-a9f6-4308-b919-fa7cb735e961" containerName="main" containerID="cri-o://ea1e86264bfdd8a3c524b53b5089f93e27e4daa4a69f594768aac8b19d576f86" gracePeriod=45
Jul 07 22:53:55.053248 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.053192    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:53:55.174634 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.174602    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-tmp-dir\") pod \"021795c5-a9f6-4308-b919-fa7cb735e961\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") "
Jul 07 22:53:55.174807 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.174658    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-dshm\") pod \"021795c5-a9f6-4308-b919-fa7cb735e961\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") "
Jul 07 22:53:55.174807 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.174701    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-home\") pod \"021795c5-a9f6-4308-b919-fa7cb735e961\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") "
Jul 07 22:53:55.174807 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.174724    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtstq\" (UniqueName: \"kubernetes.io/projected/021795c5-a9f6-4308-b919-fa7cb735e961-kube-api-access-qtstq\") pod \"021795c5-a9f6-4308-b919-fa7cb735e961\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") "
Jul 07 22:53:55.174977 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.174822    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/021795c5-a9f6-4308-b919-fa7cb735e961-tls-certs\") pod \"021795c5-a9f6-4308-b919-fa7cb735e961\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") "
Jul 07 22:53:55.174977 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.174856    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-model-cache\") pod \"021795c5-a9f6-4308-b919-fa7cb735e961\" (UID: \"021795c5-a9f6-4308-b919-fa7cb735e961\") "
Jul 07 22:53:55.174977 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.174928    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "021795c5-a9f6-4308-b919-fa7cb735e961" (UID: "021795c5-a9f6-4308-b919-fa7cb735e961"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:53:55.175140 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.175014    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-home" (OuterVolumeSpecName: "home") pod "021795c5-a9f6-4308-b919-fa7cb735e961" (UID: "021795c5-a9f6-4308-b919-fa7cb735e961"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:53:55.175199 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.175145    2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-tmp-dir\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:53:55.175199 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.175172    2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-home\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:53:55.175293 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.175210    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-model-cache" (OuterVolumeSpecName: "model-cache") pod "021795c5-a9f6-4308-b919-fa7cb735e961" (UID: "021795c5-a9f6-4308-b919-fa7cb735e961"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:53:55.176914 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.176850    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/021795c5-a9f6-4308-b919-fa7cb735e961-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "021795c5-a9f6-4308-b919-fa7cb735e961" (UID: "021795c5-a9f6-4308-b919-fa7cb735e961"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 22:53:55.176914 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.176864    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-dshm" (OuterVolumeSpecName: "dshm") pod "021795c5-a9f6-4308-b919-fa7cb735e961" (UID: "021795c5-a9f6-4308-b919-fa7cb735e961"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:53:55.177048 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.176923    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/021795c5-a9f6-4308-b919-fa7cb735e961-kube-api-access-qtstq" (OuterVolumeSpecName: "kube-api-access-qtstq") pod "021795c5-a9f6-4308-b919-fa7cb735e961" (UID: "021795c5-a9f6-4308-b919-fa7cb735e961"). InnerVolumeSpecName "kube-api-access-qtstq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:53:55.276233 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.276207    2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-model-cache\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:53:55.276233 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.276230    2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/021795c5-a9f6-4308-b919-fa7cb735e961-dshm\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:53:55.276233 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.276239    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtstq\" (UniqueName: \"kubernetes.io/projected/021795c5-a9f6-4308-b919-fa7cb735e961-kube-api-access-qtstq\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:53:55.276430 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.276249    2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/021795c5-a9f6-4308-b919-fa7cb735e961-tls-certs\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:53:55.365797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.365714    2555 generic.go:356] "Generic (PLEG): container finished" podID="021795c5-a9f6-4308-b919-fa7cb735e961" containerID="ea1e86264bfdd8a3c524b53b5089f93e27e4daa4a69f594768aac8b19d576f86" exitCode=0
Jul 07 22:53:55.365797 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.365755    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"
Jul 07 22:53:55.366010 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.365795    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2" event={"ID":"021795c5-a9f6-4308-b919-fa7cb735e961","Type":"ContainerDied","Data":"ea1e86264bfdd8a3c524b53b5089f93e27e4daa4a69f594768aac8b19d576f86"}
Jul 07 22:53:55.366010 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.365838    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2" event={"ID":"021795c5-a9f6-4308-b919-fa7cb735e961","Type":"ContainerDied","Data":"f743d772e68825e524436b9ffdb0d6134e1a50dbf8168b55d0a468f7c80a603f"}
Jul 07 22:53:55.366010 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.365855    2555 scope.go:117] "RemoveContainer" containerID="ea1e86264bfdd8a3c524b53b5089f93e27e4daa4a69f594768aac8b19d576f86"
Jul 07 22:53:55.376080 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.376058    2555 scope.go:117] "RemoveContainer" containerID="ea1e86264bfdd8a3c524b53b5089f93e27e4daa4a69f594768aac8b19d576f86"
Jul 07 22:53:55.376816 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:53:55.376790    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea1e86264bfdd8a3c524b53b5089f93e27e4daa4a69f594768aac8b19d576f86\": container with ID starting with ea1e86264bfdd8a3c524b53b5089f93e27e4daa4a69f594768aac8b19d576f86 not found: ID does not exist" containerID="ea1e86264bfdd8a3c524b53b5089f93e27e4daa4a69f594768aac8b19d576f86"
Jul 07 22:53:55.376885 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.376828    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea1e86264bfdd8a3c524b53b5089f93e27e4daa4a69f594768aac8b19d576f86"} err="failed to get container status \"ea1e86264bfdd8a3c524b53b5089f93e27e4daa4a69f594768aac8b19d576f86\": rpc error: code = NotFound desc = could not find container \"ea1e86264bfdd8a3c524b53b5089f93e27e4daa4a69f594768aac8b19d576f86\": container with ID starting with ea1e86264bfdd8a3c524b53b5089f93e27e4daa4a69f594768aac8b19d576f86 not found: ID does not exist"
Jul 07 22:53:55.388897 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.388856    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"]
Jul 07 22:53:55.393243 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:55.393217    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-af2c94/rate-test-model-af2c94-kserve-58f746dcdc-x8vq2"]
Jul 07 22:53:56.922576 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:53:56.922543    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="021795c5-a9f6-4308-b919-fa7cb735e961" path="/var/lib/kubelet/pods/021795c5-a9f6-4308-b919-fa7cb735e961/volumes"
Jul 07 22:54:24.813628 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:24.813597    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"]
Jul 07 22:54:24.814161 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:24.813814    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv" podUID="df21bb96-43a4-4a70-a23a-11b2c8630e64" containerName="istio-proxy" containerID="cri-o://ddcd7e532b450754113abe1613bbe93a47aeb948f2e115c973b7b84b7a3c9f9d" gracePeriod=30
Jul 07 22:54:25.482987 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:25.482952    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"]
Jul 07 22:54:25.483241 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:25.483196    2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w" podUID="3196425d-00b5-40b1-84f6-7477c60b105d" containerName="maas-api" containerID="cri-o://b36c98b5a7e309858115d86e943058186c8d4172d493dfc2467a3f33f99cf5bc" gracePeriod=30
Jul 07 22:54:25.924932 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:25.924910    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"
Jul 07 22:54:26.051554 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:26.051478    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3196425d-00b5-40b1-84f6-7477c60b105d-maas-api-tls\") pod \"3196425d-00b5-40b1-84f6-7477c60b105d\" (UID: \"3196425d-00b5-40b1-84f6-7477c60b105d\") "
Jul 07 22:54:26.051698 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:26.051570    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3196425d-00b5-40b1-84f6-7477c60b105d-openshift-service-ca\") pod \"3196425d-00b5-40b1-84f6-7477c60b105d\" (UID: \"3196425d-00b5-40b1-84f6-7477c60b105d\") "
Jul 07 22:54:26.051698 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:26.051601    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkhsh\" (UniqueName: \"kubernetes.io/projected/3196425d-00b5-40b1-84f6-7477c60b105d-kube-api-access-vkhsh\") pod \"3196425d-00b5-40b1-84f6-7477c60b105d\" (UID: \"3196425d-00b5-40b1-84f6-7477c60b105d\") "
Jul 07 22:54:26.051981 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:26.051951    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3196425d-00b5-40b1-84f6-7477c60b105d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3196425d-00b5-40b1-84f6-7477c60b105d" (UID: "3196425d-00b5-40b1-84f6-7477c60b105d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 22:54:26.053511 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:26.053490    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3196425d-00b5-40b1-84f6-7477c60b105d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3196425d-00b5-40b1-84f6-7477c60b105d" (UID: "3196425d-00b5-40b1-84f6-7477c60b105d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 22:54:26.053611 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:26.053589    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3196425d-00b5-40b1-84f6-7477c60b105d-kube-api-access-vkhsh" (OuterVolumeSpecName: "kube-api-access-vkhsh") pod "3196425d-00b5-40b1-84f6-7477c60b105d" (UID: "3196425d-00b5-40b1-84f6-7477c60b105d"). InnerVolumeSpecName "kube-api-access-vkhsh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:54:26.152335 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:26.152289    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3196425d-00b5-40b1-84f6-7477c60b105d-maas-api-tls\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:54:26.152335 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:26.152328    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3196425d-00b5-40b1-84f6-7477c60b105d-openshift-service-ca\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:54:26.152335 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:26.152341    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkhsh\" (UniqueName: \"kubernetes.io/projected/3196425d-00b5-40b1-84f6-7477c60b105d-kube-api-access-vkhsh\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:54:26.499030 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:26.499001    2555 generic.go:356] "Generic (PLEG): container finished" podID="3196425d-00b5-40b1-84f6-7477c60b105d" containerID="b36c98b5a7e309858115d86e943058186c8d4172d493dfc2467a3f33f99cf5bc" exitCode=0
Jul 07 22:54:26.499207 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:26.499043    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"
Jul 07 22:54:26.499207 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:26.499086    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w" event={"ID":"3196425d-00b5-40b1-84f6-7477c60b105d","Type":"ContainerDied","Data":"b36c98b5a7e309858115d86e943058186c8d4172d493dfc2467a3f33f99cf5bc"}
Jul 07 22:54:26.499207 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:26.499140    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w" event={"ID":"3196425d-00b5-40b1-84f6-7477c60b105d","Type":"ContainerDied","Data":"5d0e695938fc5598028a0f139038e14892b6c420543f6913463ff44a7d3e13ed"}
Jul 07 22:54:26.499207 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:26.499162    2555 scope.go:117] "RemoveContainer" containerID="b36c98b5a7e309858115d86e943058186c8d4172d493dfc2467a3f33f99cf5bc"
Jul 07 22:54:26.508586 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:26.508569    2555 scope.go:117] "RemoveContainer" containerID="b36c98b5a7e309858115d86e943058186c8d4172d493dfc2467a3f33f99cf5bc"
Jul 07 22:54:26.508837 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:54:26.508820    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b36c98b5a7e309858115d86e943058186c8d4172d493dfc2467a3f33f99cf5bc\": container with ID starting with b36c98b5a7e309858115d86e943058186c8d4172d493dfc2467a3f33f99cf5bc not found: ID does not exist" containerID="b36c98b5a7e309858115d86e943058186c8d4172d493dfc2467a3f33f99cf5bc"
Jul 07 22:54:26.508917 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:26.508845    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b36c98b5a7e309858115d86e943058186c8d4172d493dfc2467a3f33f99cf5bc"} err="failed to get container status \"b36c98b5a7e309858115d86e943058186c8d4172d493dfc2467a3f33f99cf5bc\": rpc error: code = NotFound desc = could not find container \"b36c98b5a7e309858115d86e943058186c8d4172d493dfc2467a3f33f99cf5bc\": container with ID starting with b36c98b5a7e309858115d86e943058186c8d4172d493dfc2467a3f33f99cf5bc not found: ID does not exist"
Jul 07 22:54:26.521053 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:26.521024    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"]
Jul 07 22:54:26.528441 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:26.528415    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-986e04-6cd4bf847c-dqj9w"]
Jul 07 22:54:26.922403 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:26.922326    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3196425d-00b5-40b1-84f6-7477c60b105d" path="/var/lib/kubelet/pods/3196425d-00b5-40b1-84f6-7477c60b105d/volumes"
Jul 07 22:54:30.061761 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.061737    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:54:30.191240 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.191148    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-podinfo\") pod \"df21bb96-43a4-4a70-a23a-11b2c8630e64\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") "
Jul 07 22:54:30.191240 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.191197    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-credential-socket\") pod \"df21bb96-43a4-4a70-a23a-11b2c8630e64\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") "
Jul 07 22:54:30.191482 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.191245    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df21bb96-43a4-4a70-a23a-11b2c8630e64-istiod-ca-cert\") pod \"df21bb96-43a4-4a70-a23a-11b2c8630e64\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") "
Jul 07 22:54:30.191482 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.191276    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thfxr\" (UniqueName: \"kubernetes.io/projected/df21bb96-43a4-4a70-a23a-11b2c8630e64-kube-api-access-thfxr\") pod \"df21bb96-43a4-4a70-a23a-11b2c8630e64\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") "
Jul 07 22:54:30.191482 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.191302    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-data\") pod \"df21bb96-43a4-4a70-a23a-11b2c8630e64\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") "
Jul 07 22:54:30.191482 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.191336    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-workload-socket\") pod \"df21bb96-43a4-4a70-a23a-11b2c8630e64\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") "
Jul 07 22:54:30.191482 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.191364    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-envoy\") pod \"df21bb96-43a4-4a70-a23a-11b2c8630e64\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") "
Jul 07 22:54:30.191482 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.191396    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-token\") pod \"df21bb96-43a4-4a70-a23a-11b2c8630e64\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") "
Jul 07 22:54:30.191482 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.191427    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-workload-certs\") pod \"df21bb96-43a4-4a70-a23a-11b2c8630e64\" (UID: \"df21bb96-43a4-4a70-a23a-11b2c8630e64\") "
Jul 07 22:54:30.191848 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.191579    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "df21bb96-43a4-4a70-a23a-11b2c8630e64" (UID: "df21bb96-43a4-4a70-a23a-11b2c8630e64"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:54:30.191848 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.191599    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "df21bb96-43a4-4a70-a23a-11b2c8630e64" (UID: "df21bb96-43a4-4a70-a23a-11b2c8630e64"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:54:30.191848 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.191752    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-workload-socket\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:54:30.191848 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.191782    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-credential-socket\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:54:30.191848 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.191806    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "df21bb96-43a4-4a70-a23a-11b2c8630e64" (UID: "df21bb96-43a4-4a70-a23a-11b2c8630e64"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:54:30.192152 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.191966    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df21bb96-43a4-4a70-a23a-11b2c8630e64-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "df21bb96-43a4-4a70-a23a-11b2c8630e64" (UID: "df21bb96-43a4-4a70-a23a-11b2c8630e64"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 22:54:30.192323 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.192291    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-data" (OuterVolumeSpecName: "istio-data") pod "df21bb96-43a4-4a70-a23a-11b2c8630e64" (UID: "df21bb96-43a4-4a70-a23a-11b2c8630e64"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:54:30.193594 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.193559    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "df21bb96-43a4-4a70-a23a-11b2c8630e64" (UID: "df21bb96-43a4-4a70-a23a-11b2c8630e64"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 22:54:30.193594 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.193578    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-token" (OuterVolumeSpecName: "istio-token") pod "df21bb96-43a4-4a70-a23a-11b2c8630e64" (UID: "df21bb96-43a4-4a70-a23a-11b2c8630e64"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:54:30.193729 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.193599    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "df21bb96-43a4-4a70-a23a-11b2c8630e64" (UID: "df21bb96-43a4-4a70-a23a-11b2c8630e64"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 22:54:30.193729 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.193630    2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df21bb96-43a4-4a70-a23a-11b2c8630e64-kube-api-access-thfxr" (OuterVolumeSpecName: "kube-api-access-thfxr") pod "df21bb96-43a4-4a70-a23a-11b2c8630e64" (UID: "df21bb96-43a4-4a70-a23a-11b2c8630e64"). InnerVolumeSpecName "kube-api-access-thfxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 22:54:30.292555 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.292526    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-envoy\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:54:30.292555 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.292550    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-token\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:54:30.292727 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.292579    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-workload-certs\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:54:30.292727 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.292589    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-podinfo\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:54:30.292727 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.292597    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df21bb96-43a4-4a70-a23a-11b2c8630e64-istiod-ca-cert\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:54:30.292727 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.292606    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thfxr\" (UniqueName: \"kubernetes.io/projected/df21bb96-43a4-4a70-a23a-11b2c8630e64-kube-api-access-thfxr\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:54:30.292727 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.292616    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df21bb96-43a4-4a70-a23a-11b2c8630e64-istio-data\") on node \"ip-10-0-128-159.ec2.internal\" DevicePath \"\""
Jul 07 22:54:30.519446 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.519415    2555 generic.go:356] "Generic (PLEG): container finished" podID="df21bb96-43a4-4a70-a23a-11b2c8630e64" containerID="ddcd7e532b450754113abe1613bbe93a47aeb948f2e115c973b7b84b7a3c9f9d" exitCode=0
Jul 07 22:54:30.519609 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.519462    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"
Jul 07 22:54:30.519609 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.519475    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv" event={"ID":"df21bb96-43a4-4a70-a23a-11b2c8630e64","Type":"ContainerDied","Data":"ddcd7e532b450754113abe1613bbe93a47aeb948f2e115c973b7b84b7a3c9f9d"}
Jul 07 22:54:30.519609 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.519542    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv" event={"ID":"df21bb96-43a4-4a70-a23a-11b2c8630e64","Type":"ContainerDied","Data":"a74008b622b53f59c708094a0f18613c6469979407cc4a344b5c4c3b8793d721"}
Jul 07 22:54:30.519609 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.519564    2555 scope.go:117] "RemoveContainer" containerID="ddcd7e532b450754113abe1613bbe93a47aeb948f2e115c973b7b84b7a3c9f9d"
Jul 07 22:54:30.529108 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.529091    2555 scope.go:117] "RemoveContainer" containerID="755f1fe663bea616eab65f8a20cbc64dc2025325e35e8afc25761277937adf04"
Jul 07 22:54:30.536638 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.536617    2555 scope.go:117] "RemoveContainer" containerID="ddcd7e532b450754113abe1613bbe93a47aeb948f2e115c973b7b84b7a3c9f9d"
Jul 07 22:54:30.536913 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:54:30.536887    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddcd7e532b450754113abe1613bbe93a47aeb948f2e115c973b7b84b7a3c9f9d\": container with ID starting with ddcd7e532b450754113abe1613bbe93a47aeb948f2e115c973b7b84b7a3c9f9d not found: ID does not exist" containerID="ddcd7e532b450754113abe1613bbe93a47aeb948f2e115c973b7b84b7a3c9f9d"
Jul 07 22:54:30.537000 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.536924    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddcd7e532b450754113abe1613bbe93a47aeb948f2e115c973b7b84b7a3c9f9d"} err="failed to get container status \"ddcd7e532b450754113abe1613bbe93a47aeb948f2e115c973b7b84b7a3c9f9d\": rpc error: code = NotFound desc = could not find container \"ddcd7e532b450754113abe1613bbe93a47aeb948f2e115c973b7b84b7a3c9f9d\": container with ID starting with ddcd7e532b450754113abe1613bbe93a47aeb948f2e115c973b7b84b7a3c9f9d not found: ID does not exist"
Jul 07 22:54:30.537000 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.536948    2555 scope.go:117] "RemoveContainer" containerID="755f1fe663bea616eab65f8a20cbc64dc2025325e35e8afc25761277937adf04"
Jul 07 22:54:30.537160 ip-10-0-128-159 kubenswrapper[2555]: E0707 22:54:30.537145    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"755f1fe663bea616eab65f8a20cbc64dc2025325e35e8afc25761277937adf04\": container with ID starting with 755f1fe663bea616eab65f8a20cbc64dc2025325e35e8afc25761277937adf04 not found: ID does not exist" containerID="755f1fe663bea616eab65f8a20cbc64dc2025325e35e8afc25761277937adf04"
Jul 07 22:54:30.537203 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.537164    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"755f1fe663bea616eab65f8a20cbc64dc2025325e35e8afc25761277937adf04"} err="failed to get container status \"755f1fe663bea616eab65f8a20cbc64dc2025325e35e8afc25761277937adf04\": rpc error: code = NotFound desc = could not find container \"755f1fe663bea616eab65f8a20cbc64dc2025325e35e8afc25761277937adf04\": container with ID starting with 755f1fe663bea616eab65f8a20cbc64dc2025325e35e8afc25761277937adf04 not found: ID does not exist"
Jul 07 22:54:30.541962 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.541942    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"]
Jul 07 22:54:30.545573 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.545553    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-af2c94-openshift-default-898788bf9-6w4pv"]
Jul 07 22:54:30.922183 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:30.922103    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df21bb96-43a4-4a70-a23a-11b2c8630e64" path="/var/lib/kubelet/pods/df21bb96-43a4-4a70-a23a-11b2c8630e64/volumes"
Jul 07 22:54:32.594442 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:32.594414    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-846f7dc586-zbgh4_1c099867-be01-4483-b206-670e283e6880/authorino/0.log"
Jul 07 22:54:37.752221 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:37.752191    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-82b4p_ff7041e3-a577-42d2-ba13-27baf97a5b02/server/0.log"
Jul 07 22:54:37.977789 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:37.977764    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg_f58441f9-4810-4b47-b7c7-5b700dc11727/copy-manifests/0.log"
Jul 07 22:54:37.985832 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:37.985801    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg_f58441f9-4810-4b47-b7c7-5b700dc11727/manager/0.log"
Jul 07 22:54:39.620055 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:39.620025    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-846f7dc586-zbgh4_1c099867-be01-4483-b206-670e283e6880/authorino/0.log"
Jul 07 22:54:39.960827 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:39.960804    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-skpm9_7aa4edef-63b9-49ee-ba93-9637cbde4b50/kuadrant-console-plugin/0.log"
Jul 07 22:54:41.114464 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:41.114421    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-776659d659-w7qkt_f615c2a4-314a-413d-a879-6c8c38d9b332/kube-auth-proxy/0.log"
Jul 07 22:54:42.040466 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:42.040429    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-n7m74_0df0bb34-ea82-4f69-bf02-11d07c910f4d/main/0.log"
Jul 07 22:54:42.379281 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:42.379201    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-8rljm_889bf831-e958-4021-a98a-6c3c0375924a/main/0.log"
Jul 07 22:54:42.488486 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:42.488451    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-h4qkl_f7192764-ed13-4253-8ec6-ea3c845bb27a/main/0.log"
Jul 07 22:54:46.653802 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.653768    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xs5cp/must-gather-xrjf2"]
Jul 07 22:54:46.654185 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.654081    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd9e3607-a143-4704-a017-9533cd98c775" containerName="maas-api"
Jul 07 22:54:46.654185 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.654093    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd9e3607-a143-4704-a017-9533cd98c775" containerName="maas-api"
Jul 07 22:54:46.654185 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.654104    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df21bb96-43a4-4a70-a23a-11b2c8630e64" containerName="istio-proxy"
Jul 07 22:54:46.654185 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.654111    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="df21bb96-43a4-4a70-a23a-11b2c8630e64" containerName="istio-proxy"
Jul 07 22:54:46.654185 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.654119    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5868da3-7c3c-4401-b71b-a1aa09f82eb4" containerName="main"
Jul 07 22:54:46.654185 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.654125    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5868da3-7c3c-4401-b71b-a1aa09f82eb4" containerName="main"
Jul 07 22:54:46.654185 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.654137    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3196425d-00b5-40b1-84f6-7477c60b105d" containerName="maas-api"
Jul 07 22:54:46.654185 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.654144    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3196425d-00b5-40b1-84f6-7477c60b105d" containerName="maas-api"
Jul 07 22:54:46.654185 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.654159    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df21bb96-43a4-4a70-a23a-11b2c8630e64" containerName="istio-proxy"
Jul 07 22:54:46.654185 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.654167    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="df21bb96-43a4-4a70-a23a-11b2c8630e64" containerName="istio-proxy"
Jul 07 22:54:46.654185 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.654179    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="021795c5-a9f6-4308-b919-fa7cb735e961" containerName="main"
Jul 07 22:54:46.654185 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.654187    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="021795c5-a9f6-4308-b919-fa7cb735e961" containerName="main"
Jul 07 22:54:46.654640 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.654249    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="df21bb96-43a4-4a70-a23a-11b2c8630e64" containerName="istio-proxy"
Jul 07 22:54:46.654640 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.654258    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3196425d-00b5-40b1-84f6-7477c60b105d" containerName="maas-api"
Jul 07 22:54:46.654640 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.654265    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd9e3607-a143-4704-a017-9533cd98c775" containerName="maas-api"
Jul 07 22:54:46.654640 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.654271    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5868da3-7c3c-4401-b71b-a1aa09f82eb4" containerName="main"
Jul 07 22:54:46.654640 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.654278    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="021795c5-a9f6-4308-b919-fa7cb735e961" containerName="main"
Jul 07 22:54:46.654640 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.654392    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="df21bb96-43a4-4a70-a23a-11b2c8630e64" containerName="istio-proxy"
Jul 07 22:54:46.657109 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.657092    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xs5cp/must-gather-xrjf2"
Jul 07 22:54:46.659554 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.659533    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xs5cp\"/\"kube-root-ca.crt\""
Jul 07 22:54:46.664116 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.660096    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xs5cp\"/\"openshift-service-ca.crt\""
Jul 07 22:54:46.664116 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.660133    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xs5cp\"/\"default-dockercfg-7sh64\""
Jul 07 22:54:46.668731 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.668705    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xs5cp/must-gather-xrjf2"]
Jul 07 22:54:46.737265 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.737233    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5xnc\" (UniqueName: \"kubernetes.io/projected/b83e599a-80a3-4e48-90e2-5257f408b13e-kube-api-access-w5xnc\") pod \"must-gather-xrjf2\" (UID: \"b83e599a-80a3-4e48-90e2-5257f408b13e\") " pod="openshift-must-gather-xs5cp/must-gather-xrjf2"
Jul 07 22:54:46.737441 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.737293    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b83e599a-80a3-4e48-90e2-5257f408b13e-must-gather-output\") pod \"must-gather-xrjf2\" (UID: \"b83e599a-80a3-4e48-90e2-5257f408b13e\") " pod="openshift-must-gather-xs5cp/must-gather-xrjf2"
Jul 07 22:54:46.838228 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.838196    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b83e599a-80a3-4e48-90e2-5257f408b13e-must-gather-output\") pod \"must-gather-xrjf2\" (UID: \"b83e599a-80a3-4e48-90e2-5257f408b13e\") " pod="openshift-must-gather-xs5cp/must-gather-xrjf2"
Jul 07 22:54:46.838401 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.838274    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5xnc\" (UniqueName: \"kubernetes.io/projected/b83e599a-80a3-4e48-90e2-5257f408b13e-kube-api-access-w5xnc\") pod \"must-gather-xrjf2\" (UID: \"b83e599a-80a3-4e48-90e2-5257f408b13e\") " pod="openshift-must-gather-xs5cp/must-gather-xrjf2"
Jul 07 22:54:46.838548 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.838527    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b83e599a-80a3-4e48-90e2-5257f408b13e-must-gather-output\") pod \"must-gather-xrjf2\" (UID: \"b83e599a-80a3-4e48-90e2-5257f408b13e\") " pod="openshift-must-gather-xs5cp/must-gather-xrjf2"
Jul 07 22:54:46.848260 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.848232    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5xnc\" (UniqueName: \"kubernetes.io/projected/b83e599a-80a3-4e48-90e2-5257f408b13e-kube-api-access-w5xnc\") pod \"must-gather-xrjf2\" (UID: \"b83e599a-80a3-4e48-90e2-5257f408b13e\") " pod="openshift-must-gather-xs5cp/must-gather-xrjf2"
Jul 07 22:54:46.972428 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:46.972402    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xs5cp/must-gather-xrjf2"
Jul 07 22:54:47.097762 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:47.097738    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xs5cp/must-gather-xrjf2"]
Jul 07 22:54:47.099525 ip-10-0-128-159 kubenswrapper[2555]: W0707 22:54:47.099502    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb83e599a_80a3_4e48_90e2_5257f408b13e.slice/crio-7b8b857b7ec2dad9ad3fba5ca32fff0cd0ef0322f4417ef055cb625f57c07547 WatchSource:0}: Error finding container 7b8b857b7ec2dad9ad3fba5ca32fff0cd0ef0322f4417ef055cb625f57c07547: Status 404 returned error can't find the container with id 7b8b857b7ec2dad9ad3fba5ca32fff0cd0ef0322f4417ef055cb625f57c07547
Jul 07 22:54:47.101360 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:47.101340    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 22:54:47.591932 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:47.591901    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xs5cp/must-gather-xrjf2" event={"ID":"b83e599a-80a3-4e48-90e2-5257f408b13e","Type":"ContainerStarted","Data":"7b8b857b7ec2dad9ad3fba5ca32fff0cd0ef0322f4417ef055cb625f57c07547"}
Jul 07 22:54:48.600481 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:48.600437    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xs5cp/must-gather-xrjf2" event={"ID":"b83e599a-80a3-4e48-90e2-5257f408b13e","Type":"ContainerStarted","Data":"c0fd046e3cf30677683dcd2c3856d85018ddce2f276e1579e9d54e8986f04439"}
Jul 07 22:54:48.600997 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:48.600497    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xs5cp/must-gather-xrjf2" event={"ID":"b83e599a-80a3-4e48-90e2-5257f408b13e","Type":"ContainerStarted","Data":"13114b9fa569abed767d999883492798bf9eefc69bb5c4221292d4d884736943"}
Jul 07 22:54:48.617106 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:48.617045    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xs5cp/must-gather-xrjf2" podStartSLOduration=1.873243518 podStartE2EDuration="2.617022601s" totalImagesPullingTime="743.779083ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:54:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:54:48.615617098 +0000 UTC m=+2386.287836088" watchObservedRunningTime="2026-07-07 22:54:48.617022601 +0000 UTC m=+2386.289241590"
Jul 07 22:54:49.470992 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:49.470959    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hdtt4_4128cbcd-5ec2-40d9-81fa-9661440e82df/global-pull-secret-syncer/0.log"
Jul 07 22:54:49.607142 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:49.607112    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hgwst_1155169f-bea8-43e4-a927-668ec8c8e77f/konnectivity-agent/0.log"
Jul 07 22:54:49.660352 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:49.660325    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-159.ec2.internal_f99cc9ba576df91ea14d32094adce059/haproxy/0.log"
Jul 07 22:54:54.568050 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:54.567813    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-846f7dc586-zbgh4_1c099867-be01-4483-b206-670e283e6880/authorino/0.log"
Jul 07 22:54:54.691005 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:54.690978    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-skpm9_7aa4edef-63b9-49ee-ba93-9637cbde4b50/kuadrant-console-plugin/0.log"
Jul 07 22:54:56.562548 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:56.562510    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_358c3c21-d142-490a-a10d-9ef15de87b62/alertmanager/0.log"
Jul 07 22:54:56.591356 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:56.591324    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_358c3c21-d142-490a-a10d-9ef15de87b62/config-reloader/0.log"
Jul 07 22:54:56.625543 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:56.625513    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_358c3c21-d142-490a-a10d-9ef15de87b62/kube-rbac-proxy-web/0.log"
Jul 07 22:54:56.664143 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:56.664118    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_358c3c21-d142-490a-a10d-9ef15de87b62/kube-rbac-proxy/0.log"
Jul 07 22:54:56.692465 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:56.692439    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_358c3c21-d142-490a-a10d-9ef15de87b62/kube-rbac-proxy-metric/0.log"
Jul 07 22:54:56.719354 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:56.719332    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_358c3c21-d142-490a-a10d-9ef15de87b62/prom-label-proxy/0.log"
Jul 07 22:54:56.754362 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:56.754339    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_358c3c21-d142-490a-a10d-9ef15de87b62/init-config-reloader/0.log"
Jul 07 22:54:56.932742 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:56.932566    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5fcd4db656-kdf9t_9291038d-df5f-4bac-8167-1449e10584aa/metrics-server/0.log"
Jul 07 22:54:56.965343 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:56.965311    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-lrgk5_177bad92-0c32-4217-8632-80e203931851/monitoring-plugin/0.log"
Jul 07 22:54:57.076008 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:57.075976    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mjfl4_21ea1eda-39bd-4514-b0b2-46e947c338f3/node-exporter/0.log"
Jul 07 22:54:57.100736 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:57.100655    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mjfl4_21ea1eda-39bd-4514-b0b2-46e947c338f3/kube-rbac-proxy/0.log"
Jul 07 22:54:57.121988 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:57.121961    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mjfl4_21ea1eda-39bd-4514-b0b2-46e947c338f3/init-textfile/0.log"
Jul 07 22:54:57.335799 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:57.335768    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2b157f2f-470a-4f09-a74f-6cc238145567/prometheus/0.log"
Jul 07 22:54:57.356689 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:57.356612    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2b157f2f-470a-4f09-a74f-6cc238145567/config-reloader/0.log"
Jul 07 22:54:57.378277 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:57.378245    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2b157f2f-470a-4f09-a74f-6cc238145567/thanos-sidecar/0.log"
Jul 07 22:54:57.402083 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:57.402055    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2b157f2f-470a-4f09-a74f-6cc238145567/kube-rbac-proxy-web/0.log"
Jul 07 22:54:57.424480 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:57.424449    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2b157f2f-470a-4f09-a74f-6cc238145567/kube-rbac-proxy/0.log"
Jul 07 22:54:57.448949 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:57.448918    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2b157f2f-470a-4f09-a74f-6cc238145567/kube-rbac-proxy-thanos/0.log"
Jul 07 22:54:57.472679 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:57.472645    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2b157f2f-470a-4f09-a74f-6cc238145567/init-config-reloader/0.log"
Jul 07 22:54:57.502192 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:57.502151    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-7gzb4_a14cc856-5969-4f2d-866d-96c609b5e2cc/prometheus-operator/0.log"
Jul 07 22:54:57.522509 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:57.522477    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-7gzb4_a14cc856-5969-4f2d-866d-96c609b5e2cc/kube-rbac-proxy/0.log"
Jul 07 22:54:59.436839 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:59.436787    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"]
Jul 07 22:54:59.442376 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:59.442352    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"
Jul 07 22:54:59.449446 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:59.449093    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"]
Jul 07 22:54:59.584223 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:59.584183    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/41917d67-01f5-4122-bb42-6157715e310a-podres\") pod \"perf-node-gather-daemonset-74qc9\" (UID: \"41917d67-01f5-4122-bb42-6157715e310a\") " pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"
Jul 07 22:54:59.584428 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:59.584254    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/41917d67-01f5-4122-bb42-6157715e310a-lib-modules\") pod \"perf-node-gather-daemonset-74qc9\" (UID: \"41917d67-01f5-4122-bb42-6157715e310a\") " pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"
Jul 07 22:54:59.584428 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:59.584284    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/41917d67-01f5-4122-bb42-6157715e310a-proc\") pod \"perf-node-gather-daemonset-74qc9\" (UID: \"41917d67-01f5-4122-bb42-6157715e310a\") " pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"
Jul 07 22:54:59.584428 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:59.584344    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41917d67-01f5-4122-bb42-6157715e310a-sys\") pod \"perf-node-gather-daemonset-74qc9\" (UID: \"41917d67-01f5-4122-bb42-6157715e310a\") " pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"
Jul 07 22:54:59.584428 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:59.584404    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc776\" (UniqueName: \"kubernetes.io/projected/41917d67-01f5-4122-bb42-6157715e310a-kube-api-access-wc776\") pod \"perf-node-gather-daemonset-74qc9\" (UID: \"41917d67-01f5-4122-bb42-6157715e310a\") " pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"
Jul 07 22:54:59.685883 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:59.685826    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc776\" (UniqueName: \"kubernetes.io/projected/41917d67-01f5-4122-bb42-6157715e310a-kube-api-access-wc776\") pod \"perf-node-gather-daemonset-74qc9\" (UID: \"41917d67-01f5-4122-bb42-6157715e310a\") " pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"
Jul 07 22:54:59.686077 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:59.685924    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/41917d67-01f5-4122-bb42-6157715e310a-podres\") pod \"perf-node-gather-daemonset-74qc9\" (UID: \"41917d67-01f5-4122-bb42-6157715e310a\") " pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"
Jul 07 22:54:59.686077 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:59.685953    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/41917d67-01f5-4122-bb42-6157715e310a-lib-modules\") pod \"perf-node-gather-daemonset-74qc9\" (UID: \"41917d67-01f5-4122-bb42-6157715e310a\") " pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"
Jul 07 22:54:59.686077 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:59.685980    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/41917d67-01f5-4122-bb42-6157715e310a-proc\") pod \"perf-node-gather-daemonset-74qc9\" (UID: \"41917d67-01f5-4122-bb42-6157715e310a\") " pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"
Jul 07 22:54:59.686077 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:59.686039    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41917d67-01f5-4122-bb42-6157715e310a-sys\") pod \"perf-node-gather-daemonset-74qc9\" (UID: \"41917d67-01f5-4122-bb42-6157715e310a\") " pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"
Jul 07 22:54:59.686328 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:59.686160    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41917d67-01f5-4122-bb42-6157715e310a-sys\") pod \"perf-node-gather-daemonset-74qc9\" (UID: \"41917d67-01f5-4122-bb42-6157715e310a\") " pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"
Jul 07 22:54:59.686601 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:59.686576    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/41917d67-01f5-4122-bb42-6157715e310a-podres\") pod \"perf-node-gather-daemonset-74qc9\" (UID: \"41917d67-01f5-4122-bb42-6157715e310a\") " pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"
Jul 07 22:54:59.686695 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:59.686676    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/41917d67-01f5-4122-bb42-6157715e310a-lib-modules\") pod \"perf-node-gather-daemonset-74qc9\" (UID: \"41917d67-01f5-4122-bb42-6157715e310a\") " pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"
Jul 07 22:54:59.686764 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:59.686722    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/41917d67-01f5-4122-bb42-6157715e310a-proc\") pod \"perf-node-gather-daemonset-74qc9\" (UID: \"41917d67-01f5-4122-bb42-6157715e310a\") " pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"
Jul 07 22:54:59.695248 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:59.695176    2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc776\" (UniqueName: \"kubernetes.io/projected/41917d67-01f5-4122-bb42-6157715e310a-kube-api-access-wc776\") pod \"perf-node-gather-daemonset-74qc9\" (UID: \"41917d67-01f5-4122-bb42-6157715e310a\") " pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"
Jul 07 22:54:59.756384 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:59.756344    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"
Jul 07 22:54:59.946386 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:54:59.946122    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"]
Jul 07 22:55:00.676588 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:00.676547    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9" event={"ID":"41917d67-01f5-4122-bb42-6157715e310a","Type":"ContainerStarted","Data":"34e2c1381191e62620fc81bad4c91a062b073ad093941ea3963c1a5b0a2381ae"}
Jul 07 22:55:00.677104 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:00.676616    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9" event={"ID":"41917d67-01f5-4122-bb42-6157715e310a","Type":"ContainerStarted","Data":"7f5b0659311f1953f80df0d27cac15e31505133a96cd93cdcaf11da1a30b5d97"}
Jul 07 22:55:00.677449 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:00.677426    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"
Jul 07 22:55:00.695888 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:00.695811    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9" podStartSLOduration=1.695792274 podStartE2EDuration="1.695792274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:54:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:55:00.692717456 +0000 UTC m=+2398.364936437" watchObservedRunningTime="2026-07-07 22:55:00.695792274 +0000 UTC m=+2398.368011264"
Jul 07 22:55:01.880043 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:01.880004    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kmnkq_301d8bec-bcb2-4a26-8c75-059321459a6b/dns/0.log"
Jul 07 22:55:01.900269 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:01.900236    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kmnkq_301d8bec-bcb2-4a26-8c75-059321459a6b/kube-rbac-proxy/0.log"
Jul 07 22:55:01.922811 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:01.922780    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t8kwx_001f8df1-bca6-441f-89d3-01bb8d3741b0/dns-node-resolver/0.log"
Jul 07 22:55:02.513023 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:02.512989    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6b6484bd4d-q59pd_9ff6ff5b-138c-4f29-ae44-ad7b6e692318/registry/0.log"
Jul 07 22:55:02.583201 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:02.583173    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vwjdg_621e2b82-009d-465d-940a-57c8bff5595d/node-ca/0.log"
Jul 07 22:55:02.983860 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:02.983824    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:55:02.991913 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:02.991855    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:55:03.714146 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:03.714114    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-776659d659-w7qkt_f615c2a4-314a-413d-a879-6c8c38d9b332/kube-auth-proxy/0.log"
Jul 07 22:55:04.581697 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:04.581659    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pplwz_ea880af9-eea2-4d99-a37f-63e976e118b0/serve-healthcheck-canary/0.log"
Jul 07 22:55:05.163252 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:05.163222    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gm7mg_d8271ced-dc02-4b00-858a-3c99ef4a9a55/kube-rbac-proxy/0.log"
Jul 07 22:55:05.185320 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:05.185291    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gm7mg_d8271ced-dc02-4b00-858a-3c99ef4a9a55/exporter/0.log"
Jul 07 22:55:05.208576 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:05.208544    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gm7mg_d8271ced-dc02-4b00-858a-3c99ef4a9a55/extractor/0.log"
Jul 07 22:55:07.697255 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:07.697223    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xs5cp/perf-node-gather-daemonset-74qc9"
Jul 07 22:55:07.836373 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:07.836345    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-82b4p_ff7041e3-a577-42d2-ba13-27baf97a5b02/server/0.log"
Jul 07 22:55:07.941559 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:07.941338    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg_f58441f9-4810-4b47-b7c7-5b700dc11727/manager/0.log"
Jul 07 22:55:07.962892 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:07.962858    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b6cfc4c4c-5q7bg_f58441f9-4810-4b47-b7c7-5b700dc11727/copy-manifests/0.log"
Jul 07 22:55:09.469041 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:09.469005    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6f8cffc658-xzwjw_e1fcc44f-3c50-4ca1-aa24-c5cc957015dc/manager/0.log"
Jul 07 22:55:09.857175 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:09.857124    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-776659d659-w7qkt_f615c2a4-314a-413d-a879-6c8c38d9b332/kube-auth-proxy/0.log"
Jul 07 22:55:11.342401 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:11.342374    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-776659d659-w7qkt_f615c2a4-314a-413d-a879-6c8c38d9b332/kube-auth-proxy/0.log"
Jul 07 22:55:15.910561 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:15.910513    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7x2ml_ad33be48-3fec-4f6d-9d2c-a01dd3e342df/kube-multus/0.log"
Jul 07 22:55:15.933650 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:15.933630    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kz7pz_e32ca0b6-d3f9-4868-9b27-06e20b06a8ae/kube-multus-additional-cni-plugins/0.log"
Jul 07 22:55:15.954398 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:15.954372    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kz7pz_e32ca0b6-d3f9-4868-9b27-06e20b06a8ae/egress-router-binary-copy/0.log"
Jul 07 22:55:15.975276 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:15.975257    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kz7pz_e32ca0b6-d3f9-4868-9b27-06e20b06a8ae/cni-plugins/0.log"
Jul 07 22:55:15.997528 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:15.997502    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kz7pz_e32ca0b6-d3f9-4868-9b27-06e20b06a8ae/bond-cni-plugin/0.log"
Jul 07 22:55:16.018759 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:16.018721    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kz7pz_e32ca0b6-d3f9-4868-9b27-06e20b06a8ae/routeoverride-cni/0.log"
Jul 07 22:55:16.039909 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:16.039885    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kz7pz_e32ca0b6-d3f9-4868-9b27-06e20b06a8ae/whereabouts-cni-bincopy/0.log"
Jul 07 22:55:16.059562 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:16.059544    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kz7pz_e32ca0b6-d3f9-4868-9b27-06e20b06a8ae/whereabouts-cni/0.log"
Jul 07 22:55:16.457274 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:16.457248    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-58clx_28fa7951-7615-4e16-96f8-c918ab062cd3/network-metrics-daemon/0.log"
Jul 07 22:55:16.477776 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:16.477742    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-58clx_28fa7951-7615-4e16-96f8-c918ab062cd3/kube-rbac-proxy/0.log"
Jul 07 22:55:17.662049 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:17.661968    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-controller/0.log"
Jul 07 22:55:17.680126 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:17.680093    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/1.log"
Jul 07 22:55:17.680306 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:17.680176    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/ovn-acl-logging/0.log"
Jul 07 22:55:17.706179 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:17.706156    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/kube-rbac-proxy-node/0.log"
Jul 07 22:55:17.730055 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:17.730017    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/kube-rbac-proxy-ovn-metrics/0.log"
Jul 07 22:55:17.752029 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:17.752002    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/northd/0.log"
Jul 07 22:55:17.774775 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:17.774751    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/nbdb/0.log"
Jul 07 22:55:17.795662 ip-10-0-128-159 kubenswrapper[2555]: I0707 22:55:17.795637    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mbznv_0f492bfc-2ea5-4bab-a34b-b6df6736724e/sbdb/0.log"