Nov 01 05:00:17.960133 ip-10-0-134-253 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Nov 01 05:00:17.960144 ip-10-0-134-253 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Nov 01 05:00:17.960151 ip-10-0-134-253 systemd[1]: kubelet.service: Failed with result 'resources'.
Nov 01 05:00:17.960373 ip-10-0-134-253 systemd[1]: Failed to start Kubernetes Kubelet.
Nov 01 05:00:28.209970 ip-10-0-134-253 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Nov 01 05:00:28.209992 ip-10-0-134-253 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 572b1f067e7147d88d18ff5273b0e0d1 --
Nov 01 05:02:19.874099 ip-10-0-134-253 systemd[1]: Starting Kubernetes Kubelet...
Nov 01 05:02:20.301255 ip-10-0-134-253 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 01 05:02:20.301255 ip-10-0-134-253 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 01 05:02:20.301255 ip-10-0-134-253 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 01 05:02:20.301255 ip-10-0-134-253 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 01 05:02:20.304823 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304688    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 05:02:20.304823 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304820    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 05:02:20.304823 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304824    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 05:02:20.304823 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304829    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 05:02:20.305077 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304833    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 05:02:20.305077 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304837    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 05:02:20.305077 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304842    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 05:02:20.305077 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304846    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 05:02:20.305077 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304850    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 05:02:20.305077 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304855    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 05:02:20.305077 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304858    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 05:02:20.305077 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304862    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 05:02:20.305077 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304865    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 05:02:20.305077 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304869    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 05:02:20.305077 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304873    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 05:02:20.305077 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304877    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 05:02:20.305077 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304881    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 05:02:20.305077 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304885    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 05:02:20.305077 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304896    2577 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 05:02:20.305077 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304901    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 05:02:20.305077 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304905    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 05:02:20.305077 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304909    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 05:02:20.305077 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304912    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 05:02:20.305077 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304916    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 05:02:20.305909 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304920    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 05:02:20.305909 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304924    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 05:02:20.305909 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304928    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 05:02:20.305909 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304932    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 05:02:20.305909 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304935    2577 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 05:02:20.305909 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304939    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 05:02:20.305909 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304943    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 05:02:20.305909 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304947    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 05:02:20.305909 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304951    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 05:02:20.305909 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304955    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 05:02:20.305909 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304959    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 05:02:20.305909 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304963    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 05:02:20.305909 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304967    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 05:02:20.305909 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304971    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 05:02:20.305909 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304977    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 05:02:20.305909 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304981    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 05:02:20.305909 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304985    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 05:02:20.305909 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304989    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 05:02:20.305909 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304993    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 05:02:20.305909 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.304997    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 05:02:20.306378 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305002    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 05:02:20.306378 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305014    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 05:02:20.306378 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305019    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 05:02:20.306378 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305023    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 05:02:20.306378 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305027    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 05:02:20.306378 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305031    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 05:02:20.306378 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305035    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 05:02:20.306378 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305039    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 05:02:20.306378 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305043    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 05:02:20.306378 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305048    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 05:02:20.306378 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305052    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 05:02:20.306378 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305056    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 05:02:20.306378 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305060    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 05:02:20.306378 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305064    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 05:02:20.306378 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305068    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 05:02:20.306378 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305072    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 05:02:20.306378 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305076    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 05:02:20.306378 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305080    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 05:02:20.306378 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305086    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 05:02:20.306864 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305092    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 05:02:20.306864 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305097    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 05:02:20.306864 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305101    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 05:02:20.306864 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305105    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 05:02:20.306864 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305109    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 05:02:20.306864 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305113    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 05:02:20.306864 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305120    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 05:02:20.306864 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305124    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 05:02:20.306864 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305129    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 05:02:20.306864 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305134    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 05:02:20.306864 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305137    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 05:02:20.306864 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305141    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 05:02:20.306864 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305146    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 05:02:20.306864 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305150    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 05:02:20.306864 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305154    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 05:02:20.306864 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305169    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 05:02:20.306864 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305174    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 05:02:20.306864 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305178    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 05:02:20.306864 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305182    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 05:02:20.307692 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305186    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 05:02:20.307692 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305190    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 05:02:20.307692 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305194    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 05:02:20.307692 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305199    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 05:02:20.307692 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305204    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 05:02:20.307692 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305208    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 05:02:20.307692 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305212    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 05:02:20.307692 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305216    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 05:02:20.307692 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305220    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 05:02:20.307692 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305224    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 05:02:20.307692 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305229    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 05:02:20.307692 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305233    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 05:02:20.307692 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305237    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 05:02:20.307692 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305241    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 05:02:20.307692 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305245    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 05:02:20.307692 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305249    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 05:02:20.307692 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305256    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 05:02:20.307692 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305264    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 05:02:20.307692 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305269    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 05:02:20.308517 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305274    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 05:02:20.308517 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305279    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 05:02:20.308517 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305283    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 05:02:20.308517 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305287    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 05:02:20.308517 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.305291    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 05:02:20.308517 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307674    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 05:02:20.308517 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307687    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 05:02:20.308517 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307692    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 05:02:20.308517 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307698    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 05:02:20.308517 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307703    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 05:02:20.308517 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307708    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 05:02:20.308517 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307712    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 05:02:20.308517 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307718    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 05:02:20.308517 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307722    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 05:02:20.308517 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307726    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 05:02:20.308517 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307730    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 05:02:20.308517 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307735    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 05:02:20.308517 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307739    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 05:02:20.308517 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307743    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 05:02:20.308517 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307747    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 05:02:20.309354 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307751    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 05:02:20.309354 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307755    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 05:02:20.309354 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307759    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 05:02:20.309354 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307763    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 05:02:20.309354 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307767    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 05:02:20.309354 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307771    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 05:02:20.309354 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307775    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 05:02:20.309354 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307799    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 05:02:20.309354 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307805    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 05:02:20.309354 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307810    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 05:02:20.309354 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307814    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 05:02:20.309354 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307818    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 05:02:20.309354 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307822    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 05:02:20.309354 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307827    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 05:02:20.309354 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307831    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 05:02:20.309354 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307835    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 05:02:20.309354 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307842    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 05:02:20.309354 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307847    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 05:02:20.309354 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307852    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 05:02:20.310166 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307856    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 05:02:20.310166 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307860    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 05:02:20.310166 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307864    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 05:02:20.310166 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307868    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 05:02:20.310166 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307874    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 05:02:20.310166 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307878    2577 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 05:02:20.310166 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307882    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 05:02:20.310166 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307887    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 05:02:20.310166 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307891    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 05:02:20.310166 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307895    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 05:02:20.310166 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307899    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 05:02:20.310166 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307904    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 05:02:20.310166 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307908    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 05:02:20.310166 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307913    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 05:02:20.310166 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307916    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 05:02:20.310166 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307921    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 05:02:20.310166 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307925    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 05:02:20.310166 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307929    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 05:02:20.310166 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307934    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 05:02:20.310166 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307938    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 05:02:20.310695 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307943    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 05:02:20.310695 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307948    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 05:02:20.310695 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307952    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 05:02:20.310695 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307956    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 05:02:20.310695 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307960    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 05:02:20.310695 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307965    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 05:02:20.310695 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307970    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 05:02:20.310695 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307975    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 05:02:20.310695 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307979    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 05:02:20.310695 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307983    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 05:02:20.310695 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307987    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 05:02:20.310695 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307991    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 05:02:20.310695 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307995    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 05:02:20.310695 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.307999    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 05:02:20.310695 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308003    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 05:02:20.310695 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308007    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 05:02:20.310695 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308011    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 05:02:20.310695 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308016    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 05:02:20.310695 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308023    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 05:02:20.311165 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308030    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 05:02:20.311165 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308035    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 05:02:20.311165 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308040    2577 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 05:02:20.311165 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308044    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 05:02:20.311165 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308048    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 05:02:20.311165 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308052    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 05:02:20.311165 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308056    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 05:02:20.311165 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308060    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 05:02:20.311165 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308064    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 05:02:20.311165 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308068    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 05:02:20.311165 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308072    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 05:02:20.311165 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308077    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 05:02:20.311165 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308082    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 05:02:20.311165 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308086    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 05:02:20.311165 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308090    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 05:02:20.311165 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308094    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 05:02:20.311165 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308099    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 05:02:20.311165 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308103    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 05:02:20.311165 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308108    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 05:02:20.311613 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308112    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 05:02:20.311613 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308116    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 05:02:20.311613 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308120    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 05:02:20.311613 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308124    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 05:02:20.311613 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308129    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 05:02:20.311613 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308133    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 05:02:20.311613 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308137    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 05:02:20.311613 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308144    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 05:02:20.311613 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308148    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 05:02:20.311613 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308153    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 05:02:20.311613 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308157    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 05:02:20.311613 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308167    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 05:02:20.311613 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308172    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 05:02:20.311613 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.308177    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 05:02:20.311613 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309403    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 05:02:20.311613 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309415    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 05:02:20.311613 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309421    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 05:02:20.311613 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309426    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 05:02:20.311613 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309431    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 05:02:20.311613 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309442    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 05:02:20.312113 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309447    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 05:02:20.312113 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309452    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 05:02:20.312113 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309457    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 05:02:20.312113 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309461    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 05:02:20.312113 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309467    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 05:02:20.312113 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309480    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 05:02:20.312113 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309484    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 05:02:20.312113 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309489    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 05:02:20.312113 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309493    2577 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 05:02:20.312113 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309498    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 05:02:20.312113 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309502    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 05:02:20.312113 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309511    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 05:02:20.312113 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309517    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 05:02:20.312113 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309521    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 05:02:20.312113 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309526    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 05:02:20.312113 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309531    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 05:02:20.312113 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309535    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 05:02:20.312113 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309542    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 05:02:20.312113 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309549    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 05:02:20.312553 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309553    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 05:02:20.312553 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309558    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 05:02:20.312553 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309562    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 05:02:20.312553 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309567    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 05:02:20.312553 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309571    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 05:02:20.312553 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309582    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 05:02:20.312553 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309586    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 05:02:20.312553 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309591    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 05:02:20.312553 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309595    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 05:02:20.312553 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309599    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 05:02:20.312553 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309604    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 05:02:20.312553 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309609    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 05:02:20.312553 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309614    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 05:02:20.312553 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309618    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 05:02:20.312553 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309623    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 05:02:20.312553 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309627    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 05:02:20.312553 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309632    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 05:02:20.312553 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309641    2577 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 05:02:20.312553 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309646    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 05:02:20.312553 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309650    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 05:02:20.313045 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309655    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 05:02:20.313045 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309659    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 05:02:20.313045 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309664    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 05:02:20.313045 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309668    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 05:02:20.313045 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309672    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 05:02:20.313045 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309679    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 05:02:20.313045 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309684    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 05:02:20.313045 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309689    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 05:02:20.313045 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309693    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 05:02:20.313045 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309702    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 05:02:20.313045 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309707    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 05:02:20.313045 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309711    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 05:02:20.313045 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309715    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 05:02:20.313045 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309720    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 05:02:20.313045 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309724    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 05:02:20.313045 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309730    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 05:02:20.313045 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309734    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 05:02:20.313045 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309739    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 05:02:20.313045 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309744    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 05:02:20.313045 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309749    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 05:02:20.313549 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309753    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 05:02:20.313549 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309758    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 05:02:20.313549 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309767    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 05:02:20.313549 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309772    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 05:02:20.313549 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309776    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 05:02:20.313549 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309797    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 05:02:20.313549 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309802    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 05:02:20.313549 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309806    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 05:02:20.313549 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309811    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 05:02:20.313549 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309815    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 05:02:20.313549 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309819    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 05:02:20.313549 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309823    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 05:02:20.313549 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309828    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 05:02:20.313549 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309831    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 05:02:20.313549 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309836    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 05:02:20.313549 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309845    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 05:02:20.313549 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309850    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 05:02:20.313549 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309854    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 05:02:20.313549 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309858    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 05:02:20.314021 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309863    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 05:02:20.314021 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309867    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 05:02:20.314021 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309871    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 05:02:20.314021 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309875    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 05:02:20.314021 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309880    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 05:02:20.314021 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309891    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 05:02:20.314021 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309926    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 05:02:20.314021 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309938    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 05:02:20.314021 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309942    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 05:02:20.314021 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309964    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 05:02:20.314021 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.309967    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 05:02:20.314021 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.310105    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 05:02:20.314021 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.310108    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 05:02:20.314021 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.310111    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 05:02:20.314021 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.310114    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 05:02:20.314021 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.310117    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 05:02:20.314021 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.310119    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 05:02:20.314021 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.310128    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 05:02:20.314021 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.310134    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.310137    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.310140    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.310143    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311155    2577 flags.go:64] FLAG: --address="0.0.0.0"
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311165    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311173    2577 flags.go:64] FLAG: --anonymous-auth="true"
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311181    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311185    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311189    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311193    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311197    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311201    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311204    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311207    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311210    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311213    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311216    2577 flags.go:64] FLAG: --cgroup-root=""
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311219    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311222    2577 flags.go:64] FLAG: --client-ca-file=""
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311226    2577 flags.go:64] FLAG: --cloud-provider="external"
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311229    2577 flags.go:64] FLAG: --cluster-dns="[]"
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311233    2577 flags.go:64] FLAG: --cluster-domain=""
Nov 01 05:02:20.314464 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311236    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311239    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311242    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311245    2577 flags.go:64] FLAG: --container-log-max-files="5"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311250    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311253    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311256    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311259    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311262    2577 flags.go:64] FLAG: --contention-profiling="false"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311265    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311268    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311271    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311274    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311279    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311282    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311285    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311288    2577 flags.go:64] FLAG: --enable-load-reader="false"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311291    2577 flags.go:64] FLAG: --enable-server="true"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311293    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311299    2577 flags.go:64] FLAG: --event-burst="100"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311302    2577 flags.go:64] FLAG: --event-qps="50"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311305    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311308    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311311    2577 flags.go:64] FLAG: --eviction-hard=""
Nov 01 05:02:20.315167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311315    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311318    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311322    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311325    2577 flags.go:64] FLAG: --eviction-soft=""
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311328    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311332    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311335    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311338    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311341    2577 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311344    2577 flags.go:64] FLAG: --fail-swap-on="true"
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311347    2577 flags.go:64] FLAG: --feature-gates=""
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311351    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311355    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311358    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311361    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311364    2577 flags.go:64] FLAG: --healthz-port="10248"
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311367    2577 flags.go:64] FLAG: --help="false"
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311370    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311373    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311376    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311379    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311382    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311385    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 01 05:02:20.315757 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311388    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311391    2577 flags.go:64] FLAG: --image-service-endpoint=""
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311394    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311397    2577 flags.go:64] FLAG: --kube-api-burst="100"
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311399    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311402    2577 flags.go:64] FLAG: --kube-api-qps="50"
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311405    2577 flags.go:64] FLAG: --kube-reserved=""
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311408    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311411    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311414    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311417    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311420    2577 flags.go:64] FLAG: --lock-file=""
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311423    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311426    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311429    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311435    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311437    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311440    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311443    2577 flags.go:64] FLAG: --logging-format="text"
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311446    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311449    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311452    2577 flags.go:64] FLAG: --manifest-url=""
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311455    2577 flags.go:64] FLAG: --manifest-url-header=""
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311459    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311462    2577 flags.go:64] FLAG: --max-open-files="1000000"
Nov 01 05:02:20.316336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311466    2577 flags.go:64] FLAG: --max-pods="110"
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311469    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311472    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311475    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311478    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311481    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311484    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311487    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311496    2577 flags.go:64] FLAG: --node-status-max-images="50"
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311499    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311502    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311505    2577 flags.go:64] FLAG: --pod-cidr=""
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311508    2577 flags.go:64] FLAG: --pod-manifest-path=""
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311510    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311516    2577 flags.go:64] FLAG: --pods-per-core="0"
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311520    2577 flags.go:64] FLAG: --port="10250"
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311523    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311526    2577 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-061474926bbfd0d9a"
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311530    2577 flags.go:64] FLAG: --qos-reserved=""
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311533    2577 flags.go:64] FLAG: --read-only-port="10255"
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311537    2577 flags.go:64] FLAG: --register-node="true"
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311540    2577 flags.go:64] FLAG: --register-with-taints=""
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311544    2577 flags.go:64] FLAG: --registry-burst="10"
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311547    2577 flags.go:64] FLAG: --registry-qps="5"
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311550    2577 flags.go:64] FLAG: --reserved-cpus=""
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311553    2577 flags.go:64] FLAG: --reserved-memory=""
Nov 01 05:02:20.316955 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311557    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311560    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311563    2577 flags.go:64] FLAG: --rotate-certificates="false"
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311566    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311569    2577 flags.go:64] FLAG: --runonce="false"
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311572    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311575    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311578    2577 flags.go:64] FLAG: --seccomp-default="false"
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311581    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311584    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311587    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311590    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311593    2577 flags.go:64] FLAG: --storage-driver-password="root"
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311595    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311598    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311601    2577 flags.go:64] FLAG: --storage-driver-user="root"
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311604    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311607    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311609    2577 flags.go:64] FLAG: --system-cgroups=""
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311612    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311619    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311622    2577 flags.go:64] FLAG: --tls-cert-file=""
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311625    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311629    2577 flags.go:64] FLAG: --tls-min-version=""
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311632    2577 flags.go:64] FLAG: --tls-private-key-file=""
Nov 01 05:02:20.317599 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311635    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 01 05:02:20.318248 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311638    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 01 05:02:20.318248 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311641    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 01 05:02:20.318248 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311644    2577 flags.go:64] FLAG: --v="2"
Nov 01 05:02:20.318248 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311649    2577 flags.go:64] FLAG: --version="false"
Nov 01 05:02:20.318248 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311653    2577 flags.go:64] FLAG: --vmodule=""
Nov 01 05:02:20.318248 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311657    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 01 05:02:20.318248 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.311660    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 01 05:02:20.318248 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311805    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 05:02:20.318248 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311808    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 05:02:20.318248 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311811    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 05:02:20.318248 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311814    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 05:02:20.318248 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311817    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 05:02:20.318248 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311819    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 05:02:20.318248 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311822    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 05:02:20.318248 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311825    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 05:02:20.318248 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311827    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 05:02:20.318248 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311830    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 05:02:20.318248 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311832    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 05:02:20.318248 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311835    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 05:02:20.318248 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311837    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 05:02:20.318248 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311840    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 05:02:20.318762 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311842    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 05:02:20.318762 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311844    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 05:02:20.318762 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311847    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 05:02:20.318762 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311850    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 05:02:20.318762 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311852    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 05:02:20.318762 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311856    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 05:02:20.318762 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311859    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 05:02:20.318762 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311861    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 05:02:20.318762 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311866    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 05:02:20.318762 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311871    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 05:02:20.318762 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311874    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 05:02:20.318762 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311878    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 05:02:20.318762 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311881    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 05:02:20.318762 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311884    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 05:02:20.318762 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311886    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 05:02:20.318762 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311889    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 05:02:20.318762 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311891    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 05:02:20.318762 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311894    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 05:02:20.318762 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311896    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 05:02:20.319263 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311899    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 05:02:20.319263 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311902    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 05:02:20.319263 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311904    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 05:02:20.319263 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311907    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 05:02:20.319263 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311910    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 05:02:20.319263 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311912    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 05:02:20.319263 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311915    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 05:02:20.319263 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311917    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 05:02:20.319263 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311920    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 05:02:20.319263 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311923    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 05:02:20.319263 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311925    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 05:02:20.319263 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311927    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 05:02:20.319263 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311930    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 05:02:20.319263 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311932    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 05:02:20.319263 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311935    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 05:02:20.319263 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311937    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 05:02:20.319263 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311940    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 05:02:20.319263 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311942    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 05:02:20.319263 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311947    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 05:02:20.319263 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311949    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 05:02:20.319729 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311952    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 05:02:20.319729 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311954    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 05:02:20.319729 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311957    2577 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 05:02:20.319729 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311959    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 05:02:20.319729 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311962    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 05:02:20.319729 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311965    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 05:02:20.319729 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311968    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 05:02:20.319729 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311971    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 05:02:20.319729 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311973    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 05:02:20.319729 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311976    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 05:02:20.319729 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311978    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 05:02:20.319729 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311982    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 05:02:20.319729 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311985    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 05:02:20.319729 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311988    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 05:02:20.319729 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311991    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 05:02:20.319729 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311994    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 05:02:20.319729 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311997    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 05:02:20.319729 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.311999    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 05:02:20.319729 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312002    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 05:02:20.319729 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312004    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 05:02:20.320208 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312007    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 05:02:20.320208 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312009    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 05:02:20.320208 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312012    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 05:02:20.320208 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312014    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 05:02:20.320208 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312017    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 05:02:20.320208 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312019    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 05:02:20.320208 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312021    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 05:02:20.320208 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312024    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 05:02:20.320208 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312026    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 05:02:20.320208 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312029    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 05:02:20.320208 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312033    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 05:02:20.320208 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312036    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 05:02:20.320208 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312038    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 05:02:20.320208 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312041    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 05:02:20.320208 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312043    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 05:02:20.320208 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312047    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 05:02:20.320208 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312050    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 05:02:20.320208 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312052    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 05:02:20.320208 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312056    2577 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 05:02:20.320208 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312058    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 05:02:20.320687 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312061    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 05:02:20.320687 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312063    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 05:02:20.320687 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312066    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 05:02:20.320687 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312068    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 05:02:20.320687 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312071    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 05:02:20.320687 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312073    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 05:02:20.320687 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312076    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 05:02:20.320687 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312078    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 05:02:20.320687 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312081    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 05:02:20.320687 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312083    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 05:02:20.320687 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312086    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 05:02:20.320687 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312088    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 05:02:20.320687 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312091    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 05:02:20.320687 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.312117    2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 01 05:02:20.320687 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312154    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 05:02:20.320687 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312159    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 05:02:20.321088 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312162    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 05:02:20.321088 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312165    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 05:02:20.321088 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312167    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 05:02:20.321088 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312170    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 05:02:20.321088 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312172    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 05:02:20.321088 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312175    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 05:02:20.321088 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312177    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 05:02:20.321088 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312182    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 05:02:20.321088 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312184    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 05:02:20.321088 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312187    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 05:02:20.321088 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312189    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 05:02:20.321088 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312194    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 05:02:20.321088 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312196    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 05:02:20.321088 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312199    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 05:02:20.321088 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312201    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 05:02:20.321088 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312204    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 05:02:20.321088 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312207    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 05:02:20.321088 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312210    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 05:02:20.321088 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312212    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 05:02:20.321088 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312214    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 05:02:20.321570 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312217    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 05:02:20.321570 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312219    2577 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 05:02:20.321570 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312222    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 05:02:20.321570 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312224    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 05:02:20.321570 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312227    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 05:02:20.321570 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312229    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 05:02:20.321570 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312232    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 05:02:20.321570 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312234    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 05:02:20.321570 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312237    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 05:02:20.321570 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312239    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 05:02:20.321570 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312243    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 05:02:20.321570 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312246    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 05:02:20.321570 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312248    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 05:02:20.321570 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312251    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 05:02:20.321570 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312254    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 05:02:20.321570 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312256    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 05:02:20.321570 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312258    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 05:02:20.321570 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312261    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 05:02:20.321570 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312263    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 05:02:20.321570 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312268    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 05:02:20.322065 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312270    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 05:02:20.322065 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312273    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 05:02:20.322065 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312276    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 05:02:20.322065 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312280    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 05:02:20.322065 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312282    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 05:02:20.322065 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312285    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 05:02:20.322065 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312287    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 05:02:20.322065 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312290    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 05:02:20.322065 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312293    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 05:02:20.322065 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312295    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 05:02:20.322065 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312298    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 05:02:20.322065 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312300    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 05:02:20.322065 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312318    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 05:02:20.322065 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312322    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 05:02:20.322065 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312326    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 05:02:20.322065 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312328    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 05:02:20.322065 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312331    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 05:02:20.322065 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312333    2577 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 05:02:20.322065 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312336    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 05:02:20.322065 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312338    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 05:02:20.322580 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312341    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 05:02:20.322580 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312343    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 05:02:20.322580 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312346    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 05:02:20.322580 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312349    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 05:02:20.322580 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312351    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 05:02:20.322580 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312354    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 05:02:20.322580 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312356    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 05:02:20.322580 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312358    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 05:02:20.322580 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312361    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 05:02:20.322580 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312363    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 05:02:20.322580 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312366    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 05:02:20.322580 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312370    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 05:02:20.322580 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312373    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 05:02:20.322580 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312375    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 05:02:20.322580 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312378    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 05:02:20.322580 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312381    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 05:02:20.322580 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312384    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 05:02:20.322580 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312387    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 05:02:20.322580 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312390    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 05:02:20.323049 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312392    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 05:02:20.323049 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312395    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 05:02:20.323049 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312404    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 05:02:20.323049 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312408    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 05:02:20.323049 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312411    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 05:02:20.323049 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312414    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 05:02:20.323049 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312417    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 05:02:20.323049 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312420    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 05:02:20.323049 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312422    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 05:02:20.323049 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312424    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 05:02:20.323049 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312427    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 05:02:20.323049 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312429    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 05:02:20.323049 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312432    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 05:02:20.323049 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312434    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 05:02:20.323049 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312438    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 05:02:20.323049 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312441    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 05:02:20.323049 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312444    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 05:02:20.323049 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312447    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 05:02:20.323049 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312450    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 05:02:20.323494 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312452    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 05:02:20.323494 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312455    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 05:02:20.323494 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312457    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 05:02:20.323494 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312460    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 05:02:20.323494 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312464    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 05:02:20.323494 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.312467    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 05:02:20.323494 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.319877    2577 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 01 05:02:20.323494 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.319895    2577 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 01 05:02:20.323494 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.319912    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 01 05:02:20.323494 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.319918    2577 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 01 05:02:20.323494 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.319974    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 05:02:20.323494 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.319980    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 05:02:20.323494 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.319983    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 05:02:20.323494 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.319986    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 05:02:20.323494 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.319988    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 05:02:20.323494 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.319991    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 05:02:20.323494 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.319994    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 05:02:20.323494 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.319997    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 05:02:20.323494 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.319999    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 05:02:20.323936 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320002    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 05:02:20.323936 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320004    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 05:02:20.323936 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320007    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 05:02:20.323936 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320010    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 05:02:20.323936 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320012    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 05:02:20.323936 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320015    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 05:02:20.323936 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320017    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 05:02:20.323936 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320020    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 05:02:20.323936 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320022    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 05:02:20.323936 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320025    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 05:02:20.323936 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320027    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 05:02:20.323936 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320030    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 05:02:20.323936 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320032    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 05:02:20.323936 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320035    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 05:02:20.323936 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320037    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 05:02:20.323936 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320039    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 05:02:20.323936 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320042    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 05:02:20.323936 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320044    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 05:02:20.323936 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320047    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 05:02:20.323936 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320049    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 05:02:20.324420 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320054    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 05:02:20.324420 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320057    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 05:02:20.324420 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320059    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 05:02:20.324420 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320062    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 05:02:20.324420 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320065    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 05:02:20.324420 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320067    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 05:02:20.324420 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320070    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 05:02:20.324420 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320072    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 05:02:20.324420 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320075    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 05:02:20.324420 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320077    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 05:02:20.324420 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320080    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 05:02:20.324420 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320082    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 05:02:20.324420 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320085    2577 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 05:02:20.324420 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320087    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 05:02:20.324420 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320090    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 05:02:20.324420 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320092    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 05:02:20.324420 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320094    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 05:02:20.324420 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320097    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 05:02:20.324420 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320099    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 05:02:20.324420 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320102    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 05:02:20.324907 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320105    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 05:02:20.324907 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320107    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 05:02:20.324907 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320109    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 05:02:20.324907 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320112    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 05:02:20.324907 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320114    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 05:02:20.324907 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320117    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 05:02:20.324907 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320119    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 05:02:20.324907 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320122    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 05:02:20.324907 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320124    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 05:02:20.324907 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320127    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 05:02:20.324907 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320129    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 05:02:20.324907 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320132    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 05:02:20.324907 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320136    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 05:02:20.324907 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320139    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 05:02:20.324907 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320142    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 05:02:20.324907 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320144    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 05:02:20.324907 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320146    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 05:02:20.324907 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320149    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 05:02:20.324907 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320151    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 05:02:20.325352 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320154    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 05:02:20.325352 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320156    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 05:02:20.325352 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320159    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 05:02:20.325352 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320161    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 05:02:20.325352 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320164    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 05:02:20.325352 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320166    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 05:02:20.325352 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320169    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 05:02:20.325352 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320171    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 05:02:20.325352 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320174    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 05:02:20.325352 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320176    2577 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 05:02:20.325352 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320178    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 05:02:20.325352 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320181    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 05:02:20.325352 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320183    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 05:02:20.325352 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320186    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 05:02:20.325352 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320188    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 05:02:20.325352 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320191    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 05:02:20.325352 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320194    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 05:02:20.325352 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320196    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 05:02:20.325352 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320199    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 05:02:20.325352 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320201    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 05:02:20.325842 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320203    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 05:02:20.325842 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320206    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 05:02:20.325842 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320208    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 05:02:20.325842 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320211    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 05:02:20.325842 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320214    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 05:02:20.325842 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320218    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 05:02:20.325842 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320223    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 05:02:20.325842 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320229    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 05:02:20.325842 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320233    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 05:02:20.325842 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320237    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 05:02:20.325842 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320240    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 05:02:20.325842 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320244    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 05:02:20.325842 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320247    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 05:02:20.325842 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320249    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 05:02:20.325842 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320252    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 05:02:20.325842 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320254    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 05:02:20.325842 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320257    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 05:02:20.325842 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320259    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 05:02:20.326261 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.320286    2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 01 05:02:20.326261 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320430    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 05:02:20.326261 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320435    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 05:02:20.326261 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320438    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 05:02:20.326261 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320441    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 05:02:20.326261 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320444    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 05:02:20.326261 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320446    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 05:02:20.326261 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320449    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 05:02:20.326261 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320452    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 05:02:20.326261 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320455    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 05:02:20.326261 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320457    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 05:02:20.326261 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320460    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 05:02:20.326261 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320462    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 05:02:20.326261 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320465    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 05:02:20.326261 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320467    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 05:02:20.326261 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320470    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 05:02:20.326641 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320472    2577 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 05:02:20.326641 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320475    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 05:02:20.326641 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320477    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 05:02:20.326641 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320480    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 05:02:20.326641 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320483    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 05:02:20.326641 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320486    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 05:02:20.326641 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320489    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 05:02:20.326641 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320492    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 05:02:20.326641 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320494    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 05:02:20.326641 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320497    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 05:02:20.326641 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320499    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 05:02:20.326641 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320502    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 05:02:20.326641 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320504    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 05:02:20.326641 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320507    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 05:02:20.326641 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320509    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 05:02:20.326641 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320512    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 05:02:20.326641 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320514    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 05:02:20.326641 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320516    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 05:02:20.326641 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320519    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 05:02:20.326641 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320522    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 05:02:20.327229 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320524    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 05:02:20.327229 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320527    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 05:02:20.327229 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320529    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 05:02:20.327229 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320531    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 05:02:20.327229 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320534    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 05:02:20.327229 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320537    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 05:02:20.327229 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320539    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 05:02:20.327229 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320541    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 05:02:20.327229 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320544    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 05:02:20.327229 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320546    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 05:02:20.327229 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320549    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 05:02:20.327229 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320551    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 05:02:20.327229 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320553    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 05:02:20.327229 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320556    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 05:02:20.327229 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320558    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 05:02:20.327229 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320561    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 05:02:20.327229 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320564    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 05:02:20.327229 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320566    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 05:02:20.327229 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320569    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 05:02:20.327229 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320575    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 05:02:20.327690 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320578    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 05:02:20.327690 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320580    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 05:02:20.327690 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320583    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 05:02:20.327690 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320586    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 05:02:20.327690 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320588    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 05:02:20.327690 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320590    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 05:02:20.327690 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320593    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 05:02:20.327690 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320595    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 05:02:20.327690 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320598    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 05:02:20.327690 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320600    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 05:02:20.327690 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320603    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 05:02:20.327690 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320606    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 05:02:20.327690 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320608    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 05:02:20.327690 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320610    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 05:02:20.327690 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320613    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 05:02:20.327690 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320616    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 05:02:20.327690 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320620    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 05:02:20.327690 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320622    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 05:02:20.327690 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320625    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 05:02:20.328149 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320627    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 05:02:20.328149 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320629    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 05:02:20.328149 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320632    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 05:02:20.328149 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320635    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 05:02:20.328149 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320638    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 05:02:20.328149 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320640    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 05:02:20.328149 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320642    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 05:02:20.328149 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320645    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 05:02:20.328149 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320648    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 05:02:20.328149 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320652    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 05:02:20.328149 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320656    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 05:02:20.328149 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320658    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 05:02:20.328149 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320661    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 05:02:20.328149 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320665    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 05:02:20.328149 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320668    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 05:02:20.328149 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320671    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 05:02:20.328149 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320673    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 05:02:20.328149 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320676    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 05:02:20.328149 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320678    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 05:02:20.328589 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320681    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 05:02:20.328589 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320683    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 05:02:20.328589 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320685    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 05:02:20.328589 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320688    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 05:02:20.328589 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320690    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 05:02:20.328589 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320693    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 05:02:20.328589 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320696    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 05:02:20.328589 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320698    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 05:02:20.328589 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320701    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 05:02:20.328589 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320703    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 05:02:20.328589 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320705    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 05:02:20.328589 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320708    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 05:02:20.328589 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320711    2577 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 05:02:20.328589 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.320733    2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 01 05:02:20.328589 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320769    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 05:02:20.328589 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320774    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 05:02:20.328962 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320777    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 05:02:20.328962 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320794    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 05:02:20.328962 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320797    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 05:02:20.328962 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320799    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 05:02:20.328962 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320802    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 05:02:20.328962 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320805    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 05:02:20.328962 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320807    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 05:02:20.328962 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320810    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 05:02:20.328962 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320812    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 05:02:20.328962 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320815    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 05:02:20.328962 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320817    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 05:02:20.328962 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320823    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 05:02:20.328962 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320826    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 05:02:20.328962 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320828    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 05:02:20.328962 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320830    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 05:02:20.328962 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320833    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 05:02:20.328962 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320835    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 05:02:20.328962 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320838    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 05:02:20.328962 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320840    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 05:02:20.328962 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320843    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 05:02:20.329441 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320845    2577 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 05:02:20.329441 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320847    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 05:02:20.329441 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320850    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 05:02:20.329441 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320852    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 05:02:20.329441 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320855    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 05:02:20.329441 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320860    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 05:02:20.329441 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320863    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 05:02:20.329441 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320866    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 05:02:20.329441 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320868    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 05:02:20.329441 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320870    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 05:02:20.329441 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320873    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 05:02:20.329441 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320875    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 05:02:20.329441 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320878    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 05:02:20.329441 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320880    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 05:02:20.329441 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320883    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 05:02:20.329441 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320887    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 05:02:20.329441 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320890    2577 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 05:02:20.329441 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320892    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 05:02:20.329441 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320895    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 05:02:20.329441 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320897    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 05:02:20.329948 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320900    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 05:02:20.329948 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320902    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 05:02:20.329948 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320904    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 05:02:20.329948 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320907    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 05:02:20.329948 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320910    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 05:02:20.329948 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320913    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 05:02:20.329948 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320915    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 05:02:20.329948 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320917    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 05:02:20.329948 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320920    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 05:02:20.329948 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320922    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 05:02:20.329948 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320924    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 05:02:20.329948 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320927    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 05:02:20.329948 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320929    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 05:02:20.329948 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320932    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 05:02:20.329948 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320934    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 05:02:20.329948 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320937    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 05:02:20.329948 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320939    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 05:02:20.329948 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320942    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 05:02:20.329948 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320944    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 05:02:20.330423 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320946    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 05:02:20.330423 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320949    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 05:02:20.330423 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320951    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 05:02:20.330423 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320954    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 05:02:20.330423 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320957    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 05:02:20.330423 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320959    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 05:02:20.330423 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320961    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 05:02:20.330423 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320964    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 05:02:20.330423 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320968    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 05:02:20.330423 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320970    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 05:02:20.330423 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320972    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 05:02:20.330423 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320975    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 05:02:20.330423 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320977    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 05:02:20.330423 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320981    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 05:02:20.330423 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320983    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 05:02:20.330423 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320986    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 05:02:20.330423 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320988    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 05:02:20.330423 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320991    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 05:02:20.330423 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320994    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 05:02:20.330423 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320996    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 05:02:20.330922 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.320999    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 05:02:20.330922 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321001    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 05:02:20.330922 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321003    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 05:02:20.330922 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321006    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 05:02:20.330922 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321008    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 05:02:20.330922 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321010    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 05:02:20.330922 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321013    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 05:02:20.330922 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321015    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 05:02:20.330922 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321021    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 05:02:20.330922 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321023    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 05:02:20.330922 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321026    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 05:02:20.330922 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321028    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 05:02:20.330922 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321032    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 05:02:20.330922 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321035    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 05:02:20.330922 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321039    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 05:02:20.330922 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321042    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 05:02:20.330922 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321045    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 05:02:20.330922 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321047    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 05:02:20.330922 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321050    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 05:02:20.330922 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321052    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 05:02:20.331389 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321057    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 05:02:20.331389 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321060    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 05:02:20.331389 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321062    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 05:02:20.331389 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321064    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 05:02:20.331389 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:20.321067    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 05:02:20.331389 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.321121    2577 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 01 05:02:20.331389 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.323402    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 01 05:02:20.331389 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.324335    2577 server.go:1014] "Starting client certificate rotation"
Nov 01 05:02:20.331389 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.324427    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 01 05:02:20.331389 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.324458    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 01 05:02:20.331389 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.326416    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 01 05:02:20.331389 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.329264    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 01 05:02:20.341433 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.341415    2577 log.go:25] "Validated CRI v1 runtime API"
Nov 01 05:02:20.345678 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.345587    2577 log.go:25] "Validated CRI v1 image API"
Nov 01 05:02:20.347478 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.347452    2577 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 01 05:02:20.355104 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.355080    2577 fs.go:135] Filesystem UUIDs: map[569cc989-5283-41df-9103-52866cfe9d73:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e1eb79c9-a3af-4e8d-8cea-fd91056eada3:/dev/nvme0n1p3]
Nov 01 05:02:20.355179 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.355103    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 01 05:02:20.359318 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.359206    2577 manager.go:217] Machine: {Timestamp:2026-11-01 05:02:20.358725516 +0000 UTC m=+0.360798659 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100822 MemoryCapacity:32812011520 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec273bbdaeb0eb3fbec17f1db8c05b76 SystemUUID:ec273bbd-aeb0-eb3f-bec1-7f1db8c05b76 BootID:572b1f06-7e71-47d8-8d18-ff5273b0e0d1 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406003712 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:11:14:e0:c2:03 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:11:14:e0:c2:03 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5e:ba:41:8e:09:32 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812011520 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 01 05:02:20.359897 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.359884    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 01 05:02:20.360004 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.359988    2577 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 01 05:02:20.362807 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.362765    2577 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 01 05:02:20.362940 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.362809    2577 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-253.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 01 05:02:20.362982 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.362943    2577 topology_manager.go:143] "Creating topology manager with none policy"
Nov 01 05:02:20.362982 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.362951    2577 container_manager_linux.go:311] "Creating device plugin manager"
Nov 01 05:02:20.363045 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.362981    2577 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 01 05:02:20.363045 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.362999    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 01 05:02:20.364130 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.364120    2577 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 01 05:02:20.368254 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.368242    2577 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 01 05:02:20.368370 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.368361    2577 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 01 05:02:20.369049 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.369040    2577 kubelet.go:493] "Attempting to sync node with API server"
Nov 01 05:02:20.369085 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.369054    2577 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 01 05:02:20.369085 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.369063    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 01 05:02:20.369085 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.369071    2577 kubelet.go:405] "Adding apiserver pod source"
Nov 01 05:02:20.369085 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.369079    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 01 05:02:20.369395 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.369382    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 01 05:02:20.371692 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.371672    2577 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 01 05:02:20.378333 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.378321    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 01 05:02:20.379135 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.379123    2577 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 01 05:02:20.379171 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.379153    2577 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 01 05:02:20.379301 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.379292    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 01 05:02:20.379334 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.379308    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 01 05:02:20.379334 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.379315    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 01 05:02:20.379334 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.379320    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 01 05:02:20.379334 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.379326    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 01 05:02:20.379334 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.379331    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 01 05:02:20.379951 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.379942    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 01 05:02:20.379982 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.379953    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 01 05:02:20.379982 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.379961    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 01 05:02:20.379982 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.379968    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 01 05:02:20.379982 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.379982    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 01 05:02:20.380103 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.379992    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 01 05:02:20.380652 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.380644    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 01 05:02:20.380687 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.380653    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 01 05:02:20.384475 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.384463    2577 server.go:1263] "Started kubelet"
Nov 01 05:02:20.384564 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.384542    2577 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 01 05:02:20.384675 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.384623    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 01 05:02:20.384712 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.384701    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 01 05:02:20.387140 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.387119    2577 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 01 05:02:20.387213 ip-10-0-134-253 systemd[1]: Started Kubernetes Kubelet.
Nov 01 05:02:20.390212 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.390197    2577 server.go:317] "Adding debug handlers to kubelet server"
Nov 01 05:02:20.391978 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:20.391960    2577 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 01 05:02:20.393833 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.393816    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 01 05:02:20.395176 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.395156    2577 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 01 05:02:20.395848 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.395826    2577 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 01 05:02:20.395848 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.395850    2577 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 01 05:02:20.396005 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.395983    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 01 05:02:20.396166 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.396152    2577 reconstruct.go:97] "Volume reconstruction finished"
Nov 01 05:02:20.396240 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.396231    2577 reconciler.go:29] "Reconciler: start to sync state"
Nov 01 05:02:20.396398 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:20.396377    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-253.us-west-1.compute.internal\" not found"
Nov 01 05:02:20.396561 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.396544    2577 factory.go:55] Registering systemd factory
Nov 01 05:02:20.396616 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.396566    2577 factory.go:223] Registration of the systemd container factory successfully
Nov 01 05:02:20.397765 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.397748    2577 factory.go:153] Registering CRI-O factory
Nov 01 05:02:20.397856 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.397768    2577 factory.go:223] Registration of the crio container factory successfully
Nov 01 05:02:20.397911 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.397899    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 01 05:02:20.397953 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.397923    2577 factory.go:103] Registering Raw factory
Nov 01 05:02:20.397953 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.397940    2577 manager.go:1199] Started watching for new ooms in manager
Nov 01 05:02:20.399017 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.399002    2577 manager.go:319] Starting recovery of all containers
Nov 01 05:02:20.408520 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.408504    2577 manager.go:324] Recovery completed
Nov 01 05:02:20.412521 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.412508    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 05:02:20.413966 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.413948    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 05:02:20.414048 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.413984    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 05:02:20.414048 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.414002    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 05:02:20.414524 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.414513    2577 cpu_manager.go:226] "Starting" policy="none"
Nov 01 05:02:20.414575 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.414523    2577 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 01 05:02:20.414575 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.414539    2577 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 01 05:02:20.414575 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.414562    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 05:02:20.415227 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.415212    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 05:02:20.415284 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.415239    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 05:02:20.415284 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.415249    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 05:02:20.416691 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.416679    2577 policy_none.go:50] "Start"
Nov 01 05:02:20.416750 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.416695    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 01 05:02:20.416750 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.416705    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 01 05:02:20.418615 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.418605    2577 policy_none.go:44] "Start"
Nov 01 05:02:20.482146 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.462758    2577 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 01 05:02:20.482146 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.473809    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 01 05:02:20.482146 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.475072    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 01 05:02:20.482146 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.475092    2577 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 01 05:02:20.482146 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.475109    2577 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 01 05:02:20.482146 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:20.475144    2577 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 01 05:02:20.482146 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.478673    2577 manager.go:341] "Starting Device Plugin manager"
Nov 01 05:02:20.482146 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:20.478720    2577 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 01 05:02:20.482146 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.478733    2577 server.go:87] "Starting device plugin registration server"
Nov 01 05:02:20.482146 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.478968    2577 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 01 05:02:20.482146 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.478980    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 01 05:02:20.482146 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.479069    2577 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 01 05:02:20.482146 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.479153    2577 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 01 05:02:20.482146 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.479161    2577 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 01 05:02:20.482146 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:20.479795    2577 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 01 05:02:20.482146 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:20.479849    2577 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-253.us-west-1.compute.internal\" not found"
Nov 01 05:02:20.528765 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.528735    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-253.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 01 05:02:20.528765 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:20.528735    2577 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-253.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 01 05:02:20.575395 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.575325    2577 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal"]
Nov 01 05:02:20.575395 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.575394    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 05:02:20.576341 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.576326    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 05:02:20.576399 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.576355    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 05:02:20.576399 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.576365    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 05:02:20.578288 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:20.577223    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-253.us-west-1.compute.internal.18e3d33e1ef59d1e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-253.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-253.us-west-1.compute.internal,},FirstTimestamp:2026-11-01 05:02:20.384435486 +0000 UTC m=+0.386508630,LastTimestamp:2026-11-01 05:02:20.384435486 +0000 UTC m=+0.386508630,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-253.us-west-1.compute.internal,}"
Nov 01 05:02:20.579992 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.579960    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 05:02:20.580848 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.580831    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 05:02:20.580945 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.580905    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:20.580945 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.580940    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 05:02:20.581368 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.581347    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 05:02:20.581431 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.581387    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 05:02:20.581431 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.581402    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 05:02:20.581431 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.581430    2577 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:20.581622 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.581609    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 05:02:20.581666 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.581631    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 05:02:20.581666 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.581633    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 05:02:20.581666 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.581641    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 05:02:20.581666 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.581657    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 05:02:20.581833 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.581670    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 05:02:20.584430 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.584413    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:20.584526 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.584439    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 05:02:20.585130 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.585114    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 05:02:20.585213 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.585141    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 05:02:20.585213 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.585155    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 05:02:20.597891 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.597864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f72249826b80cce6b307e39e964ca081-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal\" (UID: \"f72249826b80cce6b307e39e964ca081\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:20.597983 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.597895    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f72249826b80cce6b307e39e964ca081-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal\" (UID: \"f72249826b80cce6b307e39e964ca081\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:20.597983 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.597912    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c1427a4a27caa7c2e9df3986def0a2a-config\") pod \"kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal\" (UID: \"1c1427a4a27caa7c2e9df3986def0a2a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:20.602133 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.602112    2577 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 01 05:02:20.602133 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:20.602114    2577 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-253.us-west-1.compute.internal\" not found" node="ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:20.602246 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.602167    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jx4sv"
Nov 01 05:02:20.605192 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:20.605176    2577 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-253.us-west-1.compute.internal\" not found" node="ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:20.639577 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:20.639478    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-253.us-west-1.compute.internal.18e3d33e20b8348a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-253.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-253.us-west-1.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-253.us-west-1.compute.internal,},FirstTimestamp:2026-11-01 05:02:20.41396545 +0000 UTC m=+0.416038598,LastTimestamp:2026-11-01 05:02:20.41396545 +0000 UTC m=+0.416038598,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-253.us-west-1.compute.internal,}"
Nov 01 05:02:20.640280 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:20.640257    2577 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Nov 01 05:02:20.640280 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:20.640266    2577 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 01 05:02:20.640398 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:20.640260    2577 reflector.go:204] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass"
Nov 01 05:02:20.640398 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:20.640264    2577 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-253.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 01 05:02:20.692278 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:20.692243    2577 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-134-253.us-west-1.compute.internal\" not found" node="ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:20.698374 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.698351    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f72249826b80cce6b307e39e964ca081-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal\" (UID: \"f72249826b80cce6b307e39e964ca081\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:20.698466 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.698380    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f72249826b80cce6b307e39e964ca081-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal\" (UID: \"f72249826b80cce6b307e39e964ca081\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:20.698466 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.698397    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c1427a4a27caa7c2e9df3986def0a2a-config\") pod \"kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal\" (UID: \"1c1427a4a27caa7c2e9df3986def0a2a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:20.698566 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.698467    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f72249826b80cce6b307e39e964ca081-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal\" (UID: \"f72249826b80cce6b307e39e964ca081\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:20.698566 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.698520    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f72249826b80cce6b307e39e964ca081-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal\" (UID: \"f72249826b80cce6b307e39e964ca081\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:20.698566 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.698538    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c1427a4a27caa7c2e9df3986def0a2a-config\") pod \"kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal\" (UID: \"1c1427a4a27caa7c2e9df3986def0a2a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:20.835278 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:20.835190    2577 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-134-253.us-west-1.compute.internal\" not found" node="ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:20.893365 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.893328    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 05:02:20.894566 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.894546    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 05:02:20.894691 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.894583    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 05:02:20.894691 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.894600    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 05:02:20.894691 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.894633    2577 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:20.907041 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.907021    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:20.910196 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.910178    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:20.950963 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:20.950938    2577 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:20.951073 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:20.950967    2577 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-253.us-west-1.compute.internal\": node \"ip-10-0-134-253.us-west-1.compute.internal\" not found"
Nov 01 05:02:21.074049 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:21.074015    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-253.us-west-1.compute.internal\" not found"
Nov 01 05:02:21.174604 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:21.174519    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-253.us-west-1.compute.internal\" not found"
Nov 01 05:02:21.274779 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:21.274748    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-253.us-west-1.compute.internal\" not found"
Nov 01 05:02:21.324288 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:21.324263    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 01 05:02:21.324721 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:21.324437    2577 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://aa20f316a090d4ba282b8462800e02e0-246993faf7c49e49.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-134-253.us-west-1.compute.internal.18e3d33e20b8348a\": read tcp 10.0.134.253:32852->18.117.66.57:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-134-253.us-west-1.compute.internal.18e3d33e20b8348a  default   4809 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-253.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-253.us-west-1.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-253.us-west-1.compute.internal,},FirstTimestamp:2026-11-01 05:02:20 +0000 UTC,LastTimestamp:2026-11-01 05:02:20.581367841 +0000 UTC m=+0.583440995,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-253.us-west-1.compute.internal,}"
Nov 01 05:02:21.375755 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:21.375727    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-253.us-west-1.compute.internal\" not found"
Nov 01 05:02:21.476374 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:21.476313    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-253.us-west-1.compute.internal\" not found"
Nov 01 05:02:21.552537 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:21.552498    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 05:02:21.553347 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:21.553330    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 05:02:21.553417 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:21.553361    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 05:02:21.553417 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:21.553371    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 05:02:21.576936 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:21.576914    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-253.us-west-1.compute.internal\" not found"
Nov 01 05:02:21.607590 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:21.607550    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-31 04:57:21 +0000 UTC" deadline="2028-07-25 19:24:57.351010668 +0000 UTC"
Nov 01 05:02:21.607590 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:21.607588    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15182h22m35.743427517s"
Nov 01 05:02:21.677361 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:21.677335    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-253.us-west-1.compute.internal\" not found"
Nov 01 05:02:21.692854 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:21.692836    2577 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 05:02:21.777982 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:21.777906    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-253.us-west-1.compute.internal\" not found"
Nov 01 05:02:21.861073 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:21.860997    2577 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 05:02:21.878017 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:21.877978    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-253.us-west-1.compute.internal\" not found"
Nov 01 05:02:21.978577 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:21.978543    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-253.us-west-1.compute.internal\" not found"
Nov 01 05:02:22.050864 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:22.050829    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf72249826b80cce6b307e39e964ca081.slice/crio-3c1331112ec45d56fd63f15fd4809c720cc0ee0bd39f24754eef6fe229b37653 WatchSource:0}: Error finding container 3c1331112ec45d56fd63f15fd4809c720cc0ee0bd39f24754eef6fe229b37653: Status 404 returned error can't find the container with id 3c1331112ec45d56fd63f15fd4809c720cc0ee0bd39f24754eef6fe229b37653
Nov 01 05:02:22.051235 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:22.051219    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c1427a4a27caa7c2e9df3986def0a2a.slice/crio-a34c6abf742f8f286b936d09b642c7ad10b290cc9e7f39dfbfc46d59a9b0e9b6 WatchSource:0}: Error finding container a34c6abf742f8f286b936d09b642c7ad10b290cc9e7f39dfbfc46d59a9b0e9b6: Status 404 returned error can't find the container with id a34c6abf742f8f286b936d09b642c7ad10b290cc9e7f39dfbfc46d59a9b0e9b6
Nov 01 05:02:22.055889 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.055874    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 05:02:22.078853 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:22.078826    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-253.us-west-1.compute.internal\" not found"
Nov 01 05:02:22.135240 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.135209    2577 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 05:02:22.197046 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.197015    2577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:22.258901 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.258878    2577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 01 05:02:22.260050 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.260033    2577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:02:22.281260 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.281235    2577 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 05:02:22.326356 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.326287    2577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 01 05:02:22.370162 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.370132    2577 apiserver.go:52] "Watching apiserver"
Nov 01 05:02:22.396803 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.396757    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 01 05:02:22.479901 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.479841    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal" event={"ID":"f72249826b80cce6b307e39e964ca081","Type":"ContainerStarted","Data":"3c1331112ec45d56fd63f15fd4809c720cc0ee0bd39f24754eef6fe229b37653"}
Nov 01 05:02:22.481002 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.480951    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal" event={"ID":"1c1427a4a27caa7c2e9df3986def0a2a","Type":"ContainerStarted","Data":"a34c6abf742f8f286b936d09b642c7ad10b290cc9e7f39dfbfc46d59a9b0e9b6"}
Nov 01 05:02:22.482072 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.482051    2577 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 01 05:02:22.482347 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.482332    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-tl74d","openshift-network-operator/iptables-alerter-nk4wf","kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal","openshift-cluster-node-tuning-operator/tuned-m8d7h","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal","kube-system/global-pull-secret-syncer-q92dh","kube-system/konnectivity-agent-jswq7","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd","openshift-dns/node-resolver-85452","openshift-multus/multus-additional-cni-plugins-h6ht6","openshift-ovn-kubernetes/ovnkube-node-89wdz","openshift-image-registry/node-ca-m9ckd","openshift-multus/multus-gwbm4","openshift-multus/network-metrics-daemon-rvg8t"]
Nov 01 05:02:22.486472 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.486451    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.488772 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.488752    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.491079 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.491061    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:22.491182 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:22.491132    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvg8t" podUID="ef46e568-5c98-4520-99e5-c49c4c3aca2a"
Nov 01 05:02:22.493519 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.493425    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.495874 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.495853    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nk4wf"
Nov 01 05:02:22.498363 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.498346    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.500829 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.500811    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-85452"
Nov 01 05:02:22.503205 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.503188    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:22.503279 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:22.503240    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q92dh" podUID="36a4e628-5f12-4849-9d77-deaa3e93b190"
Nov 01 05:02:22.505778 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.505760    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:22.505879 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:22.505826    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl74d" podUID="a5b30117-7cb8-43ec-92a8-4e326b48c22d"
Nov 01 05:02:22.508294 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.508278    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jswq7"
Nov 01 05:02:22.510663 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.510640    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/037937cc-75c9-4416-bd8f-63589df05625-cnibin\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.510775 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.510670    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-run-k8s-cni-cncf-io\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.510775 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.510691    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-run-netns\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.510775 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.510717    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.511012 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.510785    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bdedda0-279a-41fd-a7cc-300c816985df-ovn-node-metrics-cert\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.511012 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.510819    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-tuned\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.511012 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.510838    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m9ckd"
Nov 01 05:02:22.511012 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.510848    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-system-cni-dir\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.511012 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.510920    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-cnibin\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.511012 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.510958    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-conf-dir\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.511296 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.511071    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-systemd-units\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.511296 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.511110    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bdedda0-279a-41fd-a7cc-300c816985df-ovnkube-script-lib\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.511296 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.511138    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-sysctl-d\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.511296 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.511184    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-var-lib-kubelet\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.511296 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.511213    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-tmp\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.511296 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.511268    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-run-systemd\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.511296 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.511298    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-log-socket\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.511610 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.511330    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bdedda0-279a-41fd-a7cc-300c816985df-env-overrides\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.511610 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.511360    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-modprobe-d\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.511610 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.511384    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-var-lib-cni-multus\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.511610 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.511410    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/037937cc-75c9-4416-bd8f-63589df05625-cni-binary-copy\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.511610 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.511432    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-kubernetes\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.511610 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.511468    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxknn\" (UniqueName: \"kubernetes.io/projected/bb889bee-f31e-47f1-8617-6a27bc1eebf4-kube-api-access-bxknn\") pod \"iptables-alerter-nk4wf\" (UID: \"bb889bee-f31e-47f1-8617-6a27bc1eebf4\") " pod="openshift-network-operator/iptables-alerter-nk4wf"
Nov 01 05:02:22.511610 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.511519    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-cni-dir\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.511610 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.511543    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb889bee-f31e-47f1-8617-6a27bc1eebf4-host-slash\") pod \"iptables-alerter-nk4wf\" (UID: \"bb889bee-f31e-47f1-8617-6a27bc1eebf4\") " pod="openshift-network-operator/iptables-alerter-nk4wf"
Nov 01 05:02:22.511610 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.511575    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee3cc716-4716-4207-a493-9eeee140a6fe-cni-binary-copy\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.511992 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.511622    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-hostroot\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.511992 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.511694    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-cni-netd\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.511992 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.511728    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bdedda0-279a-41fd-a7cc-300c816985df-ovnkube-config\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.512195 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.512157    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/037937cc-75c9-4416-bd8f-63589df05625-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.512281 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.512257    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-etc-openvswitch\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.512336 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.512313    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-run-ovn-kubernetes\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.512392 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.512370    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-var-lib-kubelet\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.512433 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.512413    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-run-openvswitch\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.512483 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.512467    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-run-ovn\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.512517 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.512503    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-sysctl-conf\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.512562 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.512543    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bb889bee-f31e-47f1-8617-6a27bc1eebf4-iptables-alerter-script\") pod \"iptables-alerter-nk4wf\" (UID: \"bb889bee-f31e-47f1-8617-6a27bc1eebf4\") " pod="openshift-network-operator/iptables-alerter-nk4wf"
Nov 01 05:02:22.512628 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.512610    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-sys\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.512666 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.512643    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nh4k\" (UniqueName: \"kubernetes.io/projected/0bdedda0-279a-41fd-a7cc-300c816985df-kube-api-access-2nh4k\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.512727 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.512710    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-run-netns\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.512759 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.512746    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs\") pod \"network-metrics-daemon-rvg8t\" (UID: \"ef46e568-5c98-4520-99e5-c49c4c3aca2a\") " pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:22.512798 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.512781    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-run-multus-certs\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.512980 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.512960    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-etc-kubernetes\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.514301 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.513030    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-var-lib-openvswitch\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.514301 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.513079    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-run\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.514301 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.513115    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-host\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.514301 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.513174    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-socket-dir-parent\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.514301 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.513226    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw4b5\" (UniqueName: \"kubernetes.io/projected/ee3cc716-4716-4207-a493-9eeee140a6fe-kube-api-access-tw4b5\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.514301 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.513276    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4qrj\" (UniqueName: \"kubernetes.io/projected/037937cc-75c9-4416-bd8f-63589df05625-kube-api-access-w4qrj\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.514301 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.513338    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-lib-modules\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.514301 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.513371    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/037937cc-75c9-4416-bd8f-63589df05625-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.514301 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.513427    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j8z2\" (UniqueName: \"kubernetes.io/projected/ef46e568-5c98-4520-99e5-c49c4c3aca2a-kube-api-access-2j8z2\") pod \"network-metrics-daemon-rvg8t\" (UID: \"ef46e568-5c98-4520-99e5-c49c4c3aca2a\") " pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:22.514301 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.513462    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-sysconfig\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.514301 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.513489    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-systemd\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.514301 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.513527    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-var-lib-cni-bin\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.514301 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.513561    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-slash\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.514301 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.513633    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-node-log\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.514301 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.513671    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-cni-bin\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.514301 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.513708    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/037937cc-75c9-4416-bd8f-63589df05625-system-cni-dir\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.515158 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.513737    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/037937cc-75c9-4416-bd8f-63589df05625-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.515158 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.513768    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-kubelet\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.515158 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.513803    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdwk6\" (UniqueName: \"kubernetes.io/projected/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-kube-api-access-zdwk6\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.515158 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.513839    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-daemon-config\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.516234 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.516215    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.543347 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.543322    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 01 05:02:22.543347 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.543327    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 01 05:02:22.543500 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.543370    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 01 05:02:22.543500 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.543380    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 01 05:02:22.543500 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.543417    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fbhl7\""
Nov 01 05:02:22.543500 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.543418    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 01 05:02:22.544830 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.544811    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 01 05:02:22.545028 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.545010    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4vjvv\""
Nov 01 05:02:22.545416 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.545392    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 01 05:02:22.545496 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.545427    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 01 05:02:22.545496 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.545431    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 01 05:02:22.545496 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.545430    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 01 05:02:22.545496 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.545475    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 01 05:02:22.549265 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.549247    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 01 05:02:22.549340 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.549265    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-q6tm4\""
Nov 01 05:02:22.549340 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.549322    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 01 05:02:22.551986 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.551971    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 01 05:02:22.552070 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.552033    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 01 05:02:22.552070 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.552033    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gnn2d\""
Nov 01 05:02:22.591661 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.591603    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 01 05:02:22.591766 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.591743    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 01 05:02:22.593457 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.593443    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 01 05:02:22.596520 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.596505    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 01 05:02:22.596715 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.596691    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 01 05:02:22.596715 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.596696    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 01 05:02:22.596935 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.596770    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-fhbr8\""
Nov 01 05:02:22.596935 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.596862    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 01 05:02:22.596935 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.596874    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 01 05:02:22.596935 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.596924    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6ndpw\""
Nov 01 05:02:22.596935 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.596934    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 01 05:02:22.597156 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.596952    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 01 05:02:22.597156 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.596968    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 01 05:02:22.597156 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.597000    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 01 05:02:22.597156 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.597026    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-nkwq4\""
Nov 01 05:02:22.597156 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.597067    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-b9csn\""
Nov 01 05:02:22.597156 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.597122    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-tw68j\""
Nov 01 05:02:22.597327 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.597196    2577 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 01 05:02:22.597327 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.597250    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tw8pz"
Nov 01 05:02:22.597403 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.597389    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 01 05:02:22.614703 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.614681    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-tmp\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.614767 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.614724    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-registration-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.614767 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.614744    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-run-systemd\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.614767 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.614760    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-log-socket\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.614939 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.614842    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-log-socket\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.614939 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.614840    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bdedda0-279a-41fd-a7cc-300c816985df-env-overrides\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.614939 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.614877    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-modprobe-d\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.614939 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.614899    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-run-systemd\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.614939 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.614904    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-var-lib-cni-multus\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.614939 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.614930    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-var-lib-cni-multus\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.615234 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.614952    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/36a4e628-5f12-4849-9d77-deaa3e93b190-kubelet-config\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:22.615234 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.614982    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/037937cc-75c9-4416-bd8f-63589df05625-cni-binary-copy\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.615234 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.614996    2577 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 01 05:02:22.615234 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615040    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-modprobe-d\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.615234 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615042    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-kubernetes\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.615234 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615087    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-kubernetes\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.615234 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615081    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxknn\" (UniqueName: \"kubernetes.io/projected/bb889bee-f31e-47f1-8617-6a27bc1eebf4-kube-api-access-bxknn\") pod \"iptables-alerter-nk4wf\" (UID: \"bb889bee-f31e-47f1-8617-6a27bc1eebf4\") " pod="openshift-network-operator/iptables-alerter-nk4wf"
Nov 01 05:02:22.615234 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615127    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-cni-dir\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.615234 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615153    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0197ee7e-0d4a-4e9c-8057-26356d6f34b3-hosts-file\") pod \"node-resolver-85452\" (UID: \"0197ee7e-0d4a-4e9c-8057-26356d6f34b3\") " pod="openshift-dns/node-resolver-85452"
Nov 01 05:02:22.615234 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615179    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzgwz\" (UniqueName: \"kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz\") pod \"network-check-target-tl74d\" (UID: \"a5b30117-7cb8-43ec-92a8-4e326b48c22d\") " pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:22.615234 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615207    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb889bee-f31e-47f1-8617-6a27bc1eebf4-host-slash\") pod \"iptables-alerter-nk4wf\" (UID: \"bb889bee-f31e-47f1-8617-6a27bc1eebf4\") " pod="openshift-network-operator/iptables-alerter-nk4wf"
Nov 01 05:02:22.615754 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615265    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee3cc716-4716-4207-a493-9eeee140a6fe-cni-binary-copy\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.615754 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615353    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb889bee-f31e-47f1-8617-6a27bc1eebf4-host-slash\") pod \"iptables-alerter-nk4wf\" (UID: \"bb889bee-f31e-47f1-8617-6a27bc1eebf4\") " pod="openshift-network-operator/iptables-alerter-nk4wf"
Nov 01 05:02:22.615754 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615364    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bdedda0-279a-41fd-a7cc-300c816985df-env-overrides\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.615754 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615380    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-cni-dir\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.615754 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615402    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-hostroot\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.615754 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615438    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-cni-netd\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.615754 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615445    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-hostroot\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.615754 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615466    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bdedda0-279a-41fd-a7cc-300c816985df-ovnkube-config\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.615754 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615494    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/037937cc-75c9-4416-bd8f-63589df05625-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.615754 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615486    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-cni-netd\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.615754 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-etc-openvswitch\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.615754 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615550    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-run-ovn-kubernetes\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.615754 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615574    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-var-lib-kubelet\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.615754 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615605    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-socket-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.615754 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615629    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-run-ovn-kubernetes\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.615754 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615632    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-run-openvswitch\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.615754 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615657    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/037937cc-75c9-4416-bd8f-63589df05625-cni-binary-copy\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.616592 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615665    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-run-ovn\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.616592 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615688    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-var-lib-kubelet\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.616592 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615725    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-sysctl-conf\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.616592 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615751    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bb889bee-f31e-47f1-8617-6a27bc1eebf4-iptables-alerter-script\") pod \"iptables-alerter-nk4wf\" (UID: \"bb889bee-f31e-47f1-8617-6a27bc1eebf4\") " pod="openshift-network-operator/iptables-alerter-nk4wf"
Nov 01 05:02:22.616592 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615775    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:22.616592 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615798    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-sys\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.616592 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615821    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0197ee7e-0d4a-4e9c-8057-26356d6f34b3-tmp-dir\") pod \"node-resolver-85452\" (UID: \"0197ee7e-0d4a-4e9c-8057-26356d6f34b3\") " pod="openshift-dns/node-resolver-85452"
Nov 01 05:02:22.616592 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615861    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/36a4e628-5f12-4849-9d77-deaa3e93b190-dbus\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:22.616592 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615882    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1f4c040-7436-4df1-8462-6a38acee2c5a-host\") pod \"node-ca-m9ckd\" (UID: \"d1f4c040-7436-4df1-8462-6a38acee2c5a\") " pod="openshift-image-registry/node-ca-m9ckd"
Nov 01 05:02:22.616592 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615913    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.616592 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615936    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nh4k\" (UniqueName: \"kubernetes.io/projected/0bdedda0-279a-41fd-a7cc-300c816985df-kube-api-access-2nh4k\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.616592 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615960    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-run-netns\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.616592 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615970    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee3cc716-4716-4207-a493-9eeee140a6fe-cni-binary-copy\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.616592 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.615985    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-device-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.616592 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616014    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs\") pod \"network-metrics-daemon-rvg8t\" (UID: \"ef46e568-5c98-4520-99e5-c49c4c3aca2a\") " pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:22.616592 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616036    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-run-openvswitch\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.616592 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616052    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/037937cc-75c9-4416-bd8f-63589df05625-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.617541 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616071    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bdedda0-279a-41fd-a7cc-300c816985df-ovnkube-config\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.617541 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616135    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-run-ovn\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.617541 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616139    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-sys\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.617541 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616177    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-run-multus-certs\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.617541 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616285    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-sysctl-conf\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.617541 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616342    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-etc-kubernetes\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.617541 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616369    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-run-netns\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.617541 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616369    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-run-multus-certs\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.617541 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616418    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmbj9\" (UniqueName: \"kubernetes.io/projected/0197ee7e-0d4a-4e9c-8057-26356d6f34b3-kube-api-access-xmbj9\") pod \"node-resolver-85452\" (UID: \"0197ee7e-0d4a-4e9c-8057-26356d6f34b3\") " pod="openshift-dns/node-resolver-85452"
Nov 01 05:02:22.617541 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616448    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-sys-fs\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.617541 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616489    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-var-lib-openvswitch\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.617541 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616503    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-etc-kubernetes\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.617541 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616538    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-var-lib-openvswitch\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.617541 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-run\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.617541 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616605    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-host\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.617541 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616643    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-run\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.617541 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616708    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-socket-dir-parent\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.617541 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616730    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw4b5\" (UniqueName: \"kubernetes.io/projected/ee3cc716-4716-4207-a493-9eeee140a6fe-kube-api-access-tw4b5\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.618265 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616777    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4qrj\" (UniqueName: \"kubernetes.io/projected/037937cc-75c9-4416-bd8f-63589df05625-kube-api-access-w4qrj\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.618265 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616141    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-etc-openvswitch\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.618265 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616810    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-host\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.618265 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616852    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-socket-dir-parent\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.618265 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:22.616904    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 05:02:22.618265 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616919    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-lib-modules\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.618265 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.616967    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/45a40b3e-ab72-4b18-933f-f52272a6742c-konnectivity-ca\") pod \"konnectivity-agent-jswq7\" (UID: \"45a40b3e-ab72-4b18-933f-f52272a6742c\") " pod="kube-system/konnectivity-agent-jswq7"
Nov 01 05:02:22.618265 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:22.617013    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs podName:ef46e568-5c98-4520-99e5-c49c4c3aca2a nodeName:}" failed. No retries permitted until 2026-11-01 05:02:23.116968928 +0000 UTC m=+3.119042092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs") pod "network-metrics-daemon-rvg8t" (UID: "ef46e568-5c98-4520-99e5-c49c4c3aca2a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 05:02:22.618265 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617041    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bb889bee-f31e-47f1-8617-6a27bc1eebf4-iptables-alerter-script\") pod \"iptables-alerter-nk4wf\" (UID: \"bb889bee-f31e-47f1-8617-6a27bc1eebf4\") " pod="openshift-network-operator/iptables-alerter-nk4wf"
Nov 01 05:02:22.618265 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617101    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-lib-modules\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.618265 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617121    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/037937cc-75c9-4416-bd8f-63589df05625-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.618265 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617144    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j8z2\" (UniqueName: \"kubernetes.io/projected/ef46e568-5c98-4520-99e5-c49c4c3aca2a-kube-api-access-2j8z2\") pod \"network-metrics-daemon-rvg8t\" (UID: \"ef46e568-5c98-4520-99e5-c49c4c3aca2a\") " pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:22.618265 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617191    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-sysconfig\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.618265 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617212    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-systemd\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.618265 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617249    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-var-lib-cni-bin\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.618265 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617313    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-var-lib-cni-bin\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.618265 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617321    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-sysconfig\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.619015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617318    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-systemd\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.619015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617366    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-etc-selinux\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.619015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617397    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-slash\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.619015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617490    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-slash\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.619015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617609    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-node-log\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.619015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617614    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/037937cc-75c9-4416-bd8f-63589df05625-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.619015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617654    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-node-log\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.619015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617690    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-cni-bin\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.619015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617716    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/45a40b3e-ab72-4b18-933f-f52272a6742c-agent-certs\") pod \"konnectivity-agent-jswq7\" (UID: \"45a40b3e-ab72-4b18-933f-f52272a6742c\") " pod="kube-system/konnectivity-agent-jswq7"
Nov 01 05:02:22.619015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617742    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/037937cc-75c9-4416-bd8f-63589df05625-system-cni-dir\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.619015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617766    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/037937cc-75c9-4416-bd8f-63589df05625-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.619015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617818    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-cni-bin\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.619015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617840    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/037937cc-75c9-4416-bd8f-63589df05625-system-cni-dir\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.619015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617834    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-kubelet\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.619015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617883    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdwk6\" (UniqueName: \"kubernetes.io/projected/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-kube-api-access-zdwk6\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.619015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617887    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-kubelet\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.619015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617910    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-daemon-config\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.619502 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617943    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/037937cc-75c9-4416-bd8f-63589df05625-cnibin\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.619502 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617960    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/037937cc-75c9-4416-bd8f-63589df05625-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.619502 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.617974    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-run-k8s-cni-cncf-io\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.619502 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618019    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-run-k8s-cni-cncf-io\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.619502 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618026    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/037937cc-75c9-4416-bd8f-63589df05625-cnibin\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.619502 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618048    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d1f4c040-7436-4df1-8462-6a38acee2c5a-serviceca\") pod \"node-ca-m9ckd\" (UID: \"d1f4c040-7436-4df1-8462-6a38acee2c5a\") " pod="openshift-image-registry/node-ca-m9ckd"
Nov 01 05:02:22.619502 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618080    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6vvf\" (UniqueName: \"kubernetes.io/projected/d1f4c040-7436-4df1-8462-6a38acee2c5a-kube-api-access-h6vvf\") pod \"node-ca-m9ckd\" (UID: \"d1f4c040-7436-4df1-8462-6a38acee2c5a\") " pod="openshift-image-registry/node-ca-m9ckd"
Nov 01 05:02:22.619502 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618114    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njxgf\" (UniqueName: \"kubernetes.io/projected/6a747e88-6f89-4f3a-ba71-c97babf998f6-kube-api-access-njxgf\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.619502 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618141    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-run-netns\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.619502 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618172    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.619502 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618203    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bdedda0-279a-41fd-a7cc-300c816985df-ovn-node-metrics-cert\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.619502 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618228    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-tuned\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.619502 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618243    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-system-cni-dir\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.619502 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618245    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.619502 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618266    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-cnibin\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.619502 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618287    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-run-netns\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.619502 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618322    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-system-cni-dir\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.620020 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618328    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-conf-dir\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.620020 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618413    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-systemd-units\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.620020 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618444    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bdedda0-279a-41fd-a7cc-300c816985df-ovnkube-script-lib\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.620020 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618452    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-cnibin\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.620020 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618465    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-systemd-units\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.620020 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618471    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-sysctl-d\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.620020 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618415    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-conf-dir\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.620020 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618468    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-daemon-config\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.620020 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618568    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-sysctl-d\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.620020 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-var-lib-kubelet\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.620020 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.618646    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-var-lib-kubelet\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.620020 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.619259    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-tmp\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.620020 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.619276    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bdedda0-279a-41fd-a7cc-300c816985df-ovnkube-script-lib\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.621391 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.621372    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-tuned\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.621545 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.621528    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bdedda0-279a-41fd-a7cc-300c816985df-ovn-node-metrics-cert\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.681074 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.681033    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxknn\" (UniqueName: \"kubernetes.io/projected/bb889bee-f31e-47f1-8617-6a27bc1eebf4-kube-api-access-bxknn\") pod \"iptables-alerter-nk4wf\" (UID: \"bb889bee-f31e-47f1-8617-6a27bc1eebf4\") " pod="openshift-network-operator/iptables-alerter-nk4wf"
Nov 01 05:02:22.681654 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.681623    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw4b5\" (UniqueName: \"kubernetes.io/projected/ee3cc716-4716-4207-a493-9eeee140a6fe-kube-api-access-tw4b5\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.681753 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.681654    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nh4k\" (UniqueName: \"kubernetes.io/projected/0bdedda0-279a-41fd-a7cc-300c816985df-kube-api-access-2nh4k\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.682761 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.682735    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4qrj\" (UniqueName: \"kubernetes.io/projected/037937cc-75c9-4416-bd8f-63589df05625-kube-api-access-w4qrj\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.682846 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.682779    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j8z2\" (UniqueName: \"kubernetes.io/projected/ef46e568-5c98-4520-99e5-c49c4c3aca2a-kube-api-access-2j8z2\") pod \"network-metrics-daemon-rvg8t\" (UID: \"ef46e568-5c98-4520-99e5-c49c4c3aca2a\") " pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:22.683070 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.683053    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdwk6\" (UniqueName: \"kubernetes.io/projected/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-kube-api-access-zdwk6\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.719208 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719181    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0197ee7e-0d4a-4e9c-8057-26356d6f34b3-hosts-file\") pod \"node-resolver-85452\" (UID: \"0197ee7e-0d4a-4e9c-8057-26356d6f34b3\") " pod="openshift-dns/node-resolver-85452"
Nov 01 05:02:22.719322 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719210    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzgwz\" (UniqueName: \"kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz\") pod \"network-check-target-tl74d\" (UID: \"a5b30117-7cb8-43ec-92a8-4e326b48c22d\") " pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:22.719322 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719232    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-socket-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.719322 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719250    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:22.719322 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719266    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0197ee7e-0d4a-4e9c-8057-26356d6f34b3-tmp-dir\") pod \"node-resolver-85452\" (UID: \"0197ee7e-0d4a-4e9c-8057-26356d6f34b3\") " pod="openshift-dns/node-resolver-85452"
Nov 01 05:02:22.719322 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719289    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/36a4e628-5f12-4849-9d77-deaa3e93b190-dbus\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:22.719322 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719316    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1f4c040-7436-4df1-8462-6a38acee2c5a-host\") pod \"node-ca-m9ckd\" (UID: \"d1f4c040-7436-4df1-8462-6a38acee2c5a\") " pod="openshift-image-registry/node-ca-m9ckd"
Nov 01 05:02:22.719639 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719319    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0197ee7e-0d4a-4e9c-8057-26356d6f34b3-hosts-file\") pod \"node-resolver-85452\" (UID: \"0197ee7e-0d4a-4e9c-8057-26356d6f34b3\") " pod="openshift-dns/node-resolver-85452"
Nov 01 05:02:22.719639 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719350    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.719639 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:22.719363    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 01 05:02:22.719639 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719378    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-device-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.719639 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719385    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/36a4e628-5f12-4849-9d77-deaa3e93b190-dbus\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:22.719639 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719392    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1f4c040-7436-4df1-8462-6a38acee2c5a-host\") pod \"node-ca-m9ckd\" (UID: \"d1f4c040-7436-4df1-8462-6a38acee2c5a\") " pod="openshift-image-registry/node-ca-m9ckd"
Nov 01 05:02:22.719639 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719398    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-socket-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.719639 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719422    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmbj9\" (UniqueName: \"kubernetes.io/projected/0197ee7e-0d4a-4e9c-8057-26356d6f34b3-kube-api-access-xmbj9\") pod \"node-resolver-85452\" (UID: \"0197ee7e-0d4a-4e9c-8057-26356d6f34b3\") " pod="openshift-dns/node-resolver-85452"
Nov 01 05:02:22.719639 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719453    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.719639 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719459    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-device-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.719639 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719477    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-sys-fs\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.719639 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719527    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-sys-fs\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.719639 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719527    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/45a40b3e-ab72-4b18-933f-f52272a6742c-konnectivity-ca\") pod \"konnectivity-agent-jswq7\" (UID: \"45a40b3e-ab72-4b18-933f-f52272a6742c\") " pod="kube-system/konnectivity-agent-jswq7"
Nov 01 05:02:22.719639 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:22.719560    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret podName:36a4e628-5f12-4849-9d77-deaa3e93b190 nodeName:}" failed. No retries permitted until 2026-11-01 05:02:23.219540618 +0000 UTC m=+3.221613771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret") pod "global-pull-secret-syncer-q92dh" (UID: "36a4e628-5f12-4849-9d77-deaa3e93b190") : object "kube-system"/"original-pull-secret" not registered
Nov 01 05:02:22.719639 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719606    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-etc-selinux\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.719639 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719634    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/45a40b3e-ab72-4b18-933f-f52272a6742c-agent-certs\") pod \"konnectivity-agent-jswq7\" (UID: \"45a40b3e-ab72-4b18-933f-f52272a6742c\") " pod="kube-system/konnectivity-agent-jswq7"
Nov 01 05:02:22.719639 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719640    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0197ee7e-0d4a-4e9c-8057-26356d6f34b3-tmp-dir\") pod \"node-resolver-85452\" (UID: \"0197ee7e-0d4a-4e9c-8057-26356d6f34b3\") " pod="openshift-dns/node-resolver-85452"
Nov 01 05:02:22.720306 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719687    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d1f4c040-7436-4df1-8462-6a38acee2c5a-serviceca\") pod \"node-ca-m9ckd\" (UID: \"d1f4c040-7436-4df1-8462-6a38acee2c5a\") " pod="openshift-image-registry/node-ca-m9ckd"
Nov 01 05:02:22.720306 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719719    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6vvf\" (UniqueName: \"kubernetes.io/projected/d1f4c040-7436-4df1-8462-6a38acee2c5a-kube-api-access-h6vvf\") pod \"node-ca-m9ckd\" (UID: \"d1f4c040-7436-4df1-8462-6a38acee2c5a\") " pod="openshift-image-registry/node-ca-m9ckd"
Nov 01 05:02:22.720306 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719751    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njxgf\" (UniqueName: \"kubernetes.io/projected/6a747e88-6f89-4f3a-ba71-c97babf998f6-kube-api-access-njxgf\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.720306 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719791    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-registration-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.720306 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719750    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-etc-selinux\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.720306 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719822    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/36a4e628-5f12-4849-9d77-deaa3e93b190-kubelet-config\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:22.720306 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719901    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-registration-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.720306 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.719924    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/36a4e628-5f12-4849-9d77-deaa3e93b190-kubelet-config\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:22.720306 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.720087    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/45a40b3e-ab72-4b18-933f-f52272a6742c-konnectivity-ca\") pod \"konnectivity-agent-jswq7\" (UID: \"45a40b3e-ab72-4b18-933f-f52272a6742c\") " pod="kube-system/konnectivity-agent-jswq7"
Nov 01 05:02:22.720306 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.720158    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d1f4c040-7436-4df1-8462-6a38acee2c5a-serviceca\") pod \"node-ca-m9ckd\" (UID: \"d1f4c040-7436-4df1-8462-6a38acee2c5a\") " pod="openshift-image-registry/node-ca-m9ckd"
Nov 01 05:02:22.722099 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.722075    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/45a40b3e-ab72-4b18-933f-f52272a6742c-agent-certs\") pod \"konnectivity-agent-jswq7\" (UID: \"45a40b3e-ab72-4b18-933f-f52272a6742c\") " pod="kube-system/konnectivity-agent-jswq7"
Nov 01 05:02:22.782928 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:22.782889    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 01 05:02:22.782928 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:22.782918    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 01 05:02:22.782928 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:22.782934    2577 projected.go:196] Error preparing data for projected volume kube-api-access-bzgwz for pod openshift-network-diagnostics/network-check-target-tl74d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 05:02:22.783127 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:22.783032    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz podName:a5b30117-7cb8-43ec-92a8-4e326b48c22d nodeName:}" failed. No retries permitted until 2026-11-01 05:02:23.283007058 +0000 UTC m=+3.285080223 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bzgwz" (UniqueName: "kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz") pod "network-check-target-tl74d" (UID: "a5b30117-7cb8-43ec-92a8-4e326b48c22d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 05:02:22.785085 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.785060    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6vvf\" (UniqueName: \"kubernetes.io/projected/d1f4c040-7436-4df1-8462-6a38acee2c5a-kube-api-access-h6vvf\") pod \"node-ca-m9ckd\" (UID: \"d1f4c040-7436-4df1-8462-6a38acee2c5a\") " pod="openshift-image-registry/node-ca-m9ckd"
Nov 01 05:02:22.785183 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.785151    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmbj9\" (UniqueName: \"kubernetes.io/projected/0197ee7e-0d4a-4e9c-8057-26356d6f34b3-kube-api-access-xmbj9\") pod \"node-resolver-85452\" (UID: \"0197ee7e-0d4a-4e9c-8057-26356d6f34b3\") " pod="openshift-dns/node-resolver-85452"
Nov 01 05:02:22.785219 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.785202    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njxgf\" (UniqueName: \"kubernetes.io/projected/6a747e88-6f89-4f3a-ba71-c97babf998f6-kube-api-access-njxgf\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.818870 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.818841    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:02:22.825588 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.825565    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nk4wf"
Nov 01 05:02:22.829505 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.829490    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:02:22.829926 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:22.829899    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd09454b6_6ac7_46dc_83ee_a3c60dfdf262.slice/crio-c753c1ad8bec900d09bc899c73f0a962a126099532eb2ad2dcfdf88c274a806e WatchSource:0}: Error finding container c753c1ad8bec900d09bc899c73f0a962a126099532eb2ad2dcfdf88c274a806e: Status 404 returned error can't find the container with id c753c1ad8bec900d09bc899c73f0a962a126099532eb2ad2dcfdf88c274a806e
Nov 01 05:02:22.831368 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.831351    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gwbm4"
Nov 01 05:02:22.834403 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.834381    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-85452"
Nov 01 05:02:22.834993 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:22.834970    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb889bee_f31e_47f1_8617_6a27bc1eebf4.slice/crio-5fcb2a03ebf60efd422b67e506e0739906758c878f7bf46ee7ce4f792bd0a10b WatchSource:0}: Error finding container 5fcb2a03ebf60efd422b67e506e0739906758c878f7bf46ee7ce4f792bd0a10b: Status 404 returned error can't find the container with id 5fcb2a03ebf60efd422b67e506e0739906758c878f7bf46ee7ce4f792bd0a10b
Nov 01 05:02:22.841451 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:22.841430    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod037937cc_75c9_4416_bd8f_63589df05625.slice/crio-9cb17b91c845339f4cc640538f1e369ed1363d614799a796d296164bf9b93658 WatchSource:0}: Error finding container 9cb17b91c845339f4cc640538f1e369ed1363d614799a796d296164bf9b93658: Status 404 returned error can't find the container with id 9cb17b91c845339f4cc640538f1e369ed1363d614799a796d296164bf9b93658
Nov 01 05:02:22.842515 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.842497    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jswq7"
Nov 01 05:02:22.843996 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:22.843973    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee3cc716_4716_4207_a493_9eeee140a6fe.slice/crio-94ecffcf8dc81a911dd5e6c02fac43e13e1ab76ddd815f0a6deff64dcc59a08a WatchSource:0}: Error finding container 94ecffcf8dc81a911dd5e6c02fac43e13e1ab76ddd815f0a6deff64dcc59a08a: Status 404 returned error can't find the container with id 94ecffcf8dc81a911dd5e6c02fac43e13e1ab76ddd815f0a6deff64dcc59a08a
Nov 01 05:02:22.844999 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.844939    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m9ckd"
Nov 01 05:02:22.846993 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:22.846968    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0197ee7e_0d4a_4e9c_8057_26356d6f34b3.slice/crio-3cb3ab0bf1302a3e326f502441612a54ec24b932a304b4484211e0d8adbf31c1 WatchSource:0}: Error finding container 3cb3ab0bf1302a3e326f502441612a54ec24b932a304b4484211e0d8adbf31c1: Status 404 returned error can't find the container with id 3cb3ab0bf1302a3e326f502441612a54ec24b932a304b4484211e0d8adbf31c1
Nov 01 05:02:22.847849 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.847834    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:22.853304 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:22.853279    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45a40b3e_ab72_4b18_933f_f52272a6742c.slice/crio-273a50863469fb5a4e75ef299b4cdbe92adb5a2508cc1fa9e2e393bd457b5a4d WatchSource:0}: Error finding container 273a50863469fb5a4e75ef299b4cdbe92adb5a2508cc1fa9e2e393bd457b5a4d: Status 404 returned error can't find the container with id 273a50863469fb5a4e75ef299b4cdbe92adb5a2508cc1fa9e2e393bd457b5a4d
Nov 01 05:02:22.854827 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:22.854808    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:02:22.857908 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:22.857886    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1f4c040_7436_4df1_8462_6a38acee2c5a.slice/crio-fa241532a7ae7a09f847da65293301bad95da33ebfb7b6ff170ce467ce27815a WatchSource:0}: Error finding container fa241532a7ae7a09f847da65293301bad95da33ebfb7b6ff170ce467ce27815a: Status 404 returned error can't find the container with id fa241532a7ae7a09f847da65293301bad95da33ebfb7b6ff170ce467ce27815a
Nov 01 05:02:22.858789 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:22.858755    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bdedda0_279a_41fd_a7cc_300c816985df.slice/crio-c6b287a466534cdc830fed20da4cf5655443d448ebdb9857c93bd0bc6c5cb3b8 WatchSource:0}: Error finding container c6b287a466534cdc830fed20da4cf5655443d448ebdb9857c93bd0bc6c5cb3b8: Status 404 returned error can't find the container with id c6b287a466534cdc830fed20da4cf5655443d448ebdb9857c93bd0bc6c5cb3b8
Nov 01 05:02:22.865976 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:22.865956    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a747e88_6f89_4f3a_ba71_c97babf998f6.slice/crio-5c9e60c7c7a0e875b7ac9eca1c09f7ae1992f2153833836e1d22d79bf5e54c8e WatchSource:0}: Error finding container 5c9e60c7c7a0e875b7ac9eca1c09f7ae1992f2153833836e1d22d79bf5e54c8e: Status 404 returned error can't find the container with id 5c9e60c7c7a0e875b7ac9eca1c09f7ae1992f2153833836e1d22d79bf5e54c8e
Nov 01 05:02:23.122739 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:23.122641    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs\") pod \"network-metrics-daemon-rvg8t\" (UID: \"ef46e568-5c98-4520-99e5-c49c4c3aca2a\") " pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:23.122930 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:23.122788    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 05:02:23.122930 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:23.122880    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs podName:ef46e568-5c98-4520-99e5-c49c4c3aca2a nodeName:}" failed. No retries permitted until 2026-11-01 05:02:24.122859253 +0000 UTC m=+4.124932390 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs") pod "network-metrics-daemon-rvg8t" (UID: "ef46e568-5c98-4520-99e5-c49c4c3aca2a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 05:02:23.223686 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:23.223644    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:23.223872 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:23.223852    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 01 05:02:23.223965 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:23.223926    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret podName:36a4e628-5f12-4849-9d77-deaa3e93b190 nodeName:}" failed. No retries permitted until 2026-11-01 05:02:24.223901479 +0000 UTC m=+4.225974619 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret") pod "global-pull-secret-syncer-q92dh" (UID: "36a4e628-5f12-4849-9d77-deaa3e93b190") : object "kube-system"/"original-pull-secret" not registered
Nov 01 05:02:23.325131 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:23.325094    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzgwz\" (UniqueName: \"kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz\") pod \"network-check-target-tl74d\" (UID: \"a5b30117-7cb8-43ec-92a8-4e326b48c22d\") " pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:23.325323 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:23.325230    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 01 05:02:23.325323 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:23.325246    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 01 05:02:23.325323 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:23.325255    2577 projected.go:196] Error preparing data for projected volume kube-api-access-bzgwz for pod openshift-network-diagnostics/network-check-target-tl74d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 05:02:23.325323 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:23.325309    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz podName:a5b30117-7cb8-43ec-92a8-4e326b48c22d nodeName:}" failed. No retries permitted until 2026-11-01 05:02:24.325286201 +0000 UTC m=+4.327359338 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bzgwz" (UniqueName: "kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz") pod "network-check-target-tl74d" (UID: "a5b30117-7cb8-43ec-92a8-4e326b48c22d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 05:02:23.484964 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:23.484867    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89wdz" event={"ID":"0bdedda0-279a-41fd-a7cc-300c816985df","Type":"ContainerStarted","Data":"c6b287a466534cdc830fed20da4cf5655443d448ebdb9857c93bd0bc6c5cb3b8"}
Nov 01 05:02:23.486418 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:23.486382    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jswq7" event={"ID":"45a40b3e-ab72-4b18-933f-f52272a6742c","Type":"ContainerStarted","Data":"273a50863469fb5a4e75ef299b4cdbe92adb5a2508cc1fa9e2e393bd457b5a4d"}
Nov 01 05:02:23.487647 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:23.487622    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gwbm4" event={"ID":"ee3cc716-4716-4207-a493-9eeee140a6fe","Type":"ContainerStarted","Data":"94ecffcf8dc81a911dd5e6c02fac43e13e1ab76ddd815f0a6deff64dcc59a08a"}
Nov 01 05:02:23.488891 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:23.488863    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nk4wf" event={"ID":"bb889bee-f31e-47f1-8617-6a27bc1eebf4","Type":"ContainerStarted","Data":"5fcb2a03ebf60efd422b67e506e0739906758c878f7bf46ee7ce4f792bd0a10b"}
Nov 01 05:02:23.490134 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:23.490098    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m9ckd" event={"ID":"d1f4c040-7436-4df1-8462-6a38acee2c5a","Type":"ContainerStarted","Data":"fa241532a7ae7a09f847da65293301bad95da33ebfb7b6ff170ce467ce27815a"}
Nov 01 05:02:23.491616 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:23.491567    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-85452" event={"ID":"0197ee7e-0d4a-4e9c-8057-26356d6f34b3","Type":"ContainerStarted","Data":"3cb3ab0bf1302a3e326f502441612a54ec24b932a304b4484211e0d8adbf31c1"}
Nov 01 05:02:23.493346 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:23.493296    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6ht6" event={"ID":"037937cc-75c9-4416-bd8f-63589df05625","Type":"ContainerStarted","Data":"9cb17b91c845339f4cc640538f1e369ed1363d614799a796d296164bf9b93658"}
Nov 01 05:02:23.494843 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:23.494763    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m8d7h" event={"ID":"d09454b6-6ac7-46dc-83ee-a3c60dfdf262","Type":"ContainerStarted","Data":"c753c1ad8bec900d09bc899c73f0a962a126099532eb2ad2dcfdf88c274a806e"}
Nov 01 05:02:23.496308 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:23.496260    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd" event={"ID":"6a747e88-6f89-4f3a-ba71-c97babf998f6","Type":"ContainerStarted","Data":"5c9e60c7c7a0e875b7ac9eca1c09f7ae1992f2153833836e1d22d79bf5e54c8e"}
Nov 01 05:02:23.600065 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:23.599998    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-31 04:57:23 +0000 UTC" deadline="2028-08-09 19:14:21.924239372 +0000 UTC"
Nov 01 05:02:23.600065 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:23.600046    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15542h11m58.324196084s"
Nov 01 05:02:24.132596 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:24.132556    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs\") pod \"network-metrics-daemon-rvg8t\" (UID: \"ef46e568-5c98-4520-99e5-c49c4c3aca2a\") " pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:24.132826 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:24.132719    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 05:02:24.132826 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:24.132815    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs podName:ef46e568-5c98-4520-99e5-c49c4c3aca2a nodeName:}" failed. No retries permitted until 2026-11-01 05:02:26.132773535 +0000 UTC m=+6.134846668 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs") pod "network-metrics-daemon-rvg8t" (UID: "ef46e568-5c98-4520-99e5-c49c4c3aca2a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 05:02:24.233299 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:24.232954    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:24.233299 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:24.233120    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 01 05:02:24.233299 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:24.233180    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret podName:36a4e628-5f12-4849-9d77-deaa3e93b190 nodeName:}" failed. No retries permitted until 2026-11-01 05:02:26.23316093 +0000 UTC m=+6.235234066 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret") pod "global-pull-secret-syncer-q92dh" (UID: "36a4e628-5f12-4849-9d77-deaa3e93b190") : object "kube-system"/"original-pull-secret" not registered
Nov 01 05:02:24.333619 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:24.333323    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzgwz\" (UniqueName: \"kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz\") pod \"network-check-target-tl74d\" (UID: \"a5b30117-7cb8-43ec-92a8-4e326b48c22d\") " pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:24.333619 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:24.333497    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 01 05:02:24.333619 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:24.333516    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 01 05:02:24.333619 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:24.333530    2577 projected.go:196] Error preparing data for projected volume kube-api-access-bzgwz for pod openshift-network-diagnostics/network-check-target-tl74d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 05:02:24.333619 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:24.333589    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz podName:a5b30117-7cb8-43ec-92a8-4e326b48c22d nodeName:}" failed. No retries permitted until 2026-11-01 05:02:26.333566295 +0000 UTC m=+6.335639444 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bzgwz" (UniqueName: "kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz") pod "network-check-target-tl74d" (UID: "a5b30117-7cb8-43ec-92a8-4e326b48c22d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 05:02:24.485015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:24.484001    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:24.485015 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:24.484164    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvg8t" podUID="ef46e568-5c98-4520-99e5-c49c4c3aca2a"
Nov 01 05:02:24.485015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:24.484255    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:24.485015 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:24.484328    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q92dh" podUID="36a4e628-5f12-4849-9d77-deaa3e93b190"
Nov 01 05:02:24.485015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:24.484402    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:24.485015 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:24.484471    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl74d" podUID="a5b30117-7cb8-43ec-92a8-4e326b48c22d"
Nov 01 05:02:24.600954 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:24.600904    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-31 04:57:23 +0000 UTC" deadline="2028-07-30 19:59:56.274801024 +0000 UTC"
Nov 01 05:02:24.600954 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:24.600951    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15302h57m31.673855783s"
Nov 01 05:02:25.514444 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:25.514208    2577 generic.go:356] "Generic (PLEG): container finished" podID="f72249826b80cce6b307e39e964ca081" containerID="15a12c78f0fbd68eec29581885426179bf7003ab69bf4e55a0c903c0d34c6249" exitCode=0
Nov 01 05:02:25.514896 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:25.514473    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal" event={"ID":"f72249826b80cce6b307e39e964ca081","Type":"ContainerDied","Data":"15a12c78f0fbd68eec29581885426179bf7003ab69bf4e55a0c903c0d34c6249"}
Nov 01 05:02:26.153078 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:26.152988    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs\") pod \"network-metrics-daemon-rvg8t\" (UID: \"ef46e568-5c98-4520-99e5-c49c4c3aca2a\") " pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:26.153257 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:26.153137    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 05:02:26.153257 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:26.153201    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs podName:ef46e568-5c98-4520-99e5-c49c4c3aca2a nodeName:}" failed. No retries permitted until 2026-11-01 05:02:30.153182168 +0000 UTC m=+10.155255305 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs") pod "network-metrics-daemon-rvg8t" (UID: "ef46e568-5c98-4520-99e5-c49c4c3aca2a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 05:02:26.254058 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:26.253374    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:26.254058 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:26.253586    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 01 05:02:26.254058 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:26.253653    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret podName:36a4e628-5f12-4849-9d77-deaa3e93b190 nodeName:}" failed. No retries permitted until 2026-11-01 05:02:30.253632655 +0000 UTC m=+10.255705810 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret") pod "global-pull-secret-syncer-q92dh" (UID: "36a4e628-5f12-4849-9d77-deaa3e93b190") : object "kube-system"/"original-pull-secret" not registered
Nov 01 05:02:26.354485 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:26.354038    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzgwz\" (UniqueName: \"kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz\") pod \"network-check-target-tl74d\" (UID: \"a5b30117-7cb8-43ec-92a8-4e326b48c22d\") " pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:26.354485 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:26.354324    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 01 05:02:26.354485 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:26.354349    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 01 05:02:26.354485 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:26.354365    2577 projected.go:196] Error preparing data for projected volume kube-api-access-bzgwz for pod openshift-network-diagnostics/network-check-target-tl74d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 05:02:26.354485 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:26.354451    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz podName:a5b30117-7cb8-43ec-92a8-4e326b48c22d nodeName:}" failed. No retries permitted until 2026-11-01 05:02:30.354429137 +0000 UTC m=+10.356502295 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bzgwz" (UniqueName: "kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz") pod "network-check-target-tl74d" (UID: "a5b30117-7cb8-43ec-92a8-4e326b48c22d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 05:02:26.477893 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:26.476084    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:26.477893 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:26.476210    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl74d" podUID="a5b30117-7cb8-43ec-92a8-4e326b48c22d"
Nov 01 05:02:26.477893 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:26.476596    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:26.477893 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:26.476708    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvg8t" podUID="ef46e568-5c98-4520-99e5-c49c4c3aca2a"
Nov 01 05:02:26.477893 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:26.476770    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:26.477893 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:26.476840    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q92dh" podUID="36a4e628-5f12-4849-9d77-deaa3e93b190"
Nov 01 05:02:26.553963 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:26.553134    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal" event={"ID":"f72249826b80cce6b307e39e964ca081","Type":"ContainerStarted","Data":"fce34e94868ccc120ea970a54bf3762aec3b5601f8d380c8506b8addd40d7584"}
Nov 01 05:02:26.563948 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:26.563345    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6ht6" event={"ID":"037937cc-75c9-4416-bd8f-63589df05625","Type":"ContainerStarted","Data":"cc33c90d2bcb9d8ac3e212ae4edb2dd664a5216ff84b08c87c28b8ed90772039"}
Nov 01 05:02:26.572989 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:26.572589    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal" event={"ID":"1c1427a4a27caa7c2e9df3986def0a2a","Type":"ContainerStarted","Data":"5c64cedd44856c8229fa05011eb151f97f1bee4f59accbad7f01e94c871d24bd"}
Nov 01 05:02:26.810952 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:26.810818    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal" podStartSLOduration=3.81079312 podStartE2EDuration="3.81079312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 05:02:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:02:26.678856265 +0000 UTC m=+6.680929432" watchObservedRunningTime="2026-11-01 05:02:26.81079312 +0000 UTC m=+6.812866286"
Nov 01 05:02:27.582698 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:27.582658    2577 generic.go:356] "Generic (PLEG): container finished" podID="037937cc-75c9-4416-bd8f-63589df05625" containerID="cc33c90d2bcb9d8ac3e212ae4edb2dd664a5216ff84b08c87c28b8ed90772039" exitCode=0
Nov 01 05:02:27.583783 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:27.583754    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6ht6" event={"ID":"037937cc-75c9-4416-bd8f-63589df05625","Type":"ContainerDied","Data":"cc33c90d2bcb9d8ac3e212ae4edb2dd664a5216ff84b08c87c28b8ed90772039"}
Nov 01 05:02:27.718027 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:27.717966    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal" podStartSLOduration=4.717944331 podStartE2EDuration="4.717944331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 05:02:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:02:26.93572928 +0000 UTC m=+6.937802444" watchObservedRunningTime="2026-11-01 05:02:27.717944331 +0000 UTC m=+7.720017489"
Nov 01 05:02:28.476361 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:28.476325    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:28.476582 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:28.476469    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl74d" podUID="a5b30117-7cb8-43ec-92a8-4e326b48c22d"
Nov 01 05:02:28.476582 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:28.476536    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:28.476692 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:28.476619    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvg8t" podUID="ef46e568-5c98-4520-99e5-c49c4c3aca2a"
Nov 01 05:02:28.476692 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:28.476665    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:28.476772 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:28.476736    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q92dh" podUID="36a4e628-5f12-4849-9d77-deaa3e93b190"
Nov 01 05:02:28.591154 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:28.591108    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jswq7" event={"ID":"45a40b3e-ab72-4b18-933f-f52272a6742c","Type":"ContainerStarted","Data":"ed3c741b3de430233c9abc70e5899afa2fb89b5dadf77468b0ecd3bd0de488de"}
Nov 01 05:02:28.714639 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:28.714577    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jswq7" podStartSLOduration=1.7059426 podStartE2EDuration="6.714547561s" totalImagesPullingTime="5.008604961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:02:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:02:28.714229466 +0000 UTC m=+8.716302630" watchObservedRunningTime="2026-11-01 05:02:28.714547561 +0000 UTC m=+8.716620720"
Nov 01 05:02:29.598787 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:29.598747    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m9ckd" event={"ID":"d1f4c040-7436-4df1-8462-6a38acee2c5a","Type":"ContainerStarted","Data":"cdd357d8d7c87903095fbb03f61f21032ae7277af052f2ff99225e3f78fa8155"}
Nov 01 05:02:29.722959 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:29.722891    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-m9ckd" podStartSLOduration=1.5472796039999999 podStartE2EDuration="7.722869354s" totalImagesPullingTime="6.17558975s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 05:02:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:02:29.722307532 +0000 UTC m=+9.724380690" watchObservedRunningTime="2026-11-01 05:02:29.722869354 +0000 UTC m=+9.724942504"
Nov 01 05:02:30.186493 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:30.186405    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs\") pod \"network-metrics-daemon-rvg8t\" (UID: \"ef46e568-5c98-4520-99e5-c49c4c3aca2a\") " pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:30.186677 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:30.186583    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 05:02:30.186677 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:30.186661    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs podName:ef46e568-5c98-4520-99e5-c49c4c3aca2a nodeName:}" failed. No retries permitted until 2026-11-01 05:02:38.186638156 +0000 UTC m=+18.188711294 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs") pod "network-metrics-daemon-rvg8t" (UID: "ef46e568-5c98-4520-99e5-c49c4c3aca2a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 05:02:30.287911 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:30.287841    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:30.288118 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:30.288013    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 01 05:02:30.288118 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:30.288079    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret podName:36a4e628-5f12-4849-9d77-deaa3e93b190 nodeName:}" failed. No retries permitted until 2026-11-01 05:02:38.288059288 +0000 UTC m=+18.290132440 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret") pod "global-pull-secret-syncer-q92dh" (UID: "36a4e628-5f12-4849-9d77-deaa3e93b190") : object "kube-system"/"original-pull-secret" not registered
Nov 01 05:02:30.388663 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:30.388617    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzgwz\" (UniqueName: \"kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz\") pod \"network-check-target-tl74d\" (UID: \"a5b30117-7cb8-43ec-92a8-4e326b48c22d\") " pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:30.388859 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:30.388840    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 01 05:02:30.388859 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:30.388861    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 01 05:02:30.388859 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:30.388876    2577 projected.go:196] Error preparing data for projected volume kube-api-access-bzgwz for pod openshift-network-diagnostics/network-check-target-tl74d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 05:02:30.389111 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:30.388967    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz podName:a5b30117-7cb8-43ec-92a8-4e326b48c22d nodeName:}" failed. No retries permitted until 2026-11-01 05:02:38.388920189 +0000 UTC m=+18.390993351 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bzgwz" (UniqueName: "kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz") pod "network-check-target-tl74d" (UID: "a5b30117-7cb8-43ec-92a8-4e326b48c22d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 05:02:30.476929 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:30.476832    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:30.477093 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:30.476989    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl74d" podUID="a5b30117-7cb8-43ec-92a8-4e326b48c22d"
Nov 01 05:02:30.477456 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:30.477433    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:30.477636 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:30.477548    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvg8t" podUID="ef46e568-5c98-4520-99e5-c49c4c3aca2a"
Nov 01 05:02:30.477722 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:30.477704    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:30.477833 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:30.477811    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q92dh" podUID="36a4e628-5f12-4849-9d77-deaa3e93b190"
Nov 01 05:02:30.604720 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:30.604681    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd" event={"ID":"6a747e88-6f89-4f3a-ba71-c97babf998f6","Type":"ContainerStarted","Data":"75d02432fe4eecfc62eba32c05e8bd0d0d8d3b9bf12e9e9fbb95f74d6ef728dd"}
Nov 01 05:02:30.933781 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:30.933703    2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jswq7"
Nov 01 05:02:30.934122 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:30.934100    2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jswq7"
Nov 01 05:02:32.204461 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:32.204430    2577 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 01 05:02:32.476628 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:32.476539    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:32.476784 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:32.476686    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q92dh" podUID="36a4e628-5f12-4849-9d77-deaa3e93b190"
Nov 01 05:02:32.476784 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:32.476762    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:32.476921 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:32.476864    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl74d" podUID="a5b30117-7cb8-43ec-92a8-4e326b48c22d"
Nov 01 05:02:32.477043 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:32.477027    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:32.477153 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:32.477134    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvg8t" podUID="ef46e568-5c98-4520-99e5-c49c4c3aca2a"
Nov 01 05:02:32.491507 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:32.491383    2577 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-01T05:02:32.204457935Z","UUID":"f438b539-c644-44e4-a1d5-00896c53d224","Handler":null,"Name":"","Endpoint":""}
Nov 01 05:02:32.493116 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:32.493093    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 01 05:02:32.493235 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:32.493126    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 01 05:02:32.612678 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:32.612645    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd" event={"ID":"6a747e88-6f89-4f3a-ba71-c97babf998f6","Type":"ContainerStarted","Data":"42861951ef73b8f9548e712cb341dbd267551580ee8b46bdce555392ef962047"}
Nov 01 05:02:32.614768 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:32.614738    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gwbm4" event={"ID":"ee3cc716-4716-4207-a493-9eeee140a6fe","Type":"ContainerStarted","Data":"2aac462f7a2d9698cb2868f4d1b264519f87fdce4a828fdfca0b97d9f0c7a779"}
Nov 01 05:02:32.617014 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:32.616984    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-85452" event={"ID":"0197ee7e-0d4a-4e9c-8057-26356d6f34b3","Type":"ContainerStarted","Data":"68904dd0dcc6e9da7dc82d7d7255d13510e4b8ebc59709eafe079c912168899e"}
Nov 01 05:02:32.741373 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:32.741263    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-gwbm4" podStartSLOduration=2.949719896 podStartE2EDuration="11.741243922s" totalImagesPullingTime="8.791524026s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 05:02:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:02:32.740580006 +0000 UTC m=+12.742653166" watchObservedRunningTime="2026-11-01 05:02:32.741243922 +0000 UTC m=+12.743317080"
Nov 01 05:02:33.623989 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:33.623939    2577 generic.go:356] "Generic (PLEG): container finished" podID="037937cc-75c9-4416-bd8f-63589df05625" containerID="17a7da1fd3d42b0cfc389fc7eca716b4f6a24b066b4eeef259416e1e97b82928" exitCode=0
Nov 01 05:02:33.624785 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:33.624061    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6ht6" event={"ID":"037937cc-75c9-4416-bd8f-63589df05625","Type":"ContainerDied","Data":"17a7da1fd3d42b0cfc389fc7eca716b4f6a24b066b4eeef259416e1e97b82928"}
Nov 01 05:02:33.629323 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:33.629299    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nk4wf" event={"ID":"bb889bee-f31e-47f1-8617-6a27bc1eebf4","Type":"ContainerStarted","Data":"1e8eee8dc1b5036c961f17e7623ab2f2513d391b8cde7531f8a282dafa39c284"}
Nov 01 05:02:33.773989 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:33.773877    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-85452" podStartSLOduration=2.4167261 podStartE2EDuration="11.773853483s" totalImagesPullingTime="9.357127383s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 05:02:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:02:32.863971155 +0000 UTC m=+12.866044312" watchObservedRunningTime="2026-11-01 05:02:33.773853483 +0000 UTC m=+13.775926642"
Nov 01 05:02:34.475994 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:34.475967    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:34.476118 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:34.475975    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:34.476118 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:34.476093    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q92dh" podUID="36a4e628-5f12-4849-9d77-deaa3e93b190"
Nov 01 05:02:34.476118 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:34.476098    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:34.476252 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:34.476202    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvg8t" podUID="ef46e568-5c98-4520-99e5-c49c4c3aca2a"
Nov 01 05:02:34.476304 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:34.476262    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl74d" podUID="a5b30117-7cb8-43ec-92a8-4e326b48c22d"
Nov 01 05:02:34.633917 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:34.633879    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd" event={"ID":"6a747e88-6f89-4f3a-ba71-c97babf998f6","Type":"ContainerStarted","Data":"22117567e2acd41964d512f4b92df940681e9ecb1c7390b367a93aecc5e5dde9"}
Nov 01 05:02:34.637049 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:34.636990    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m8d7h" event={"ID":"d09454b6-6ac7-46dc-83ee-a3c60dfdf262","Type":"ContainerStarted","Data":"0b8f0cf523f71a50ac2d0ccb17d88c00f65539481d8d9b7d10be3ced14a8aa80"}
Nov 01 05:02:34.751329 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:34.751205    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nk4wf" podStartSLOduration=4.628960596 podStartE2EDuration="13.751186948s" totalImagesPullingTime="9.122226352s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 05:02:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:02:33.893021625 +0000 UTC m=+13.895094778" watchObservedRunningTime="2026-11-01 05:02:34.751186948 +0000 UTC m=+14.753260101"
Nov 01 05:02:34.751594 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:34.751563    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd" podStartSLOduration=1.544062341 podStartE2EDuration="12.751551189s" totalImagesPullingTime="11.207488848s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 05:02:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:02:34.750668304 +0000 UTC m=+14.752741457" watchObservedRunningTime="2026-11-01 05:02:34.751551189 +0000 UTC m=+14.753624342"
Nov 01 05:02:34.867958 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:34.867910    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-m8d7h" podStartSLOduration=2.265443975 podStartE2EDuration="13.867896827s" totalImagesPullingTime="11.602452852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 05:02:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:02:34.867326785 +0000 UTC m=+14.869399938" watchObservedRunningTime="2026-11-01 05:02:34.867896827 +0000 UTC m=+14.869969981"
Nov 01 05:02:34.988323 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:34.988288    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jswq7"
Nov 01 05:02:34.988501 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:34.988415    2577 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 01 05:02:34.988930 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:34.988911    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jswq7"
Nov 01 05:02:35.642035 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:35.641954    2577 generic.go:356] "Generic (PLEG): container finished" podID="037937cc-75c9-4416-bd8f-63589df05625" containerID="830433732fb06158fc96420a38f6905d39231e1fe4d7f66adb013cb454906191" exitCode=0
Nov 01 05:02:35.642595 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:35.642047    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6ht6" event={"ID":"037937cc-75c9-4416-bd8f-63589df05625","Type":"ContainerDied","Data":"830433732fb06158fc96420a38f6905d39231e1fe4d7f66adb013cb454906191"}
Nov 01 05:02:36.476323 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:36.476106    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:36.476507 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:36.476154    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:36.476507 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:36.476426    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl74d" podUID="a5b30117-7cb8-43ec-92a8-4e326b48c22d"
Nov 01 05:02:36.476507 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:36.476199    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:36.476658 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:36.476483    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvg8t" podUID="ef46e568-5c98-4520-99e5-c49c4c3aca2a"
Nov 01 05:02:36.476658 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:36.476599    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q92dh" podUID="36a4e628-5f12-4849-9d77-deaa3e93b190"
Nov 01 05:02:37.648394 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:37.648360    2577 generic.go:356] "Generic (PLEG): container finished" podID="037937cc-75c9-4416-bd8f-63589df05625" containerID="82c0241cd126dd94e764d8579f64fc37c278283e9297961f82a58be0f0db15b4" exitCode=0
Nov 01 05:02:37.648921 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:37.648411    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6ht6" event={"ID":"037937cc-75c9-4416-bd8f-63589df05625","Type":"ContainerDied","Data":"82c0241cd126dd94e764d8579f64fc37c278283e9297961f82a58be0f0db15b4"}
Nov 01 05:02:38.250031 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:38.249997    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs\") pod \"network-metrics-daemon-rvg8t\" (UID: \"ef46e568-5c98-4520-99e5-c49c4c3aca2a\") " pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:38.250228 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:38.250154    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 05:02:38.250228 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:38.250208    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs podName:ef46e568-5c98-4520-99e5-c49c4c3aca2a nodeName:}" failed. No retries permitted until 2026-11-01 05:02:54.250192064 +0000 UTC m=+34.252265195 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs") pod "network-metrics-daemon-rvg8t" (UID: "ef46e568-5c98-4520-99e5-c49c4c3aca2a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 05:02:38.350662 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:38.350627    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:38.350863 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:38.350820    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 01 05:02:38.350912 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:38.350892    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret podName:36a4e628-5f12-4849-9d77-deaa3e93b190 nodeName:}" failed. No retries permitted until 2026-11-01 05:02:54.350871122 +0000 UTC m=+34.352944258 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret") pod "global-pull-secret-syncer-q92dh" (UID: "36a4e628-5f12-4849-9d77-deaa3e93b190") : object "kube-system"/"original-pull-secret" not registered
Nov 01 05:02:38.451413 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:38.451378    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzgwz\" (UniqueName: \"kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz\") pod \"network-check-target-tl74d\" (UID: \"a5b30117-7cb8-43ec-92a8-4e326b48c22d\") " pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:38.451595 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:38.451531    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 01 05:02:38.451595 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:38.451551    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 01 05:02:38.451595 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:38.451563    2577 projected.go:196] Error preparing data for projected volume kube-api-access-bzgwz for pod openshift-network-diagnostics/network-check-target-tl74d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 05:02:38.451733 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:38.451621    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz podName:a5b30117-7cb8-43ec-92a8-4e326b48c22d nodeName:}" failed. No retries permitted until 2026-11-01 05:02:54.451602064 +0000 UTC m=+34.453675215 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bzgwz" (UniqueName: "kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz") pod "network-check-target-tl74d" (UID: "a5b30117-7cb8-43ec-92a8-4e326b48c22d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 05:02:38.475602 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:38.475577    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:38.475602 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:38.475591    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:38.475799 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:38.475588    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:38.475799 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:38.475702    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvg8t" podUID="ef46e568-5c98-4520-99e5-c49c4c3aca2a"
Nov 01 05:02:38.475894 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:38.475777    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q92dh" podUID="36a4e628-5f12-4849-9d77-deaa3e93b190"
Nov 01 05:02:38.475894 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:38.475866    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl74d" podUID="a5b30117-7cb8-43ec-92a8-4e326b48c22d"
Nov 01 05:02:38.651149 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:38.651123    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89wdz" event={"ID":"0bdedda0-279a-41fd-a7cc-300c816985df","Type":"ContainerStarted","Data":"8bceb557a200f9a4d228dd7c9555022d1ca5e45f377923a4c12563cc0fbb5cd7"}
Nov 01 05:02:39.656574 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:39.656409    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89wdz" event={"ID":"0bdedda0-279a-41fd-a7cc-300c816985df","Type":"ContainerStarted","Data":"6b5398d9061df9175de45bc8bf56c7301662f6c5892a9257d1699f0454f52d45"}
Nov 01 05:02:39.656879 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:39.656634    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89wdz" event={"ID":"0bdedda0-279a-41fd-a7cc-300c816985df","Type":"ContainerStarted","Data":"d7ccaa8bf740e688ab32bb8011a91a925b6de2b088059321f61fce4fbae765d5"}
Nov 01 05:02:39.656879 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:39.656663    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89wdz" event={"ID":"0bdedda0-279a-41fd-a7cc-300c816985df","Type":"ContainerStarted","Data":"144ee4d514586cd71e7711c7777d5e5a8d335b468fc79b68c0fba03803bc2b75"}
Nov 01 05:02:39.656879 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:39.656691    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89wdz" event={"ID":"0bdedda0-279a-41fd-a7cc-300c816985df","Type":"ContainerStarted","Data":"e390a22cadd1fce229e6f20856725f60d8e093aeacc9261784ff94c6d1d27a19"}
Nov 01 05:02:39.656879 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:39.656716    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89wdz" event={"ID":"0bdedda0-279a-41fd-a7cc-300c816985df","Type":"ContainerStarted","Data":"6b07aa74ad0ed25121bc31e9a0cc2da9e7047aaaec97e35fd217a60bf41d3bc2"}
Nov 01 05:02:40.476624 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:40.476586    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:40.476835 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:40.476722    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:40.476835 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:40.476743    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q92dh" podUID="36a4e628-5f12-4849-9d77-deaa3e93b190"
Nov 01 05:02:40.476835 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:40.476776    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl74d" podUID="a5b30117-7cb8-43ec-92a8-4e326b48c22d"
Nov 01 05:02:40.477003 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:40.476846    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:40.477003 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:40.476928    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvg8t" podUID="ef46e568-5c98-4520-99e5-c49c4c3aca2a"
Nov 01 05:02:40.660583 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:40.660552    2577 generic.go:356] "Generic (PLEG): container finished" podID="037937cc-75c9-4416-bd8f-63589df05625" containerID="743ede4a72caa98cdee31e3610e9fa408b72535a4e830add8d8a0c61c4b17333" exitCode=0
Nov 01 05:02:40.661153 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:40.660593    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6ht6" event={"ID":"037937cc-75c9-4416-bd8f-63589df05625","Type":"ContainerDied","Data":"743ede4a72caa98cdee31e3610e9fa408b72535a4e830add8d8a0c61c4b17333"}
Nov 01 05:02:41.665929 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:41.665883    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89wdz" event={"ID":"0bdedda0-279a-41fd-a7cc-300c816985df","Type":"ContainerStarted","Data":"74ff532bf0b8bfa02cf608769a3f2f1eb559b2bd1a448b17ee3ab7b2e9881b72"}
Nov 01 05:02:41.668666 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:41.668642    2577 generic.go:356] "Generic (PLEG): container finished" podID="037937cc-75c9-4416-bd8f-63589df05625" containerID="9c39cf5c3fd64c0d3a409845499f530bcbc26eeb6b80ff85a77a1576f14136d7" exitCode=0
Nov 01 05:02:41.668804 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:41.668674    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6ht6" event={"ID":"037937cc-75c9-4416-bd8f-63589df05625","Type":"ContainerDied","Data":"9c39cf5c3fd64c0d3a409845499f530bcbc26eeb6b80ff85a77a1576f14136d7"}
Nov 01 05:02:42.476065 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:42.476031    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:42.476065 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:42.476052    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:42.476321 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:42.476140    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:42.476321 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:42.476144    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl74d" podUID="a5b30117-7cb8-43ec-92a8-4e326b48c22d"
Nov 01 05:02:42.476321 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:42.476228    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q92dh" podUID="36a4e628-5f12-4849-9d77-deaa3e93b190"
Nov 01 05:02:42.476321 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:42.476308    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvg8t" podUID="ef46e568-5c98-4520-99e5-c49c4c3aca2a"
Nov 01 05:02:42.674472 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:42.674443    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6ht6" event={"ID":"037937cc-75c9-4416-bd8f-63589df05625","Type":"ContainerStarted","Data":"9cf3fd5500e1c2ff000b51b077bdff58ecaac75b85e56d746aeed509a57b956a"}
Nov 01 05:02:42.795424 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:42.795323    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-h6ht6" podStartSLOduration=7.8405720819999996 podStartE2EDuration="21.795310008s" totalImagesPullingTime="13.954737926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 05:02:21 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:02:42.794928112 +0000 UTC m=+22.797001265" watchObservedRunningTime="2026-11-01 05:02:42.795310008 +0000 UTC m=+22.797383160"
Nov 01 05:02:43.682694 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:43.682657    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89wdz" event={"ID":"0bdedda0-279a-41fd-a7cc-300c816985df","Type":"ContainerStarted","Data":"c5d63c4291dd901c07ac854494bc466efd74d36435dda579a3e3becb0ad59f41"}
Nov 01 05:02:43.958152 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:43.957905    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-89wdz" podStartSLOduration=7.242674579 podStartE2EDuration="22.957890635s" totalImagesPullingTime="15.715216056s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 05:02:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:02:43.908197888 +0000 UTC m=+23.910271040" watchObservedRunningTime="2026-11-01 05:02:43.957890635 +0000 UTC m=+23.959963787"
Nov 01 05:02:44.475597 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:44.475556    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:44.475759 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:44.475574    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:44.475759 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:44.475665    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q92dh" podUID="36a4e628-5f12-4849-9d77-deaa3e93b190"
Nov 01 05:02:44.475759 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:44.475688    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:44.475891 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:44.475812    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvg8t" podUID="ef46e568-5c98-4520-99e5-c49c4c3aca2a"
Nov 01 05:02:44.475922 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:44.475879    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl74d" podUID="a5b30117-7cb8-43ec-92a8-4e326b48c22d"
Nov 01 05:02:44.685218 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:44.685184    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:44.685613 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:44.685271    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:44.685613 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:44.685316    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:44.701194 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:44.701167    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:44.701387 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:44.701374    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:02:45.788581 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:45.788553    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q92dh"]
Nov 01 05:02:45.789069 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:45.788844    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:45.789069 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:45.788971    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q92dh" podUID="36a4e628-5f12-4849-9d77-deaa3e93b190"
Nov 01 05:02:45.791359 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:45.791334    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rvg8t"]
Nov 01 05:02:45.791454 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:45.791436    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:45.791542 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:45.791526    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvg8t" podUID="ef46e568-5c98-4520-99e5-c49c4c3aca2a"
Nov 01 05:02:45.795338 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:45.795312    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tl74d"]
Nov 01 05:02:45.795427 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:45.795384    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:45.795470 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:45.795460    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl74d" podUID="a5b30117-7cb8-43ec-92a8-4e326b48c22d"
Nov 01 05:02:47.475646 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:47.475578    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:47.475646 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:47.475621    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:47.476022 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:47.475578    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:47.476022 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:47.475700    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q92dh" podUID="36a4e628-5f12-4849-9d77-deaa3e93b190"
Nov 01 05:02:47.476022 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:47.475758    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvg8t" podUID="ef46e568-5c98-4520-99e5-c49c4c3aca2a"
Nov 01 05:02:47.476022 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:47.475838    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl74d" podUID="a5b30117-7cb8-43ec-92a8-4e326b48c22d"
Nov 01 05:02:49.475561 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:49.475528    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:49.475561 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:49.475550    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:49.476140 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:49.475528    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:49.476140 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:49.475646    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q92dh" podUID="36a4e628-5f12-4849-9d77-deaa3e93b190"
Nov 01 05:02:49.476140 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:49.475718    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvg8t" podUID="ef46e568-5c98-4520-99e5-c49c4c3aca2a"
Nov 01 05:02:49.476140 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:49.475769    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl74d" podUID="a5b30117-7cb8-43ec-92a8-4e326b48c22d"
Nov 01 05:02:51.476082 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:51.476007    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:51.476443 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:51.476010    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:51.476443 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:51.476114    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvg8t" podUID="ef46e568-5c98-4520-99e5-c49c4c3aca2a"
Nov 01 05:02:51.476443 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:51.476146    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:51.476443 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:51.476231    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q92dh" podUID="36a4e628-5f12-4849-9d77-deaa3e93b190"
Nov 01 05:02:51.476443 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:51.476293    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tl74d" podUID="a5b30117-7cb8-43ec-92a8-4e326b48c22d"
Nov 01 05:02:51.834847 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:51.834651    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeReady"
Nov 01 05:02:51.835025 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:51.834967    2577 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 01 05:02:51.953567 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:51.953536    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-d2cm4"]
Nov 01 05:02:51.995016 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:51.994980    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-qb994"]
Nov 01 05:02:51.995189 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:51.995110    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-d2cm4"
Nov 01 05:02:52.008490 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.008469    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-qszhw"]
Nov 01 05:02:52.008602 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.008581    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-qb994"
Nov 01 05:02:52.033402 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.033371    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6969bc8dd6-85rsc"]
Nov 01 05:02:52.034024 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.033779    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-qszhw"
Nov 01 05:02:52.052751 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.052700    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 01 05:02:52.052751 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.052717    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 01 05:02:52.053400 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.053369    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-6twtw"]
Nov 01 05:02:52.053594 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.053574    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.071580 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.071559    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27"]
Nov 01 05:02:52.071677 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.071644    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:02:52.087931 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.087883    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27"
Nov 01 05:02:52.087931 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.087888    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"]
Nov 01 05:02:52.101015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.100993    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-79659c8779-dnfbf"]
Nov 01 05:02:52.101139 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.101120    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:02:52.106022 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.105999    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 01 05:02:52.106022 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.105999    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 01 05:02:52.106163 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.106047    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-n5nx4\""
Nov 01 05:02:52.106163 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.106137    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 01 05:02:52.115025 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.115006    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz"]
Nov 01 05:02:52.115139 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.115123    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.129225 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.129203    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"]
Nov 01 05:02:52.129369 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.129355    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz"
Nov 01 05:02:52.143466 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.143450    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-cvn58"]
Nov 01 05:02:52.143574 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.143560    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:02:52.147946 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.147848    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 01 05:02:52.147946 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.147856    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 01 05:02:52.147946 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.147895    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 01 05:02:52.147946 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.147915    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 01 05:02:52.147946 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.147923    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 01 05:02:52.148252 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.147999    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 01 05:02:52.152520 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.152502    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48c05e4f-a673-4f5f-8c21-8dc1ff716683-image-registry-private-configuration\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.152596 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.152528    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhm7l\" (UniqueName: \"kubernetes.io/projected/fb78a5ab-3a2d-45fa-abaa-7b26c33ec929-kube-api-access-dhm7l\") pod \"kas-connection-checker-685895c7c9-d2cm4\" (UID: \"fb78a5ab-3a2d-45fa-abaa-7b26c33ec929\") " pod="kube-system/kas-connection-checker-685895c7c9-d2cm4"
Nov 01 05:02:52.152596 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.152544    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bce36747-2159-4b72-8c0e-d98226588d18-config\") pod \"service-ca-operator-544d749c8c-97k27\" (UID: \"bce36747-2159-4b72-8c0e-d98226588d18\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27"
Nov 01 05:02:52.152596 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.152560    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkbbr\" (UniqueName: \"kubernetes.io/projected/bce36747-2159-4b72-8c0e-d98226588d18-kube-api-access-hkbbr\") pod \"service-ca-operator-544d749c8c-97k27\" (UID: \"bce36747-2159-4b72-8c0e-d98226588d18\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27"
Nov 01 05:02:52.152684 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.152618    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvgdb\" (UniqueName: \"kubernetes.io/projected/b91ced44-8be6-47c8-9229-549400076eb7-kube-api-access-hvgdb\") pod \"kas-connection-checker-685895c7c9-qszhw\" (UID: \"b91ced44-8be6-47c8-9229-549400076eb7\") " pod="kube-system/kas-connection-checker-685895c7c9-qszhw"
Nov 01 05:02:52.152684 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.152658    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.152744 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.152683    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9hd4\" (UniqueName: \"kubernetes.io/projected/df7d0108-685b-4c27-8d0f-bccf4ca5dbbe-kube-api-access-z9hd4\") pod \"kas-connection-checker-685895c7c9-qb994\" (UID: \"df7d0108-685b-4c27-8d0f-bccf4ca5dbbe\") " pod="kube-system/kas-connection-checker-685895c7c9-qb994"
Nov 01 05:02:52.152813 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.152799    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjg2z\" (UniqueName: \"kubernetes.io/projected/878896d2-fa82-46ef-af5e-0bf491216b03-kube-api-access-qjg2z\") pod \"console-operator-7cd56b8dbc-6twtw\" (UID: \"878896d2-fa82-46ef-af5e-0bf491216b03\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:02:52.152850 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.152823    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bce36747-2159-4b72-8c0e-d98226588d18-serving-cert\") pod \"service-ca-operator-544d749c8c-97k27\" (UID: \"bce36747-2159-4b72-8c0e-d98226588d18\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27"
Nov 01 05:02:52.152881 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.152856    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48c05e4f-a673-4f5f-8c21-8dc1ff716683-trusted-ca\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.152949 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.152933    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48c05e4f-a673-4f5f-8c21-8dc1ff716683-ca-trust-extracted\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.152984 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.152975    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48c05e4f-a673-4f5f-8c21-8dc1ff716683-installation-pull-secrets\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.153029 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.153018    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-certificates\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.153058 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.153036    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/878896d2-fa82-46ef-af5e-0bf491216b03-trusted-ca\") pod \"console-operator-7cd56b8dbc-6twtw\" (UID: \"878896d2-fa82-46ef-af5e-0bf491216b03\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:02:52.153089 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.153065    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-bound-sa-token\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.153121 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.153106    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/878896d2-fa82-46ef-af5e-0bf491216b03-config\") pod \"console-operator-7cd56b8dbc-6twtw\" (UID: \"878896d2-fa82-46ef-af5e-0bf491216b03\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:02:52.153152 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.153125    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/878896d2-fa82-46ef-af5e-0bf491216b03-serving-cert\") pod \"console-operator-7cd56b8dbc-6twtw\" (UID: \"878896d2-fa82-46ef-af5e-0bf491216b03\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:02:52.153179 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.153160    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdspf\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-kube-api-access-rdspf\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.157575 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.157560    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-74dd76c985-n79rt"]
Nov 01 05:02:52.157698 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.157684    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:02:52.183006 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.182982    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j67hv"]
Nov 01 05:02:52.183097 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.183086    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:52.198249 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.198228    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 01 05:02:52.198389 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.198235    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 01 05:02:52.198389 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.198255    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 01 05:02:52.198559 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.198294    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 01 05:02:52.198559 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.198294    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 01 05:02:52.198559 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.198316    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 01 05:02:52.198559 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.198325    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 01 05:02:52.198559 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.198367    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 01 05:02:52.198559 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.198440    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 01 05:02:52.198559 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.198519    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 01 05:02:52.198752 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.198610    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 01 05:02:52.204503 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.204488    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"]
Nov 01 05:02:52.204622 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.204607    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j67hv"
Nov 01 05:02:52.206015 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.205992    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 01 05:02:52.206110 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.206023    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 01 05:02:52.206169 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.206109    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 01 05:02:52.206169 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.206117    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 01 05:02:52.206169 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.206151    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 01 05:02:52.209427 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.209410    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 01 05:02:52.209530 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.209518    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 01 05:02:52.209618 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.209602    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 01 05:02:52.209675 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.209658    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 01 05:02:52.211143 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.211124    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 01 05:02:52.226548 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.226522    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-srv7d"]
Nov 01 05:02:52.226651 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.226637    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"
Nov 01 05:02:52.234885 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.234869    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 01 05:02:52.235030 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.235009    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 01 05:02:52.235125 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.235042    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 01 05:02:52.235125 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.235115    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 01 05:02:52.235218 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.235129    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 01 05:02:52.235218 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.235166    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 01 05:02:52.249644 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.249626    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wcrwt"]
Nov 01 05:02:52.249751 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.249738    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-srv7d"
Nov 01 05:02:52.253988 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.253966    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9hd4\" (UniqueName: \"kubernetes.io/projected/df7d0108-685b-4c27-8d0f-bccf4ca5dbbe-kube-api-access-z9hd4\") pod \"kas-connection-checker-685895c7c9-qb994\" (UID: \"df7d0108-685b-4c27-8d0f-bccf4ca5dbbe\") " pod="kube-system/kas-connection-checker-685895c7c9-qb994"
Nov 01 05:02:52.254068 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.254002    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjg2z\" (UniqueName: \"kubernetes.io/projected/878896d2-fa82-46ef-af5e-0bf491216b03-kube-api-access-qjg2z\") pod \"console-operator-7cd56b8dbc-6twtw\" (UID: \"878896d2-fa82-46ef-af5e-0bf491216b03\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:02:52.254068 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.254033    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.254170 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.254064    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.254170 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.254155    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pj75\" (UniqueName: \"kubernetes.io/projected/636cccf5-2397-4dca-899a-3598a7b8096b-kube-api-access-5pj75\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz\" (UID: \"636cccf5-2397-4dca-899a-3598a7b8096b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz"
Nov 01 05:02:52.254240 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.254170    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 05:02:52.254240 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.254184    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6969bc8dd6-85rsc: secret "image-registry-tls" not found
Nov 01 05:02:52.254240 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.254198    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:52.254387 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.254245    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls podName:48c05e4f-a673-4f5f-8c21-8dc1ff716683 nodeName:}" failed. No retries permitted until 2026-11-01 05:02:52.754216405 +0000 UTC m=+32.756289540 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls") pod "image-registry-6969bc8dd6-85rsc" (UID: "48c05e4f-a673-4f5f-8c21-8dc1ff716683") : secret "image-registry-tls" not found
Nov 01 05:02:52.254387 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.254263    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bce36747-2159-4b72-8c0e-d98226588d18-serving-cert\") pod \"service-ca-operator-544d749c8c-97k27\" (UID: \"bce36747-2159-4b72-8c0e-d98226588d18\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27"
Nov 01 05:02:52.254387 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.254297    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8l8g\" (UniqueName: \"kubernetes.io/projected/82335b9e-5584-4afc-b1fc-cc003dbcad4b-kube-api-access-d8l8g\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:52.254387 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.254327    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1967227b-a4ea-4ee6-945d-efb0b5040a13-serving-cert\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:02:52.254387 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.254378    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfldn\" (UniqueName: \"kubernetes.io/projected/1967227b-a4ea-4ee6-945d-efb0b5040a13-kube-api-access-xfldn\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:02:52.254645 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.254416    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:02:52.254645 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.254451    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n4xr\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-kube-api-access-9n4xr\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.254645 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.254516    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48c05e4f-a673-4f5f-8c21-8dc1ff716683-installation-pull-secrets\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.254645 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.254558    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1967227b-a4ea-4ee6-945d-efb0b5040a13-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:02:52.254645 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.254606    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/62652d56-1be6-47cc-a774-77d4fa9d206a-trusted-ca\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.254645 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.254633    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-stats-auth\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:52.255009 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.254711    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/62652d56-1be6-47cc-a774-77d4fa9d206a-image-registry-private-configuration\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.255009 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.254771    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/636cccf5-2397-4dca-899a-3598a7b8096b-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz\" (UID: \"636cccf5-2397-4dca-899a-3598a7b8096b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz"
Nov 01 05:02:52.255009 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.254844    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/878896d2-fa82-46ef-af5e-0bf491216b03-serving-cert\") pod \"console-operator-7cd56b8dbc-6twtw\" (UID: \"878896d2-fa82-46ef-af5e-0bf491216b03\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:02:52.255915 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.255897    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/62652d56-1be6-47cc-a774-77d4fa9d206a-ca-trust-extracted\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.256008 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.255960    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhm7l\" (UniqueName: \"kubernetes.io/projected/fb78a5ab-3a2d-45fa-abaa-7b26c33ec929-kube-api-access-dhm7l\") pod \"kas-connection-checker-685895c7c9-d2cm4\" (UID: \"fb78a5ab-3a2d-45fa-abaa-7b26c33ec929\") " pod="kube-system/kas-connection-checker-685895c7c9-d2cm4"
Nov 01 05:02:52.256088 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.256022    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bce36747-2159-4b72-8c0e-d98226588d18-config\") pod \"service-ca-operator-544d749c8c-97k27\" (UID: \"bce36747-2159-4b72-8c0e-d98226588d18\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27"
Nov 01 05:02:52.257413 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.256103    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-certificates\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.257413 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.256318    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvgdb\" (UniqueName: \"kubernetes.io/projected/b91ced44-8be6-47c8-9229-549400076eb7-kube-api-access-hvgdb\") pod \"kas-connection-checker-685895c7c9-qszhw\" (UID: \"b91ced44-8be6-47c8-9229-549400076eb7\") " pod="kube-system/kas-connection-checker-685895c7c9-qszhw"
Nov 01 05:02:52.257413 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.256342    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/62652d56-1be6-47cc-a774-77d4fa9d206a-installation-pull-secrets\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.257413 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.256396    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:52.257413 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.256444    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48c05e4f-a673-4f5f-8c21-8dc1ff716683-trusted-ca\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.257413 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.256472    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:02:52.257413 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.256499    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1967227b-a4ea-4ee6-945d-efb0b5040a13-snapshots\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:02:52.257413 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.256522    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-default-certificate\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:52.257413 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.256545    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48c05e4f-a673-4f5f-8c21-8dc1ff716683-ca-trust-extracted\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.257413 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.256567    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlhpw\" (UniqueName: \"kubernetes.io/projected/7fb489a8-8a4f-45f2-977b-594b6126281b-kube-api-access-hlhpw\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:02:52.257413 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.256594    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-bound-sa-token\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.257413 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.256618    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:02:52.257413 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.256646    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/636cccf5-2397-4dca-899a-3598a7b8096b-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz\" (UID: \"636cccf5-2397-4dca-899a-3598a7b8096b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz"
Nov 01 05:02:52.257413 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.256675    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-certificates\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.257413 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.256697    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/878896d2-fa82-46ef-af5e-0bf491216b03-trusted-ca\") pod \"console-operator-7cd56b8dbc-6twtw\" (UID: \"878896d2-fa82-46ef-af5e-0bf491216b03\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:02:52.258180 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.256735    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:02:52.258180 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.256743    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bce36747-2159-4b72-8c0e-d98226588d18-config\") pod \"service-ca-operator-544d749c8c-97k27\" (UID: \"bce36747-2159-4b72-8c0e-d98226588d18\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27"
Nov 01 05:02:52.258180 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.256774    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6qrt\" (UniqueName: \"kubernetes.io/projected/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-kube-api-access-h6qrt\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:02:52.258180 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.256899    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1967227b-a4ea-4ee6-945d-efb0b5040a13-service-ca-bundle\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:02:52.258180 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.257512    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-bound-sa-token\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.258180 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.257561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/878896d2-fa82-46ef-af5e-0bf491216b03-config\") pod \"console-operator-7cd56b8dbc-6twtw\" (UID: \"878896d2-fa82-46ef-af5e-0bf491216b03\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:02:52.258180 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.257660    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdspf\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-kube-api-access-rdspf\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.258180 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.257703    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkbbr\" (UniqueName: \"kubernetes.io/projected/bce36747-2159-4b72-8c0e-d98226588d18-kube-api-access-hkbbr\") pod \"service-ca-operator-544d749c8c-97k27\" (UID: \"bce36747-2159-4b72-8c0e-d98226588d18\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27"
Nov 01 05:02:52.258180 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.257729    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1967227b-a4ea-4ee6-945d-efb0b5040a13-tmp\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:02:52.258180 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.257909    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-certificates\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.258180 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.257938    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48c05e4f-a673-4f5f-8c21-8dc1ff716683-image-registry-private-configuration\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.258180 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.257995    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48c05e4f-a673-4f5f-8c21-8dc1ff716683-trusted-ca\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.258180 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.258025    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/878896d2-fa82-46ef-af5e-0bf491216b03-trusted-ca\") pod \"console-operator-7cd56b8dbc-6twtw\" (UID: \"878896d2-fa82-46ef-af5e-0bf491216b03\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:02:52.258180 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.258175    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48c05e4f-a673-4f5f-8c21-8dc1ff716683-ca-trust-extracted\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.259094 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.258586    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-hjhjw\""
Nov 01 05:02:52.259193 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.259171    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48c05e4f-a673-4f5f-8c21-8dc1ff716683-installation-pull-secrets\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.259252 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.259203    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bce36747-2159-4b72-8c0e-d98226588d18-serving-cert\") pod \"service-ca-operator-544d749c8c-97k27\" (UID: \"bce36747-2159-4b72-8c0e-d98226588d18\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27"
Nov 01 05:02:52.259322 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.259304    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/878896d2-fa82-46ef-af5e-0bf491216b03-config\") pod \"console-operator-7cd56b8dbc-6twtw\" (UID: \"878896d2-fa82-46ef-af5e-0bf491216b03\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:02:52.259510 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.259488    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 01 05:02:52.260383 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.260360    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48c05e4f-a673-4f5f-8c21-8dc1ff716683-image-registry-private-configuration\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.261913 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.261884    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 01 05:02:52.262397 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.262377    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/878896d2-fa82-46ef-af5e-0bf491216b03-serving-cert\") pod \"console-operator-7cd56b8dbc-6twtw\" (UID: \"878896d2-fa82-46ef-af5e-0bf491216b03\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:02:52.277677 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.277657    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:02:52.277677 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.277660    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-n8nnj"]
Nov 01 05:02:52.278459 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.278437    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 01 05:02:52.293271 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.293255    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-d2cm4"]
Nov 01 05:02:52.293356 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.293284    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-qszhw"]
Nov 01 05:02:52.293356 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.293306    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-qb994"]
Nov 01 05:02:52.293356 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.293326    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27"]
Nov 01 05:02:52.293356 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.293346    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"]
Nov 01 05:02:52.293471 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.293368    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n8nnj"
Nov 01 05:02:52.293471 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.293370    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6969bc8dd6-85rsc"]
Nov 01 05:02:52.293539 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.293507    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-cvn58"]
Nov 01 05:02:52.293539 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.293536    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"]
Nov 01 05:02:52.293688 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.293560    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74dd76c985-n79rt"]
Nov 01 05:02:52.293688 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.293580    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-srv7d"]
Nov 01 05:02:52.293688 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.293598    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j67hv"]
Nov 01 05:02:52.293688 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.293614    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-6twtw"]
Nov 01 05:02:52.293688 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.293632    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz"]
Nov 01 05:02:52.293688 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.293645    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wcrwt"]
Nov 01 05:02:52.293688 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.293663    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79659c8779-dnfbf"]
Nov 01 05:02:52.293688 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.293678    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"]
Nov 01 05:02:52.293972 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.293696    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n8nnj"]
Nov 01 05:02:52.301646 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.301627    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 01 05:02:52.301724 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.301667    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 01 05:02:52.305990 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.305968    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 01 05:02:52.306111 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.306098    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-mk4l6\""
Nov 01 05:02:52.306148 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.306099    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-7fb86\""
Nov 01 05:02:52.314945 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.314898    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9hd4\" (UniqueName: \"kubernetes.io/projected/df7d0108-685b-4c27-8d0f-bccf4ca5dbbe-kube-api-access-z9hd4\") pod \"kas-connection-checker-685895c7c9-qb994\" (UID: \"df7d0108-685b-4c27-8d0f-bccf4ca5dbbe\") " pod="kube-system/kas-connection-checker-685895c7c9-qb994"
Nov 01 05:02:52.315277 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.315251    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjg2z\" (UniqueName: \"kubernetes.io/projected/878896d2-fa82-46ef-af5e-0bf491216b03-kube-api-access-qjg2z\") pod \"console-operator-7cd56b8dbc-6twtw\" (UID: \"878896d2-fa82-46ef-af5e-0bf491216b03\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:02:52.315846 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.315822    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhm7l\" (UniqueName: \"kubernetes.io/projected/fb78a5ab-3a2d-45fa-abaa-7b26c33ec929-kube-api-access-dhm7l\") pod \"kas-connection-checker-685895c7c9-d2cm4\" (UID: \"fb78a5ab-3a2d-45fa-abaa-7b26c33ec929\") " pod="kube-system/kas-connection-checker-685895c7c9-d2cm4"
Nov 01 05:02:52.316077 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.316052    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvgdb\" (UniqueName: \"kubernetes.io/projected/b91ced44-8be6-47c8-9229-549400076eb7-kube-api-access-hvgdb\") pod \"kas-connection-checker-685895c7c9-qszhw\" (UID: \"b91ced44-8be6-47c8-9229-549400076eb7\") " pod="kube-system/kas-connection-checker-685895c7c9-qszhw"
Nov 01 05:02:52.317248 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.317219    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-bound-sa-token\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.317416 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.317398    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdspf\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-kube-api-access-rdspf\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.317889 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.317874    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkbbr\" (UniqueName: \"kubernetes.io/projected/bce36747-2159-4b72-8c0e-d98226588d18-kube-api-access-hkbbr\") pod \"service-ca-operator-544d749c8c-97k27\" (UID: \"bce36747-2159-4b72-8c0e-d98226588d18\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27"
Nov 01 05:02:52.334447 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.334430    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-qb994"
Nov 01 05:02:52.358360 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.358336    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-qszhw"
Nov 01 05:02:52.358565 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.358544    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:02:52.358639 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.358572    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6qrt\" (UniqueName: \"kubernetes.io/projected/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-kube-api-access-h6qrt\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:02:52.358639 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.358595    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1967227b-a4ea-4ee6-945d-efb0b5040a13-service-ca-bundle\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:02:52.358639 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.358625    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9lkh\" (UniqueName: \"kubernetes.io/projected/57fa3738-c3e8-4c18-9582-f6e8377a041e-kube-api-access-p9lkh\") pod \"network-check-source-6f78d59f99-srv7d\" (UID: \"57fa3738-c3e8-4c18-9582-f6e8377a041e\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-srv7d"
Nov 01 05:02:52.358821 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.358669    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-ht4j4\" (UID: \"f6185c12-8b19-47d8-b0b3-01aa0f1192a7\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"
Nov 01 05:02:52.358821 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.358699    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b53e86a9-ca94-4753-b57b-47eb94bbd879-cert\") pod \"ingress-canary-n8nnj\" (UID: \"b53e86a9-ca94-4753-b57b-47eb94bbd879\") " pod="openshift-ingress-canary/ingress-canary-n8nnj"
Nov 01 05:02:52.358821 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.358726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1967227b-a4ea-4ee6-945d-efb0b5040a13-tmp\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:02:52.358821 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.358806    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.359031 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.358853    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pj75\" (UniqueName: \"kubernetes.io/projected/636cccf5-2397-4dca-899a-3598a7b8096b-kube-api-access-5pj75\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz\" (UID: \"636cccf5-2397-4dca-899a-3598a7b8096b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz"
Nov 01 05:02:52.359031 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.358881    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:52.359031 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.358912    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8l8g\" (UniqueName: \"kubernetes.io/projected/82335b9e-5584-4afc-b1fc-cc003dbcad4b-kube-api-access-d8l8g\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:52.359031 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.358938    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1967227b-a4ea-4ee6-945d-efb0b5040a13-serving-cert\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:02:52.359031 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.358959    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfldn\" (UniqueName: \"kubernetes.io/projected/1967227b-a4ea-4ee6-945d-efb0b5040a13-kube-api-access-xfldn\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:02:52.359031 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.358989    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:02:52.359031 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359018    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n4xr\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-kube-api-access-9n4xr\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.359398 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359045    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dc708b0-6c35-4f79-a415-e7533381b95b-tmp-dir\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:02:52.359398 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359087    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1967227b-a4ea-4ee6-945d-efb0b5040a13-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:02:52.359398 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359093    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1967227b-a4ea-4ee6-945d-efb0b5040a13-tmp\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:02:52.359398 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359114    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/62652d56-1be6-47cc-a774-77d4fa9d206a-trusted-ca\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.359398 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359140    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-stats-auth\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:52.359398 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.359173    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 05:02:52.359398 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359175    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/62652d56-1be6-47cc-a774-77d4fa9d206a-image-registry-private-configuration\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.359398 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.359186    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79659c8779-dnfbf: secret "image-registry-tls" not found
Nov 01 05:02:52.359398 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359208    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/636cccf5-2397-4dca-899a-3598a7b8096b-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz\" (UID: \"636cccf5-2397-4dca-899a-3598a7b8096b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz"
Nov 01 05:02:52.359398 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.359214    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 01 05:02:52.359398 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.359230    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls podName:62652d56-1be6-47cc-a774-77d4fa9d206a nodeName:}" failed. No retries permitted until 2026-11-01 05:02:52.859215237 +0000 UTC m=+32.861288393 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls") pod "image-registry-79659c8779-dnfbf" (UID: "62652d56-1be6-47cc-a774-77d4fa9d206a") : secret "image-registry-tls" not found
Nov 01 05:02:52.359398 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359266    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/62652d56-1be6-47cc-a774-77d4fa9d206a-ca-trust-extracted\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.359398 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.359278    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls podName:7fb489a8-8a4f-45f2-977b-594b6126281b nodeName:}" failed. No retries permitted until 2026-11-01 05:02:52.859262081 +0000 UTC m=+32.861335231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-d8pkk" (UID: "7fb489a8-8a4f-45f2-977b-594b6126281b") : secret "samples-operator-tls" not found
Nov 01 05:02:52.359398 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359308    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:02:52.359398 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359317    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:02:52.359398 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359342    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-certificates\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.360761 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359368    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wblsz\" (UniqueName: \"kubernetes.io/projected/5dc708b0-6c35-4f79-a415-e7533381b95b-kube-api-access-wblsz\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:02:52.360761 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359381    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1967227b-a4ea-4ee6-945d-efb0b5040a13-service-ca-bundle\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:02:52.360761 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359392    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj67m\" (UniqueName: \"kubernetes.io/projected/b53e86a9-ca94-4753-b57b-47eb94bbd879-kube-api-access-bj67m\") pod \"ingress-canary-n8nnj\" (UID: \"b53e86a9-ca94-4753-b57b-47eb94bbd879\") " pod="openshift-ingress-canary/ingress-canary-n8nnj"
Nov 01 05:02:52.360761 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359468    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-nginx-conf\") pod \"networking-console-plugin-86c478bb88-ht4j4\" (UID: \"f6185c12-8b19-47d8-b0b3-01aa0f1192a7\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"
Nov 01 05:02:52.360761 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359499    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/62652d56-1be6-47cc-a774-77d4fa9d206a-installation-pull-secrets\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.360761 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359551    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/62652d56-1be6-47cc-a774-77d4fa9d206a-ca-trust-extracted\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.360761 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359552    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5dc708b0-6c35-4f79-a415-e7533381b95b-config-volume\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:02:52.360761 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359710    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 01 05:02:52.360761 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359718    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 01 05:02:52.360761 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.359778    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 01 05:02:52.360761 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.360202    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:52.360761 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.360323    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle podName:82335b9e-5584-4afc-b1fc-cc003dbcad4b nodeName:}" failed. No retries permitted until 2026-11-01 05:02:52.860308027 +0000 UTC m=+32.862381176 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle") pod "router-default-74dd76c985-n79rt" (UID: "82335b9e-5584-4afc-b1fc-cc003dbcad4b") : configmap references non-existent config key: service-ca.crt
Nov 01 05:02:52.360761 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.360397    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:02:52.360761 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.360427    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1967227b-a4ea-4ee6-945d-efb0b5040a13-snapshots\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:02:52.360761 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.360457    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-default-certificate\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:52.360761 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.360488    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlhpw\" (UniqueName: \"kubernetes.io/projected/7fb489a8-8a4f-45f2-977b-594b6126281b-kube-api-access-hlhpw\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:02:52.360761 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.360541    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1967227b-a4ea-4ee6-945d-efb0b5040a13-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:02:52.361583 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.360552    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6b98\" (UniqueName: \"kubernetes.io/projected/a01a018f-2c6e-413f-b88f-37b4e91699b1-kube-api-access-v6b98\") pod \"volume-data-source-validator-58c9f5f46d-j67hv\" (UID: \"a01a018f-2c6e-413f-b88f-37b4e91699b1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j67hv"
Nov 01 05:02:52.361583 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.360584    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-bound-sa-token\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.361583 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.360609    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:02:52.361583 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.360639    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/636cccf5-2397-4dca-899a-3598a7b8096b-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz\" (UID: \"636cccf5-2397-4dca-899a-3598a7b8096b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz"
Nov 01 05:02:52.361583 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.360983    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 01 05:02:52.361583 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.361033    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls podName:c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47 nodeName:}" failed. No retries permitted until 2026-11-01 05:02:52.861018854 +0000 UTC m=+32.863092002 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-mtms9" (UID: "c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47") : secret "cluster-monitoring-operator-tls" not found
Nov 01 05:02:52.361583 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.361224    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-certificates\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.361583 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.361475    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1967227b-a4ea-4ee6-945d-efb0b5040a13-snapshots\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:02:52.361995 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.361801    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/62652d56-1be6-47cc-a774-77d4fa9d206a-trusted-ca\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.361995 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.361971    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/636cccf5-2397-4dca-899a-3598a7b8096b-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz\" (UID: \"636cccf5-2397-4dca-899a-3598a7b8096b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz"
Nov 01 05:02:52.362721 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.362388    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 01 05:02:52.362721 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.362580    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs podName:82335b9e-5584-4afc-b1fc-cc003dbcad4b nodeName:}" failed. No retries permitted until 2026-11-01 05:02:52.862563764 +0000 UTC m=+32.864636904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs") pod "router-default-74dd76c985-n79rt" (UID: "82335b9e-5584-4afc-b1fc-cc003dbcad4b") : secret "router-metrics-certs-default" not found
Nov 01 05:02:52.362721 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.362643    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:02:52.362909 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.362733    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/636cccf5-2397-4dca-899a-3598a7b8096b-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz\" (UID: \"636cccf5-2397-4dca-899a-3598a7b8096b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz"
Nov 01 05:02:52.363451 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.363428    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1967227b-a4ea-4ee6-945d-efb0b5040a13-serving-cert\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:02:52.363728 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.363704    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/62652d56-1be6-47cc-a774-77d4fa9d206a-installation-pull-secrets\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.364053 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.364034    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-default-certificate\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:52.364921 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.364906    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-stats-auth\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:52.365056 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.365040    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/62652d56-1be6-47cc-a774-77d4fa9d206a-image-registry-private-configuration\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.397662 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.397632    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:02:52.406469 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.406420    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 01 05:02:52.406610 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.406512    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-27c4k\""
Nov 01 05:02:52.406610 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.406540    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 01 05:02:52.406610 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.406542    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-7m49c\""
Nov 01 05:02:52.411718 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.411698    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27"
Nov 01 05:02:52.420970 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.420942    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6qrt\" (UniqueName: \"kubernetes.io/projected/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-kube-api-access-h6qrt\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:02:52.421387 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.421360    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pj75\" (UniqueName: \"kubernetes.io/projected/636cccf5-2397-4dca-899a-3598a7b8096b-kube-api-access-5pj75\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz\" (UID: \"636cccf5-2397-4dca-899a-3598a7b8096b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz"
Nov 01 05:02:52.421778 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.421756    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8l8g\" (UniqueName: \"kubernetes.io/projected/82335b9e-5584-4afc-b1fc-cc003dbcad4b-kube-api-access-d8l8g\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:52.422203 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.422179    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfldn\" (UniqueName: \"kubernetes.io/projected/1967227b-a4ea-4ee6-945d-efb0b5040a13-kube-api-access-xfldn\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:02:52.422642 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.422620    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-bound-sa-token\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.422959 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.422941    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlhpw\" (UniqueName: \"kubernetes.io/projected/7fb489a8-8a4f-45f2-977b-594b6126281b-kube-api-access-hlhpw\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:02:52.423387 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.423366    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n4xr\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-kube-api-access-9n4xr\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.464589 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.463005    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz"
Nov 01 05:02:52.464589 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.463656    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wblsz\" (UniqueName: \"kubernetes.io/projected/5dc708b0-6c35-4f79-a415-e7533381b95b-kube-api-access-wblsz\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:02:52.464589 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.463681    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj67m\" (UniqueName: \"kubernetes.io/projected/b53e86a9-ca94-4753-b57b-47eb94bbd879-kube-api-access-bj67m\") pod \"ingress-canary-n8nnj\" (UID: \"b53e86a9-ca94-4753-b57b-47eb94bbd879\") " pod="openshift-ingress-canary/ingress-canary-n8nnj"
Nov 01 05:02:52.464589 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.463698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-nginx-conf\") pod \"networking-console-plugin-86c478bb88-ht4j4\" (UID: \"f6185c12-8b19-47d8-b0b3-01aa0f1192a7\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"
Nov 01 05:02:52.464589 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.463717    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5dc708b0-6c35-4f79-a415-e7533381b95b-config-volume\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:02:52.467959 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.465214    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6b98\" (UniqueName: \"kubernetes.io/projected/a01a018f-2c6e-413f-b88f-37b4e91699b1-kube-api-access-v6b98\") pod \"volume-data-source-validator-58c9f5f46d-j67hv\" (UID: \"a01a018f-2c6e-413f-b88f-37b4e91699b1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j67hv"
Nov 01 05:02:52.467959 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.465289    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9lkh\" (UniqueName: \"kubernetes.io/projected/57fa3738-c3e8-4c18-9582-f6e8377a041e-kube-api-access-p9lkh\") pod \"network-check-source-6f78d59f99-srv7d\" (UID: \"57fa3738-c3e8-4c18-9582-f6e8377a041e\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-srv7d"
Nov 01 05:02:52.467959 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.465323    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-ht4j4\" (UID: \"f6185c12-8b19-47d8-b0b3-01aa0f1192a7\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"
Nov 01 05:02:52.467959 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.465351    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b53e86a9-ca94-4753-b57b-47eb94bbd879-cert\") pod \"ingress-canary-n8nnj\" (UID: \"b53e86a9-ca94-4753-b57b-47eb94bbd879\") " pod="openshift-ingress-canary/ingress-canary-n8nnj"
Nov 01 05:02:52.467959 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.465469    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dc708b0-6c35-4f79-a415-e7533381b95b-tmp-dir\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:02:52.467959 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.465546    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:02:52.467959 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.465680    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 01 05:02:52.467959 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.465735    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls podName:5dc708b0-6c35-4f79-a415-e7533381b95b nodeName:}" failed. No retries permitted until 2026-11-01 05:02:52.965717593 +0000 UTC m=+32.967790741 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls") pod "dns-default-wcrwt" (UID: "5dc708b0-6c35-4f79-a415-e7533381b95b") : secret "dns-default-metrics-tls" not found
Nov 01 05:02:52.467959 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.466359    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5dc708b0-6c35-4f79-a415-e7533381b95b-config-volume\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:02:52.467959 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.466889    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 01 05:02:52.467959 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.466941    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert podName:f6185c12-8b19-47d8-b0b3-01aa0f1192a7 nodeName:}" failed. No retries permitted until 2026-11-01 05:02:52.966925165 +0000 UTC m=+32.968998296 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-ht4j4" (UID: "f6185c12-8b19-47d8-b0b3-01aa0f1192a7") : secret "networking-console-plugin-cert" not found
Nov 01 05:02:52.467959 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.467525    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dc708b0-6c35-4f79-a415-e7533381b95b-tmp-dir\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:02:52.470411 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.470347    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-nginx-conf\") pod \"networking-console-plugin-86c478bb88-ht4j4\" (UID: \"f6185c12-8b19-47d8-b0b3-01aa0f1192a7\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"
Nov 01 05:02:52.480126 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.480073    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b53e86a9-ca94-4753-b57b-47eb94bbd879-cert\") pod \"ingress-canary-n8nnj\" (UID: \"b53e86a9-ca94-4753-b57b-47eb94bbd879\") " pod="openshift-ingress-canary/ingress-canary-n8nnj"
Nov 01 05:02:52.499709 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.499679    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:02:52.525841 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.524818    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wblsz\" (UniqueName: \"kubernetes.io/projected/5dc708b0-6c35-4f79-a415-e7533381b95b-kube-api-access-wblsz\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:02:52.526405 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.526366    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj67m\" (UniqueName: \"kubernetes.io/projected/b53e86a9-ca94-4753-b57b-47eb94bbd879-kube-api-access-bj67m\") pod \"ingress-canary-n8nnj\" (UID: \"b53e86a9-ca94-4753-b57b-47eb94bbd879\") " pod="openshift-ingress-canary/ingress-canary-n8nnj"
Nov 01 05:02:52.531555 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.531511    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9lkh\" (UniqueName: \"kubernetes.io/projected/57fa3738-c3e8-4c18-9582-f6e8377a041e-kube-api-access-p9lkh\") pod \"network-check-source-6f78d59f99-srv7d\" (UID: \"57fa3738-c3e8-4c18-9582-f6e8377a041e\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-srv7d"
Nov 01 05:02:52.531696 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.531671    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6b98\" (UniqueName: \"kubernetes.io/projected/a01a018f-2c6e-413f-b88f-37b4e91699b1-kube-api-access-v6b98\") pod \"volume-data-source-validator-58c9f5f46d-j67hv\" (UID: \"a01a018f-2c6e-413f-b88f-37b4e91699b1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j67hv"
Nov 01 05:02:52.537067 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.536847    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j67hv"
Nov 01 05:02:52.588008 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.587974    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-srv7d"
Nov 01 05:02:52.616042 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.615764    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-d2cm4"
Nov 01 05:02:52.635619 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.635591    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n8nnj"
Nov 01 05:02:52.768921 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.768892    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-qb994"]
Nov 01 05:02:52.769076 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.769056    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:52.769244 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.769227    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 05:02:52.769322 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.769247    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6969bc8dd6-85rsc: secret "image-registry-tls" not found
Nov 01 05:02:52.769322 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.769303    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls podName:48c05e4f-a673-4f5f-8c21-8dc1ff716683 nodeName:}" failed. No retries permitted until 2026-11-01 05:02:53.76928898 +0000 UTC m=+33.771362111 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls") pod "image-registry-6969bc8dd6-85rsc" (UID: "48c05e4f-a673-4f5f-8c21-8dc1ff716683") : secret "image-registry-tls" not found
Nov 01 05:02:52.772656 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:52.772630    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf7d0108_685b_4c27_8d0f_bccf4ca5dbbe.slice/crio-996b756136bcfaab4303a0eeb394dd84c47b6b8ad7c5830fff57e8cfb084ec6a WatchSource:0}: Error finding container 996b756136bcfaab4303a0eeb394dd84c47b6b8ad7c5830fff57e8cfb084ec6a: Status 404 returned error can't find the container with id 996b756136bcfaab4303a0eeb394dd84c47b6b8ad7c5830fff57e8cfb084ec6a
Nov 01 05:02:52.775352 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.775333    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-qszhw"]
Nov 01 05:02:52.778184 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:52.778163    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb91ced44_8be6_47c8_9229_549400076eb7.slice/crio-f3fd3df15230b3254b41078ef4220d57100eb071bc9a1b029a7e3da10c795e43 WatchSource:0}: Error finding container f3fd3df15230b3254b41078ef4220d57100eb071bc9a1b029a7e3da10c795e43: Status 404 returned error can't find the container with id f3fd3df15230b3254b41078ef4220d57100eb071bc9a1b029a7e3da10c795e43
Nov 01 05:02:52.805819 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.805710    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-6twtw"]
Nov 01 05:02:52.814294 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:52.814254    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod878896d2_fa82_46ef_af5e_0bf491216b03.slice/crio-de33dda5ec106d52d85fa7e3acb23fe0845dd765a8b3e7a9e2e274f6da780bb5 WatchSource:0}: Error finding container de33dda5ec106d52d85fa7e3acb23fe0845dd765a8b3e7a9e2e274f6da780bb5: Status 404 returned error can't find the container with id de33dda5ec106d52d85fa7e3acb23fe0845dd765a8b3e7a9e2e274f6da780bb5
Nov 01 05:02:52.837236 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.836814    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27"]
Nov 01 05:02:52.840989 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:52.840944    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbce36747_2159_4b72_8c0e_d98226588d18.slice/crio-914e556f074a17e41fd70606c911cc4933035569ca01d47ea3cdf87f5b697df1 WatchSource:0}: Error finding container 914e556f074a17e41fd70606c911cc4933035569ca01d47ea3cdf87f5b697df1: Status 404 returned error can't find the container with id 914e556f074a17e41fd70606c911cc4933035569ca01d47ea3cdf87f5b697df1
Nov 01 05:02:52.863852 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.863821    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz"]
Nov 01 05:02:52.865828 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:52.865802    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod636cccf5_2397_4dca_899a_3598a7b8096b.slice/crio-89381bfb4116e1748e052dd89cce07fe6c629b844d963fb75c618944a05a1707 WatchSource:0}: Error finding container 89381bfb4116e1748e052dd89cce07fe6c629b844d963fb75c618944a05a1707: Status 404 returned error can't find the container with id 89381bfb4116e1748e052dd89cce07fe6c629b844d963fb75c618944a05a1707
Nov 01 05:02:52.869967 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.869750    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:52.870060 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.869886    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle podName:82335b9e-5584-4afc-b1fc-cc003dbcad4b nodeName:}" failed. No retries permitted until 2026-11-01 05:02:53.869866546 +0000 UTC m=+33.871939694 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle") pod "router-default-74dd76c985-n79rt" (UID: "82335b9e-5584-4afc-b1fc-cc003dbcad4b") : configmap references non-existent config key: service-ca.crt
Nov 01 05:02:52.870171 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.870052    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:02:52.870237 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.870204    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 01 05:02:52.870314 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.870270    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls podName:c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47 nodeName:}" failed. No retries permitted until 2026-11-01 05:02:53.870249643 +0000 UTC m=+33.872322773 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-mtms9" (UID: "c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47") : secret "cluster-monitoring-operator-tls" not found
Nov 01 05:02:52.870388 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.870152    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:52.870388 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.870363    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:52.870473 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.870393    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:02:52.870511 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.870479    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 05:02:52.870511 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.870495    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79659c8779-dnfbf: secret "image-registry-tls" not found
Nov 01 05:02:52.870588 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.870560    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls podName:62652d56-1be6-47cc-a774-77d4fa9d206a nodeName:}" failed. No retries permitted until 2026-11-01 05:02:53.870524754 +0000 UTC m=+33.872597904 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls") pod "image-registry-79659c8779-dnfbf" (UID: "62652d56-1be6-47cc-a774-77d4fa9d206a") : secret "image-registry-tls" not found
Nov 01 05:02:52.870588 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.870571    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 01 05:02:52.870661 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.870621    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls podName:7fb489a8-8a4f-45f2-977b-594b6126281b nodeName:}" failed. No retries permitted until 2026-11-01 05:02:53.870597535 +0000 UTC m=+33.872670668 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-d8pkk" (UID: "7fb489a8-8a4f-45f2-977b-594b6126281b") : secret "samples-operator-tls" not found
Nov 01 05:02:52.870661 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.870622    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 01 05:02:52.870661 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.870654    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs podName:82335b9e-5584-4afc-b1fc-cc003dbcad4b nodeName:}" failed. No retries permitted until 2026-11-01 05:02:53.870647198 +0000 UTC m=+33.872720347 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs") pod "router-default-74dd76c985-n79rt" (UID: "82335b9e-5584-4afc-b1fc-cc003dbcad4b") : secret "router-metrics-certs-default" not found
Nov 01 05:02:52.888461 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.888437    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-cvn58"]
Nov 01 05:02:52.892381 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:52.892357    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1967227b_a4ea_4ee6_945d_efb0b5040a13.slice/crio-bca57b561e663cb1f3a26e52878ee097de3b83d75b9c9b60932ccbfe1bb2b4e1 WatchSource:0}: Error finding container bca57b561e663cb1f3a26e52878ee097de3b83d75b9c9b60932ccbfe1bb2b4e1: Status 404 returned error can't find the container with id bca57b561e663cb1f3a26e52878ee097de3b83d75b9c9b60932ccbfe1bb2b4e1
Nov 01 05:02:52.921577 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.921551    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j67hv"]
Nov 01 05:02:52.924485 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:52.924448    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda01a018f_2c6e_413f_b88f_37b4e91699b1.slice/crio-7bfde12401734403cb360b41d973d858fe8124d5920b6c4f2023738afd03bf5d WatchSource:0}: Error finding container 7bfde12401734403cb360b41d973d858fe8124d5920b6c4f2023738afd03bf5d: Status 404 returned error can't find the container with id 7bfde12401734403cb360b41d973d858fe8124d5920b6c4f2023738afd03bf5d
Nov 01 05:02:52.971232 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.971198    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:02:52.971418 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.971288    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-ht4j4\" (UID: \"f6185c12-8b19-47d8-b0b3-01aa0f1192a7\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"
Nov 01 05:02:52.971418 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.971373    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 01 05:02:52.971418 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.971407    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 01 05:02:52.971548 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.971450    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls podName:5dc708b0-6c35-4f79-a415-e7533381b95b nodeName:}" failed. No retries permitted until 2026-11-01 05:02:53.971427978 +0000 UTC m=+33.973501126 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls") pod "dns-default-wcrwt" (UID: "5dc708b0-6c35-4f79-a415-e7533381b95b") : secret "dns-default-metrics-tls" not found
Nov 01 05:02:52.971548 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:52.971471    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert podName:f6185c12-8b19-47d8-b0b3-01aa0f1192a7 nodeName:}" failed. No retries permitted until 2026-11-01 05:02:53.971461356 +0000 UTC m=+33.973534493 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-ht4j4" (UID: "f6185c12-8b19-47d8-b0b3-01aa0f1192a7") : secret "networking-console-plugin-cert" not found
Nov 01 05:02:52.976165 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:52.976140    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-srv7d"]
Nov 01 05:02:52.978956 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:52.978922    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57fa3738_c3e8_4c18_9582_f6e8377a041e.slice/crio-d017aa4053940b857f08931573d7b582976c3cdb191c0ab67f7b8c6858b5c33f WatchSource:0}: Error finding container d017aa4053940b857f08931573d7b582976c3cdb191c0ab67f7b8c6858b5c33f: Status 404 returned error can't find the container with id d017aa4053940b857f08931573d7b582976c3cdb191c0ab67f7b8c6858b5c33f
Nov 01 05:02:53.007683 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.007657    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-d2cm4"]
Nov 01 05:02:53.010318 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:53.010295    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb78a5ab_3a2d_45fa_abaa_7b26c33ec929.slice/crio-410d7603e73aa5da9a5b4a98d3484392406cb6c36f6d22b85cd73dfab13fe0c0 WatchSource:0}: Error finding container 410d7603e73aa5da9a5b4a98d3484392406cb6c36f6d22b85cd73dfab13fe0c0: Status 404 returned error can't find the container with id 410d7603e73aa5da9a5b4a98d3484392406cb6c36f6d22b85cd73dfab13fe0c0
Nov 01 05:02:53.028310 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.028284    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n8nnj"]
Nov 01 05:02:53.031214 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:53.031187    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb53e86a9_ca94_4753_b57b_47eb94bbd879.slice/crio-fc7aa10b2d58b36d8ae5ee178b2b253e0f9e24d5684ced6ee4ea7636a3c5cd36 WatchSource:0}: Error finding container fc7aa10b2d58b36d8ae5ee178b2b253e0f9e24d5684ced6ee4ea7636a3c5cd36: Status 404 returned error can't find the container with id fc7aa10b2d58b36d8ae5ee178b2b253e0f9e24d5684ced6ee4ea7636a3c5cd36
Nov 01 05:02:53.475636 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.475548    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:53.475636 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.475631    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:53.475902 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.475659    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:53.530545 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.530498    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 01 05:02:53.531101 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.530581    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 01 05:02:53.531101 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.530607    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-7kc5j\""
Nov 01 05:02:53.531101 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.530628    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-g7ktw\""
Nov 01 05:02:53.531101 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.530661    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6k7br\""
Nov 01 05:02:53.708054 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.708018    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz" event={"ID":"636cccf5-2397-4dca-899a-3598a7b8096b","Type":"ContainerStarted","Data":"89381bfb4116e1748e052dd89cce07fe6c629b844d963fb75c618944a05a1707"}
Nov 01 05:02:53.709242 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.709213    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27" event={"ID":"bce36747-2159-4b72-8c0e-d98226588d18","Type":"ContainerStarted","Data":"914e556f074a17e41fd70606c911cc4933035569ca01d47ea3cdf87f5b697df1"}
Nov 01 05:02:53.710680 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.710646    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-qszhw" event={"ID":"b91ced44-8be6-47c8-9229-549400076eb7","Type":"ContainerStarted","Data":"2fd761d73fd0408c7a0f8d9c3dbab3a64df8ed6793a4fe53a84cc29e8939e429"}
Nov 01 05:02:53.710806 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.710682    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-qszhw" event={"ID":"b91ced44-8be6-47c8-9229-549400076eb7","Type":"ContainerStarted","Data":"f3fd3df15230b3254b41078ef4220d57100eb071bc9a1b029a7e3da10c795e43"}
Nov 01 05:02:53.712108 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.712092    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-d2cm4" event={"ID":"fb78a5ab-3a2d-45fa-abaa-7b26c33ec929","Type":"ContainerStarted","Data":"c98e263a3671c13afcb44ab3dcd16ca71256c1792aa8fec6d9dfd6c68d3efd95"}
Nov 01 05:02:53.712193 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.712119    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-d2cm4" event={"ID":"fb78a5ab-3a2d-45fa-abaa-7b26c33ec929","Type":"ContainerStarted","Data":"410d7603e73aa5da9a5b4a98d3484392406cb6c36f6d22b85cd73dfab13fe0c0"}
Nov 01 05:02:53.713170 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.713155    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-srv7d" event={"ID":"57fa3738-c3e8-4c18-9582-f6e8377a041e","Type":"ContainerStarted","Data":"d017aa4053940b857f08931573d7b582976c3cdb191c0ab67f7b8c6858b5c33f"}
Nov 01 05:02:53.714185 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.714163    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-cvn58" event={"ID":"1967227b-a4ea-4ee6-945d-efb0b5040a13","Type":"ContainerStarted","Data":"bca57b561e663cb1f3a26e52878ee097de3b83d75b9c9b60932ccbfe1bb2b4e1"}
Nov 01 05:02:53.715427 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.715401    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n8nnj" event={"ID":"b53e86a9-ca94-4753-b57b-47eb94bbd879","Type":"ContainerStarted","Data":"fc7aa10b2d58b36d8ae5ee178b2b253e0f9e24d5684ced6ee4ea7636a3c5cd36"}
Nov 01 05:02:53.716489 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.716469    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j67hv" event={"ID":"a01a018f-2c6e-413f-b88f-37b4e91699b1","Type":"ContainerStarted","Data":"7bfde12401734403cb360b41d973d858fe8124d5920b6c4f2023738afd03bf5d"}
Nov 01 05:02:53.717531 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.717515    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw" event={"ID":"878896d2-fa82-46ef-af5e-0bf491216b03","Type":"ContainerStarted","Data":"de33dda5ec106d52d85fa7e3acb23fe0845dd765a8b3e7a9e2e274f6da780bb5"}
Nov 01 05:02:53.718778 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.718761    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-qb994" event={"ID":"df7d0108-685b-4c27-8d0f-bccf4ca5dbbe","Type":"ContainerStarted","Data":"112e19b11a4c559a426517222ee5e41d44a116d5e6a0d827085165489bc845f0"}
Nov 01 05:02:53.718862 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.718812    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-qb994" event={"ID":"df7d0108-685b-4c27-8d0f-bccf4ca5dbbe","Type":"ContainerStarted","Data":"996b756136bcfaab4303a0eeb394dd84c47b6b8ad7c5830fff57e8cfb084ec6a"}
Nov 01 05:02:53.779045 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.778957    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:53.779171 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:53.779133    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 05:02:53.779171 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:53.779151    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6969bc8dd6-85rsc: secret "image-registry-tls" not found
Nov 01 05:02:53.779233 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:53.779214    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls podName:48c05e4f-a673-4f5f-8c21-8dc1ff716683 nodeName:}" failed. No retries permitted until 2026-11-01 05:02:55.779200191 +0000 UTC m=+35.781273322 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls") pod "image-registry-6969bc8dd6-85rsc" (UID: "48c05e4f-a673-4f5f-8c21-8dc1ff716683") : secret "image-registry-tls" not found
Nov 01 05:02:53.879419 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.879390    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:53.879574 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.879435    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:02:53.879574 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.879495    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:53.879574 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:53.879563    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 01 05:02:53.879681 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:53.879581    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle podName:82335b9e-5584-4afc-b1fc-cc003dbcad4b nodeName:}" failed. No retries permitted until 2026-11-01 05:02:55.879562805 +0000 UTC m=+35.881635941 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle") pod "router-default-74dd76c985-n79rt" (UID: "82335b9e-5584-4afc-b1fc-cc003dbcad4b") : configmap references non-existent config key: service-ca.crt
Nov 01 05:02:53.879681 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:53.879584    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 05:02:53.879681 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:53.879600    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79659c8779-dnfbf: secret "image-registry-tls" not found
Nov 01 05:02:53.879681 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:53.879622    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls podName:c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47 nodeName:}" failed. No retries permitted until 2026-11-01 05:02:55.879611089 +0000 UTC m=+35.881684236 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-mtms9" (UID: "c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47") : secret "cluster-monitoring-operator-tls" not found
Nov 01 05:02:53.879681 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:53.879644    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls podName:62652d56-1be6-47cc-a774-77d4fa9d206a nodeName:}" failed. No retries permitted until 2026-11-01 05:02:55.879629772 +0000 UTC m=+35.881702923 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls") pod "image-registry-79659c8779-dnfbf" (UID: "62652d56-1be6-47cc-a774-77d4fa9d206a") : secret "image-registry-tls" not found
Nov 01 05:02:53.879681 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.879662    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:53.879904 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.879699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:02:53.879904 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:53.879738    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 01 05:02:53.879904 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:53.879767    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 01 05:02:53.879904 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:53.879771    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs podName:82335b9e-5584-4afc-b1fc-cc003dbcad4b nodeName:}" failed. No retries permitted until 2026-11-01 05:02:55.879762182 +0000 UTC m=+35.881835314 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs") pod "router-default-74dd76c985-n79rt" (UID: "82335b9e-5584-4afc-b1fc-cc003dbcad4b") : secret "router-metrics-certs-default" not found
Nov 01 05:02:53.880042 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:53.879913    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls podName:7fb489a8-8a4f-45f2-977b-594b6126281b nodeName:}" failed. No retries permitted until 2026-11-01 05:02:55.879900314 +0000 UTC m=+35.881973451 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-d8pkk" (UID: "7fb489a8-8a4f-45f2-977b-594b6126281b") : secret "samples-operator-tls" not found
Nov 01 05:02:53.981511 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.981461    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:02:53.981676 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:53.981597    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-ht4j4\" (UID: \"f6185c12-8b19-47d8-b0b3-01aa0f1192a7\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"
Nov 01 05:02:53.981738 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:53.981600    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 01 05:02:53.981892 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:53.981660    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 01 05:02:53.981892 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:53.981824    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls podName:5dc708b0-6c35-4f79-a415-e7533381b95b nodeName:}" failed. No retries permitted until 2026-11-01 05:02:55.981799193 +0000 UTC m=+35.983872340 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls") pod "dns-default-wcrwt" (UID: "5dc708b0-6c35-4f79-a415-e7533381b95b") : secret "dns-default-metrics-tls" not found
Nov 01 05:02:53.981892 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:53.981866    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert podName:f6185c12-8b19-47d8-b0b3-01aa0f1192a7 nodeName:}" failed. No retries permitted until 2026-11-01 05:02:55.981848116 +0000 UTC m=+35.983921251 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-ht4j4" (UID: "f6185c12-8b19-47d8-b0b3-01aa0f1192a7") : secret "networking-console-plugin-cert" not found
Nov 01 05:02:54.102723 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:54.102573    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-qb994" podStartSLOduration=397.102555797 podStartE2EDuration="6m37.102555797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 04:56:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:02:54.102269658 +0000 UTC m=+34.104342813" watchObservedRunningTime="2026-11-01 05:02:54.102555797 +0000 UTC m=+34.104628951"
Nov 01 05:02:54.217964 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:54.217849    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-d2cm4" podStartSLOduration=397.217833406 podStartE2EDuration="6m37.217833406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 04:56:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:02:54.216460447 +0000 UTC m=+34.218533599" watchObservedRunningTime="2026-11-01 05:02:54.217833406 +0000 UTC m=+34.219906558"
Nov 01 05:02:54.285827 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:54.285481    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs\") pod \"network-metrics-daemon-rvg8t\" (UID: \"ef46e568-5c98-4520-99e5-c49c4c3aca2a\") " pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:02:54.285827 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:54.285646    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 01 05:02:54.285827 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:54.285706    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs podName:ef46e568-5c98-4520-99e5-c49c4c3aca2a nodeName:}" failed. No retries permitted until 2026-11-01 05:03:26.285688051 +0000 UTC m=+66.287761199 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs") pod "network-metrics-daemon-rvg8t" (UID: "ef46e568-5c98-4520-99e5-c49c4c3aca2a") : secret "metrics-daemon-secret" not found
Nov 01 05:02:54.335889 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:54.334022    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-qszhw" podStartSLOduration=397.334003522 podStartE2EDuration="6m37.334003522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 04:56:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:02:54.333840808 +0000 UTC m=+34.335913961" watchObservedRunningTime="2026-11-01 05:02:54.334003522 +0000 UTC m=+34.336076677"
Nov 01 05:02:54.388018 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:54.387140    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:54.394853 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:54.394775    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:54.433039 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:54.432837    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:02:54.488098 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:54.487866    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzgwz\" (UniqueName: \"kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz\") pod \"network-check-target-tl74d\" (UID: \"a5b30117-7cb8-43ec-92a8-4e326b48c22d\") " pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:54.513280 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:54.513246    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzgwz\" (UniqueName: \"kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz\") pod \"network-check-target-tl74d\" (UID: \"a5b30117-7cb8-43ec-92a8-4e326b48c22d\") " pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:54.718490 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:54.718416    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:02:54.885198 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:54.884865    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q92dh"]
Nov 01 05:02:55.176202 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:55.176115    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tl74d"]
Nov 01 05:02:55.183417 ip-10-0-134-253 kubenswrapper[2577]: W1101 05:02:55.183378    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5b30117_7cb8_43ec_92a8_4e326b48c22d.slice/crio-256bbfe43b8fe4de7449f137b92cb1b8759725d201896ca0c5d890f4850969c5 WatchSource:0}: Error finding container 256bbfe43b8fe4de7449f137b92cb1b8759725d201896ca0c5d890f4850969c5: Status 404 returned error can't find the container with id 256bbfe43b8fe4de7449f137b92cb1b8759725d201896ca0c5d890f4850969c5
Nov 01 05:02:55.733839 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:55.733714    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tl74d" event={"ID":"a5b30117-7cb8-43ec-92a8-4e326b48c22d","Type":"ContainerStarted","Data":"256bbfe43b8fe4de7449f137b92cb1b8759725d201896ca0c5d890f4850969c5"}
Nov 01 05:02:55.736434 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:55.736365    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q92dh" event={"ID":"36a4e628-5f12-4849-9d77-deaa3e93b190","Type":"ContainerStarted","Data":"cb074bfe629a379116a5c363d223c78cd86d750828f0543b494ad362282da496"}
Nov 01 05:02:55.802085 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:55.801982    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:55.802271 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:55.802254    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 05:02:55.802355 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:55.802275    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6969bc8dd6-85rsc: secret "image-registry-tls" not found
Nov 01 05:02:55.802410 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:55.802369    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls podName:48c05e4f-a673-4f5f-8c21-8dc1ff716683 nodeName:}" failed. No retries permitted until 2026-11-01 05:02:59.802312674 +0000 UTC m=+39.804385819 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls") pod "image-registry-6969bc8dd6-85rsc" (UID: "48c05e4f-a673-4f5f-8c21-8dc1ff716683") : secret "image-registry-tls" not found
Nov 01 05:02:55.902953 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:55.902920    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:55.903126 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:55.902971    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:55.903126 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:55.902998    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:02:55.903126 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:55.903053    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:55.903126 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:55.903081    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:02:55.903325 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:55.903224    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 01 05:02:55.903325 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:55.903295    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls podName:c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47 nodeName:}" failed. No retries permitted until 2026-11-01 05:02:59.903274521 +0000 UTC m=+39.905347671 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-mtms9" (UID: "c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47") : secret "cluster-monitoring-operator-tls" not found
Nov 01 05:02:55.903764 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:55.903740    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 05:02:55.903764 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:55.903767    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79659c8779-dnfbf: secret "image-registry-tls" not found
Nov 01 05:02:55.904042 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:55.903977    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle podName:82335b9e-5584-4afc-b1fc-cc003dbcad4b nodeName:}" failed. No retries permitted until 2026-11-01 05:02:59.903952605 +0000 UTC m=+39.906025766 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle") pod "router-default-74dd76c985-n79rt" (UID: "82335b9e-5584-4afc-b1fc-cc003dbcad4b") : configmap references non-existent config key: service-ca.crt
Nov 01 05:02:55.904042 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:55.903995    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 01 05:02:55.904242 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:55.904062    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls podName:7fb489a8-8a4f-45f2-977b-594b6126281b nodeName:}" failed. No retries permitted until 2026-11-01 05:02:59.904044583 +0000 UTC m=+39.906117719 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-d8pkk" (UID: "7fb489a8-8a4f-45f2-977b-594b6126281b") : secret "samples-operator-tls" not found
Nov 01 05:02:55.904242 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:55.903996    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 01 05:02:55.904242 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:55.904107    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs podName:82335b9e-5584-4afc-b1fc-cc003dbcad4b nodeName:}" failed. No retries permitted until 2026-11-01 05:02:59.904096098 +0000 UTC m=+39.906169252 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs") pod "router-default-74dd76c985-n79rt" (UID: "82335b9e-5584-4afc-b1fc-cc003dbcad4b") : secret "router-metrics-certs-default" not found
Nov 01 05:02:55.904533 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:55.904503    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls podName:62652d56-1be6-47cc-a774-77d4fa9d206a nodeName:}" failed. No retries permitted until 2026-11-01 05:02:59.904483366 +0000 UTC m=+39.906556525 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls") pod "image-registry-79659c8779-dnfbf" (UID: "62652d56-1be6-47cc-a774-77d4fa9d206a") : secret "image-registry-tls" not found
Nov 01 05:02:56.003847 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:56.003722    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-ht4j4\" (UID: \"f6185c12-8b19-47d8-b0b3-01aa0f1192a7\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"
Nov 01 05:02:56.004074 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:56.003938    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:02:56.004139 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:56.004117    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 01 05:02:56.004251 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:56.004189    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls podName:5dc708b0-6c35-4f79-a415-e7533381b95b nodeName:}" failed. No retries permitted until 2026-11-01 05:03:00.004169485 +0000 UTC m=+40.006242622 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls") pod "dns-default-wcrwt" (UID: "5dc708b0-6c35-4f79-a415-e7533381b95b") : secret "dns-default-metrics-tls" not found
Nov 01 05:02:56.004379 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:56.004261    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 01 05:02:56.004379 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:56.004300    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert podName:f6185c12-8b19-47d8-b0b3-01aa0f1192a7 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:00.004287613 +0000 UTC m=+40.006360748 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-ht4j4" (UID: "f6185c12-8b19-47d8-b0b3-01aa0f1192a7") : secret "networking-console-plugin-cert" not found
Nov 01 05:02:56.750332 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:56.750287    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j67hv" event={"ID":"a01a018f-2c6e-413f-b88f-37b4e91699b1","Type":"ContainerStarted","Data":"0268f4da45b6213ed2dd5fb0e379816b76469cea6fa108ce39324219f0d81988"}
Nov 01 05:02:56.866801 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:56.866720    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j67hv" podStartSLOduration=363.734879066 podStartE2EDuration="6m6.866698453s" totalImagesPullingTime="3.131819387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 04:56:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:02:56.865956149 +0000 UTC m=+36.868029303" watchObservedRunningTime="2026-11-01 05:02:56.866698453 +0000 UTC m=+36.868771606"
Nov 01 05:02:57.759101 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:57.759061    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw" event={"ID":"878896d2-fa82-46ef-af5e-0bf491216b03","Type":"ContainerStarted","Data":"378ed44ba7165c82d349106d2179889c1a492741dd0c5b22ee5e3badbb042815"}
Nov 01 05:02:57.759620 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:57.759578    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:02:57.759775 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:57.759687    2577 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-6twtw container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.17:8443/readyz\": dial tcp 10.132.0.17:8443: connect: connection refused" start-of-body=
Nov 01 05:02:57.759775 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:57.759728    2577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw" podUID="878896d2-fa82-46ef-af5e-0bf491216b03" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.17:8443/readyz\": dial tcp 10.132.0.17:8443: connect: connection refused"
Nov 01 05:02:57.876296 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:57.874612    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw" podStartSLOduration=368.087554784 podStartE2EDuration="6m12.874592478s" totalImagesPullingTime="4.787037694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 04:56:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:02:57.874123978 +0000 UTC m=+37.876197145" watchObservedRunningTime="2026-11-01 05:02:57.874592478 +0000 UTC m=+37.876665631"
Nov 01 05:02:58.773107 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:58.764634    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/0.log"
Nov 01 05:02:58.773107 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:58.764673    2577 generic.go:356] "Generic (PLEG): container finished" podID="878896d2-fa82-46ef-af5e-0bf491216b03" containerID="378ed44ba7165c82d349106d2179889c1a492741dd0c5b22ee5e3badbb042815" exitCode=255
Nov 01 05:02:58.773107 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:58.764757    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw" event={"ID":"878896d2-fa82-46ef-af5e-0bf491216b03","Type":"ContainerDied","Data":"378ed44ba7165c82d349106d2179889c1a492741dd0c5b22ee5e3badbb042815"}
Nov 01 05:02:58.773107 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:58.765450    2577 scope.go:122] "RemoveContainer" containerID="378ed44ba7165c82d349106d2179889c1a492741dd0c5b22ee5e3badbb042815"
Nov 01 05:02:58.773107 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:58.769696    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz" event={"ID":"636cccf5-2397-4dca-899a-3598a7b8096b","Type":"ContainerStarted","Data":"a1dfbb52ff2fe274c4d3b3679a0eb12b7fb4994e68479f0aacc58ccff75a48cd"}
Nov 01 05:02:58.775316 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:58.774861    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-cvn58" event={"ID":"1967227b-a4ea-4ee6-945d-efb0b5040a13","Type":"ContainerStarted","Data":"89b65be52ff10c382485b53bbed1d83d875379e0e2807574783c565d4fea6754"}
Nov 01 05:02:59.000378 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:59.000323    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-cvn58" podStartSLOduration=364.40116901 podStartE2EDuration="6m10.000307988s" totalImagesPullingTime="5.599138978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 04:56:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:02:58.999026838 +0000 UTC m=+39.001099991" watchObservedRunningTime="2026-11-01 05:02:59.000307988 +0000 UTC m=+39.002381141"
Nov 01 05:02:59.114250 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:59.114200    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz" podStartSLOduration=366.472492243 podStartE2EDuration="6m12.114186417s" totalImagesPullingTime="5.641694174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 04:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:02:59.113882306 +0000 UTC m=+39.115955459" watchObservedRunningTime="2026-11-01 05:02:59.114186417 +0000 UTC m=+39.116259570"
Nov 01 05:02:59.783920 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:59.783822    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/1.log"
Nov 01 05:02:59.785107 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:59.784495    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/0.log"
Nov 01 05:02:59.785107 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:59.784530    2577 generic.go:356] "Generic (PLEG): container finished" podID="878896d2-fa82-46ef-af5e-0bf491216b03" containerID="2bfdcabf49fcf5bed5a8daab44aa1f9b67a4866acfc31c80777617c2ac716e5f" exitCode=255
Nov 01 05:02:59.785107 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:59.784629    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw" event={"ID":"878896d2-fa82-46ef-af5e-0bf491216b03","Type":"ContainerDied","Data":"2bfdcabf49fcf5bed5a8daab44aa1f9b67a4866acfc31c80777617c2ac716e5f"}
Nov 01 05:02:59.785107 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:59.784711    2577 scope.go:122] "RemoveContainer" containerID="378ed44ba7165c82d349106d2179889c1a492741dd0c5b22ee5e3badbb042815"
Nov 01 05:02:59.785917 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:59.785894    2577 scope.go:122] "RemoveContainer" containerID="2bfdcabf49fcf5bed5a8daab44aa1f9b67a4866acfc31c80777617c2ac716e5f"
Nov 01 05:02:59.786253 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:59.786130    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-6twtw_openshift-console-operator(878896d2-fa82-46ef-af5e-0bf491216b03)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw" podUID="878896d2-fa82-46ef-af5e-0bf491216b03"
Nov 01 05:02:59.788647 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:59.788255    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27" event={"ID":"bce36747-2159-4b72-8c0e-d98226588d18","Type":"ContainerStarted","Data":"78316fc3e2e1580ee4441b2fe703f1ae50bcd7fc40646f1152ca1c8644fc7e07"}
Nov 01 05:02:59.861778 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:59.861742    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:02:59.863428 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:59.863259    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 05:02:59.863428 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:59.863282    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6969bc8dd6-85rsc: secret "image-registry-tls" not found
Nov 01 05:02:59.863428 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:59.863348    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls podName:48c05e4f-a673-4f5f-8c21-8dc1ff716683 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:07.863327456 +0000 UTC m=+47.865400614 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls") pod "image-registry-6969bc8dd6-85rsc" (UID: "48c05e4f-a673-4f5f-8c21-8dc1ff716683") : secret "image-registry-tls" not found
Nov 01 05:02:59.963078 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:59.962997    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:02:59.963231 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:59.963102    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:59.963231 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:59.963140    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:02:59.963231 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:59.963172    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 01 05:02:59.963231 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:59.963195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:02:59.963231 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:02:59.963222    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:02:59.963410 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:59.963240    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls podName:7fb489a8-8a4f-45f2-977b-594b6126281b nodeName:}" failed. No retries permitted until 2026-11-01 05:03:07.96322063 +0000 UTC m=+47.965293791 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-d8pkk" (UID: "7fb489a8-8a4f-45f2-977b-594b6126281b") : secret "samples-operator-tls" not found
Nov 01 05:02:59.963410 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:59.963268    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle podName:82335b9e-5584-4afc-b1fc-cc003dbcad4b nodeName:}" failed. No retries permitted until 2026-11-01 05:03:07.963249156 +0000 UTC m=+47.965322323 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle") pod "router-default-74dd76c985-n79rt" (UID: "82335b9e-5584-4afc-b1fc-cc003dbcad4b") : configmap references non-existent config key: service-ca.crt
Nov 01 05:02:59.963410 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:59.963300    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 01 05:02:59.963410 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:59.963307    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 01 05:02:59.963410 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:59.963311    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 05:02:59.963410 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:59.963327    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79659c8779-dnfbf: secret "image-registry-tls" not found
Nov 01 05:02:59.963410 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:59.963336    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs podName:82335b9e-5584-4afc-b1fc-cc003dbcad4b nodeName:}" failed. No retries permitted until 2026-11-01 05:03:07.963326624 +0000 UTC m=+47.965399780 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs") pod "router-default-74dd76c985-n79rt" (UID: "82335b9e-5584-4afc-b1fc-cc003dbcad4b") : secret "router-metrics-certs-default" not found
Nov 01 05:02:59.963410 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:59.963353    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls podName:c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:07.963341045 +0000 UTC m=+47.965414205 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-mtms9" (UID: "c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47") : secret "cluster-monitoring-operator-tls" not found
Nov 01 05:02:59.963410 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:02:59.963372    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls podName:62652d56-1be6-47cc-a774-77d4fa9d206a nodeName:}" failed. No retries permitted until 2026-11-01 05:03:07.963362136 +0000 UTC m=+47.965435270 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls") pod "image-registry-79659c8779-dnfbf" (UID: "62652d56-1be6-47cc-a774-77d4fa9d206a") : secret "image-registry-tls" not found
Nov 01 05:03:00.018501 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.018434    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27" podStartSLOduration=368.712015627 podStartE2EDuration="6m15.018412223s" totalImagesPullingTime="6.306396596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 04:56:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:03:00.016872214 +0000 UTC m=+40.018945369" watchObservedRunningTime="2026-11-01 05:03:00.018412223 +0000 UTC m=+40.020485377"
Nov 01 05:03:00.064102 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.064006    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:03:00.064280 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.064123    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-ht4j4\" (UID: \"f6185c12-8b19-47d8-b0b3-01aa0f1192a7\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"
Nov 01 05:03:00.064338 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:03:00.064304    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 01 05:03:00.064382 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:03:00.064375    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert podName:f6185c12-8b19-47d8-b0b3-01aa0f1192a7 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:08.06435662 +0000 UTC m=+48.066429754 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-ht4j4" (UID: "f6185c12-8b19-47d8-b0b3-01aa0f1192a7") : secret "networking-console-plugin-cert" not found
Nov 01 05:03:00.064442 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:03:00.064434    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 01 05:03:00.064495 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:03:00.064466    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls podName:5dc708b0-6c35-4f79-a415-e7533381b95b nodeName:}" failed. No retries permitted until 2026-11-01 05:03:08.064455566 +0000 UTC m=+48.066528702 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls") pod "dns-default-wcrwt" (UID: "5dc708b0-6c35-4f79-a415-e7533381b95b") : secret "dns-default-metrics-tls" not found
Nov 01 05:03:00.308478 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.307111    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-b6sl7"]
Nov 01 05:03:00.320920 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.320889    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:00.366619 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.366585    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7c9eec03-4159-4a5e-a284-174d634ae0c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:00.366842 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.366666    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:00.366842 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.366807    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7c9eec03-4159-4a5e-a284-174d634ae0c9-crio-socket\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:00.366978 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.366932    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzrjv\" (UniqueName: \"kubernetes.io/projected/7c9eec03-4159-4a5e-a284-174d634ae0c9-kube-api-access-rzrjv\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:00.367030 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.366991    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c9eec03-4159-4a5e-a284-174d634ae0c9-data-volume\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:00.370154 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.370001    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-b6sl7"]
Nov 01 05:03:00.408623 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.408588    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 01 05:03:00.408821 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.408728    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xvz28\""
Nov 01 05:03:00.415960 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.415933    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 01 05:03:00.467699 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.467666    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7c9eec03-4159-4a5e-a284-174d634ae0c9-crio-socket\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:00.467900 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.467724    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzrjv\" (UniqueName: \"kubernetes.io/projected/7c9eec03-4159-4a5e-a284-174d634ae0c9-kube-api-access-rzrjv\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:00.467900 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.467775    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c9eec03-4159-4a5e-a284-174d634ae0c9-data-volume\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:00.467900 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.467853    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7c9eec03-4159-4a5e-a284-174d634ae0c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:00.467900 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.467881    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:00.468120 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.467983    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7c9eec03-4159-4a5e-a284-174d634ae0c9-crio-socket\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:00.468120 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:03:00.468052    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 01 05:03:00.468120 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:03:00.468106    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls podName:7c9eec03-4159-4a5e-a284-174d634ae0c9 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:00.968089944 +0000 UTC m=+40.970163088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b6sl7" (UID: "7c9eec03-4159-4a5e-a284-174d634ae0c9") : secret "insights-runtime-extractor-tls" not found
Nov 01 05:03:00.468422 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:03:00.468157    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 01 05:03:00.468422 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:03:00.468197    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7c9eec03-4159-4a5e-a284-174d634ae0c9-kube-rbac-proxy-cm podName:7c9eec03-4159-4a5e-a284-174d634ae0c9 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:00.968185003 +0000 UTC m=+40.970258135 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7c9eec03-4159-4a5e-a284-174d634ae0c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-b6sl7" (UID: "7c9eec03-4159-4a5e-a284-174d634ae0c9") : configmap "kube-rbac-proxy" not found
Nov 01 05:03:00.468422 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.468274    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c9eec03-4159-4a5e-a284-174d634ae0c9-data-volume\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:00.531739 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.531711    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzrjv\" (UniqueName: \"kubernetes.io/projected/7c9eec03-4159-4a5e-a284-174d634ae0c9-kube-api-access-rzrjv\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:00.793426 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.793386    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-srv7d" event={"ID":"57fa3738-c3e8-4c18-9582-f6e8377a041e","Type":"ContainerStarted","Data":"52f05ba13fb38910f99ae1598106c9be819ee9e405f508ec209e3574e8d98f21"}
Nov 01 05:03:00.795715 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.795681    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n8nnj" event={"ID":"b53e86a9-ca94-4753-b57b-47eb94bbd879","Type":"ContainerStarted","Data":"157163990c37bdeeb5f712c6da6f026fb4a16f9255927d7692beb664db87a891"}
Nov 01 05:03:00.797653 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.797627    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/1.log"
Nov 01 05:03:00.798167 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.798147    2577 scope.go:122] "RemoveContainer" containerID="2bfdcabf49fcf5bed5a8daab44aa1f9b67a4866acfc31c80777617c2ac716e5f"
Nov 01 05:03:00.798318 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:03:00.798297    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-6twtw_openshift-console-operator(878896d2-fa82-46ef-af5e-0bf491216b03)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw" podUID="878896d2-fa82-46ef-af5e-0bf491216b03"
Nov 01 05:03:00.911080 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.910961    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-srv7d" podStartSLOduration=360.366982397 podStartE2EDuration="6m7.91094171s" totalImagesPullingTime="7.543959313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 04:56:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:03:00.908922348 +0000 UTC m=+40.910995502" watchObservedRunningTime="2026-11-01 05:03:00.91094171 +0000 UTC m=+40.913014863"
Nov 01 05:03:00.974260 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.974221    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7c9eec03-4159-4a5e-a284-174d634ae0c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:00.974447 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:00.974272    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:00.974576 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:03:00.974563    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 01 05:03:00.974643 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:03:00.974629    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls podName:7c9eec03-4159-4a5e-a284-174d634ae0c9 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:01.974610199 +0000 UTC m=+41.976683344 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b6sl7" (UID: "7c9eec03-4159-4a5e-a284-174d634ae0c9") : secret "insights-runtime-extractor-tls" not found
Nov 01 05:03:00.975033 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:03:00.975016    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 01 05:03:00.975088 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:03:00.975065    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7c9eec03-4159-4a5e-a284-174d634ae0c9-kube-rbac-proxy-cm podName:7c9eec03-4159-4a5e-a284-174d634ae0c9 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:01.975051472 +0000 UTC m=+41.977124620 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7c9eec03-4159-4a5e-a284-174d634ae0c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-b6sl7" (UID: "7c9eec03-4159-4a5e-a284-174d634ae0c9") : configmap "kube-rbac-proxy" not found
Nov 01 05:03:01.027459 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:01.027394    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-n8nnj" podStartSLOduration=3.394671232 podStartE2EDuration="10.027373648s" totalImagesPullingTime="6.632702416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:02:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:03:01.025077356 +0000 UTC m=+41.027150510" watchObservedRunningTime="2026-11-01 05:03:01.027373648 +0000 UTC m=+41.029446803"
Nov 01 05:03:01.803424 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:01.803387    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tl74d" event={"ID":"a5b30117-7cb8-43ec-92a8-4e326b48c22d","Type":"ContainerStarted","Data":"6e944e60937aca5ecce28921a498b47a450e4bd229f76992e2f600f9caf00755"}
Nov 01 05:03:01.918541 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:01.918481    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tl74d" podStartSLOduration=33.707869315 podStartE2EDuration="39.918465883s" totalImagesPullingTime="6.210596568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:02:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:03:01.917746103 +0000 UTC m=+41.919819257" watchObservedRunningTime="2026-11-01 05:03:01.918465883 +0000 UTC m=+41.920539066"
Nov 01 05:03:01.985454 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:01.985417    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7c9eec03-4159-4a5e-a284-174d634ae0c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:01.985454 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:01.985459    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:01.985697 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:03:01.985556    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 01 05:03:01.985697 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:03:01.985593    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 01 05:03:01.985697 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:03:01.985625    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7c9eec03-4159-4a5e-a284-174d634ae0c9-kube-rbac-proxy-cm podName:7c9eec03-4159-4a5e-a284-174d634ae0c9 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:03.985607952 +0000 UTC m=+43.987681100 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7c9eec03-4159-4a5e-a284-174d634ae0c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-b6sl7" (UID: "7c9eec03-4159-4a5e-a284-174d634ae0c9") : configmap "kube-rbac-proxy" not found
Nov 01 05:03:01.985697 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:03:01.985639    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls podName:7c9eec03-4159-4a5e-a284-174d634ae0c9 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:03.985633577 +0000 UTC m=+43.987706709 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b6sl7" (UID: "7c9eec03-4159-4a5e-a284-174d634ae0c9") : secret "insights-runtime-extractor-tls" not found
Nov 01 05:03:02.398387 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:02.398349    2577 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:03:02.398768 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:02.398751    2577 scope.go:122] "RemoveContainer" containerID="2bfdcabf49fcf5bed5a8daab44aa1f9b67a4866acfc31c80777617c2ac716e5f"
Nov 01 05:03:02.398997 ip-10-0-134-253 kubenswrapper[2577]: E1101 05:03:02.398976    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-6twtw_openshift-console-operator(878896d2-fa82-46ef-af5e-0bf491216b03)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw" podUID="878896d2-fa82-46ef-af5e-0bf491216b03"
Nov 01 05:03:02.806301 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:02.806281    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:03:03.044602 ip-10-0-134-253 kubenswrapper[2577]: I1101 05:03:03.044486    2577 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 01 05:03:03.044653 ip-10-0-134-253 systemd[1]: Stopping Kubernetes Kubelet...
Nov 01 05:03:03.056224 ip-10-0-134-253 systemd[1]: kubelet.service: Deactivated successfully.
Nov 01 05:03:03.056395 ip-10-0-134-253 systemd[1]: Stopped Kubernetes Kubelet.
Nov 01 05:03:03.056571 ip-10-0-134-253 systemd[1]: kubelet.service: Consumed 3.061s CPU time.
Nov 01 05:03:03.071305 ip-10-0-134-253 systemd[1]: Starting Kubernetes Kubelet...
Nov 01 05:03:03.152478 ip-10-0-134-253 kubenswrapper[4864]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 01 05:03:03.152478 ip-10-0-134-253 kubenswrapper[4864]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 01 05:03:03.152478 ip-10-0-134-253 kubenswrapper[4864]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 01 05:03:03.152478 ip-10-0-134-253 kubenswrapper[4864]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 01 05:03:03.154720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154630    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 05:03:03.154720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154716    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 05:03:03.154720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154722    4864 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 05:03:03.154720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154726    4864 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 05:03:03.154999 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154731    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 05:03:03.154999 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154736    4864 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 05:03:03.154999 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154743    4864 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 05:03:03.154999 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154749    4864 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 05:03:03.154999 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154753    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 05:03:03.154999 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154757    4864 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 05:03:03.154999 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154761    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 05:03:03.154999 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154765    4864 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 05:03:03.154999 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154769    4864 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 05:03:03.154999 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154773    4864 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 05:03:03.154999 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154776    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 05:03:03.154999 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154801    4864 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 05:03:03.154999 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154805    4864 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 05:03:03.154999 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154809    4864 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 05:03:03.154999 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154813    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 05:03:03.154999 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154817    4864 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 05:03:03.154999 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154821    4864 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 05:03:03.154999 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154825    4864 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 05:03:03.154999 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154829    4864 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 05:03:03.155839 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154838    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 05:03:03.155839 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154842    4864 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 05:03:03.155839 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154846    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 05:03:03.155839 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154850    4864 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 05:03:03.155839 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154854    4864 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 05:03:03.155839 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154859    4864 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 05:03:03.155839 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154863    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 05:03:03.155839 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154867    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 05:03:03.155839 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154871    4864 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 05:03:03.155839 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154875    4864 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 05:03:03.155839 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154879    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 05:03:03.155839 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154884    4864 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 05:03:03.155839 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154888    4864 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 05:03:03.155839 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154893    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 05:03:03.155839 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154897    4864 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 05:03:03.155839 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154901    4864 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 05:03:03.155839 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154905    4864 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 05:03:03.155839 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154911    4864 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 05:03:03.155839 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154915    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 05:03:03.155839 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154919    4864 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 05:03:03.156701 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154923    4864 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 05:03:03.156701 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154928    4864 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 05:03:03.156701 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154932    4864 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 05:03:03.156701 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154936    4864 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 05:03:03.156701 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154940    4864 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 05:03:03.156701 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154944    4864 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 05:03:03.156701 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154948    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 05:03:03.156701 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154952    4864 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 05:03:03.156701 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154956    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 05:03:03.156701 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154961    4864 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 05:03:03.156701 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154965    4864 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 05:03:03.156701 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154970    4864 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 05:03:03.156701 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154974    4864 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 05:03:03.156701 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154978    4864 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 05:03:03.156701 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154982    4864 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 05:03:03.156701 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154987    4864 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 05:03:03.156701 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154992    4864 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 05:03:03.156701 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.154996    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 05:03:03.156701 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155000    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 05:03:03.156701 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155003    4864 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 05:03:03.157599 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155007    4864 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 05:03:03.157599 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155011    4864 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 05:03:03.157599 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155017    4864 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 05:03:03.157599 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155023    4864 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 05:03:03.157599 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155029    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 05:03:03.157599 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155035    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 05:03:03.157599 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155039    4864 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 05:03:03.157599 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155043    4864 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 05:03:03.157599 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155047    4864 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 05:03:03.157599 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155053    4864 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 05:03:03.157599 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155058    4864 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 05:03:03.157599 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155062    4864 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 05:03:03.157599 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155066    4864 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 05:03:03.157599 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155070    4864 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 05:03:03.157599 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155074    4864 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 05:03:03.157599 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155078    4864 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 05:03:03.157599 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155082    4864 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 05:03:03.157599 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155086    4864 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 05:03:03.157599 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155090    4864 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 05:03:03.158452 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155095    4864 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 05:03:03.158452 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155099    4864 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 05:03:03.158452 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155102    4864 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 05:03:03.158452 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155107    4864 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 05:03:03.158452 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155112    4864 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 05:03:03.158452 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155115    4864 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 05:03:03.158452 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155119    4864 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 05:03:03.158452 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155124    4864 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 05:03:03.158452 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155128    4864 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 05:03:03.158452 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155133    4864 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 05:03:03.158452 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155137    4864 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 05:03:03.158452 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155141    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 05:03:03.158452 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155145    4864 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 05:03:03.158452 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155149    4864 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 05:03:03.158452 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155153    4864 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 05:03:03.158452 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155157    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 05:03:03.158452 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155162    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 05:03:03.158452 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155166    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 05:03:03.158452 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155171    4864 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 05:03:03.158452 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155175    4864 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 05:03:03.159374 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155181    4864 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 05:03:03.159374 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155185    4864 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 05:03:03.159374 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155189    4864 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 05:03:03.159374 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.155195    4864 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 05:03:03.159374 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156426    4864 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 05:03:03.159374 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156436    4864 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 05:03:03.159374 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156441    4864 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 05:03:03.159374 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156446    4864 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 05:03:03.159374 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156450    4864 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 05:03:03.159374 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156454    4864 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 05:03:03.159374 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156458    4864 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 05:03:03.159374 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156462    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 05:03:03.159374 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156467    4864 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 05:03:03.159374 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156471    4864 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 05:03:03.159374 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156475    4864 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 05:03:03.159374 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156479    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 05:03:03.159374 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156484    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 05:03:03.159374 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156488    4864 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 05:03:03.159374 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156492    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 05:03:03.160303 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156496    4864 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 05:03:03.160303 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156501    4864 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 05:03:03.160303 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156505    4864 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 05:03:03.160303 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156510    4864 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 05:03:03.160303 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156515    4864 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 05:03:03.160303 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156519    4864 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 05:03:03.160303 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156523    4864 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 05:03:03.160303 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156528    4864 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 05:03:03.160303 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156532    4864 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 05:03:03.160303 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156535    4864 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 05:03:03.160303 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156540    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 05:03:03.160303 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156545    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 05:03:03.160303 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156549    4864 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 05:03:03.160303 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156554    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 05:03:03.160303 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156558    4864 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 05:03:03.160303 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156562    4864 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 05:03:03.160303 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156566    4864 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 05:03:03.160303 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156571    4864 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 05:03:03.160303 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156576    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 05:03:03.160303 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156580    4864 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 05:03:03.161226 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156584    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 05:03:03.161226 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156589    4864 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 05:03:03.161226 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156593    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 05:03:03.161226 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156597    4864 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 05:03:03.161226 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156602    4864 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 05:03:03.161226 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156606    4864 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 05:03:03.161226 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156610    4864 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 05:03:03.161226 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156615    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 05:03:03.161226 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156620    4864 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 05:03:03.161226 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156624    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 05:03:03.161226 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156628    4864 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 05:03:03.161226 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156632    4864 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 05:03:03.161226 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156637    4864 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 05:03:03.161226 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156641    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 05:03:03.161226 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156645    4864 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 05:03:03.161226 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156649    4864 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 05:03:03.161226 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156653    4864 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 05:03:03.161226 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156657    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 05:03:03.161226 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156661    4864 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 05:03:03.161226 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156665    4864 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 05:03:03.161722 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156669    4864 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 05:03:03.161722 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156673    4864 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 05:03:03.161722 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156678    4864 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 05:03:03.161722 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156682    4864 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 05:03:03.161722 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156686    4864 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 05:03:03.161722 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156691    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 05:03:03.161722 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156695    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 05:03:03.161722 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156699    4864 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 05:03:03.161722 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156703    4864 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 05:03:03.161722 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156707    4864 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 05:03:03.161722 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156717    4864 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 05:03:03.161722 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156722    4864 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 05:03:03.161722 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156727    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 05:03:03.161722 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156732    4864 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 05:03:03.161722 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156737    4864 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 05:03:03.161722 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156742    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 05:03:03.161722 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156746    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 05:03:03.161722 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156751    4864 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 05:03:03.161722 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156755    4864 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 05:03:03.162234 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156759    4864 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 05:03:03.162234 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156763    4864 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 05:03:03.162234 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156767    4864 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 05:03:03.162234 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156771    4864 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 05:03:03.162234 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156776    4864 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 05:03:03.162234 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156798    4864 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 05:03:03.162234 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156803    4864 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 05:03:03.162234 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156807    4864 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 05:03:03.162234 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156811    4864 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 05:03:03.162234 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156815    4864 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 05:03:03.162234 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156820    4864 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 05:03:03.162234 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156824    4864 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 05:03:03.162234 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156828    4864 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 05:03:03.162234 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156833    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 05:03:03.162234 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156837    4864 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 05:03:03.162234 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156841    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 05:03:03.162234 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156845    4864 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 05:03:03.162234 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156849    4864 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 05:03:03.162234 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156853    4864 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 05:03:03.162234 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156858    4864 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 05:03:03.162746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156863    4864 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 05:03:03.162746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156871    4864 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 05:03:03.162746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156875    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 05:03:03.162746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156879    4864 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 05:03:03.162746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156884    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 05:03:03.162746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156889    4864 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 05:03:03.162746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156895    4864 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 05:03:03.162746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156901    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 05:03:03.162746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156907    4864 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 05:03:03.162746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156912    4864 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 05:03:03.162746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156916    4864 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 05:03:03.162746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.156920    4864 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 05:03:03.162746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157727    4864 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 05:03:03.162746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157736    4864 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 05:03:03.162746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157741    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 05:03:03.162746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157745    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 05:03:03.162746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157749    4864 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 05:03:03.162746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157754    4864 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 05:03:03.162746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157758    4864 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 05:03:03.163283 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157762    4864 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 05:03:03.163283 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157767    4864 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 05:03:03.163283 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157771    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 05:03:03.163283 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157775    4864 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 05:03:03.163283 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157779    4864 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 05:03:03.163283 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157804    4864 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 05:03:03.163283 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157808    4864 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 05:03:03.163283 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157812    4864 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 05:03:03.163283 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157816    4864 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 05:03:03.163283 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157820    4864 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 05:03:03.163283 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157824    4864 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 05:03:03.163283 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157829    4864 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 05:03:03.163283 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157833    4864 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 05:03:03.163283 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157837    4864 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 05:03:03.163283 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157844    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 05:03:03.163283 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157849    4864 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 05:03:03.163283 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157853    4864 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 05:03:03.163283 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157861    4864 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 05:03:03.163283 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157868    4864 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 05:03:03.163746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157873    4864 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 05:03:03.163746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157878    4864 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 05:03:03.163746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157882    4864 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 05:03:03.163746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157887    4864 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 05:03:03.163746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157891    4864 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 05:03:03.163746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157895    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 05:03:03.163746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157899    4864 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 05:03:03.163746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157903    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 05:03:03.163746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157908    4864 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 05:03:03.163746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157912    4864 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 05:03:03.163746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157917    4864 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 05:03:03.163746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157921    4864 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 05:03:03.163746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157925    4864 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 05:03:03.163746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157929    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 05:03:03.163746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157933    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 05:03:03.163746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157937    4864 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 05:03:03.163746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157941    4864 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 05:03:03.163746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157945    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 05:03:03.163746 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157949    4864 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 05:03:03.164251 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157954    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 05:03:03.164251 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157958    4864 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 05:03:03.164251 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157962    4864 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 05:03:03.164251 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157967    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 05:03:03.164251 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157971    4864 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 05:03:03.164251 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157975    4864 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 05:03:03.164251 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157979    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 05:03:03.164251 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157984    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 05:03:03.164251 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157990    4864 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 05:03:03.164251 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157994    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 05:03:03.164251 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.157998    4864 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 05:03:03.164251 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158002    4864 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 05:03:03.164251 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158007    4864 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 05:03:03.164251 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158011    4864 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 05:03:03.164251 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158016    4864 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 05:03:03.164251 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158020    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 05:03:03.164251 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158024    4864 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 05:03:03.164251 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158028    4864 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 05:03:03.164251 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158032    4864 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 05:03:03.164720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158037    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 05:03:03.164720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158041    4864 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 05:03:03.164720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158045    4864 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 05:03:03.164720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158052    4864 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 05:03:03.164720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158057    4864 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 05:03:03.164720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158062    4864 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 05:03:03.164720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158070    4864 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 05:03:03.164720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158074    4864 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 05:03:03.164720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158078    4864 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 05:03:03.164720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158082    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 05:03:03.164720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158087    4864 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 05:03:03.164720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158092    4864 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 05:03:03.164720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158096    4864 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 05:03:03.164720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158100    4864 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 05:03:03.164720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158104    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 05:03:03.164720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158108    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 05:03:03.164720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158112    4864 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 05:03:03.164720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158116    4864 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 05:03:03.164720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158120    4864 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 05:03:03.164720 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158125    4864 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 05:03:03.165268 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158129    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 05:03:03.165268 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158136    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 05:03:03.165268 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158140    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 05:03:03.165268 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158144    4864 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 05:03:03.165268 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158148    4864 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 05:03:03.165268 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158152    4864 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 05:03:03.165268 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158157    4864 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 05:03:03.165268 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158161    4864 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 05:03:03.165268 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158166    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 05:03:03.165268 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158170    4864 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 05:03:03.165268 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158174    4864 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 05:03:03.165268 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158178    4864 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 05:03:03.165268 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158182    4864 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 05:03:03.165268 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158186    4864 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 05:03:03.165268 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158190    4864 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 05:03:03.165268 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158194    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 05:03:03.165268 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158198    4864 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 05:03:03.165268 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158202    4864 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 05:03:03.165268 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158208    4864 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158213    4864 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158217    4864 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.158222    4864 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158361    4864 flags.go:64] FLAG: --address="0.0.0.0"
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158372    4864 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158382    4864 flags.go:64] FLAG: --anonymous-auth="true"
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158389    4864 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158397    4864 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158402    4864 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158409    4864 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158415    4864 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158420    4864 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158425    4864 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158430    4864 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158438    4864 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158443    4864 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158447    4864 flags.go:64] FLAG: --cgroup-root=""
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158452    4864 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158456    4864 flags.go:64] FLAG: --client-ca-file=""
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158460    4864 flags.go:64] FLAG: --cloud-provider="external"
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158466    4864 flags.go:64] FLAG: --cluster-dns="[]"
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158472    4864 flags.go:64] FLAG: --cluster-domain=""
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158476    4864 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 01 05:03:03.166035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158481    4864 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158486    4864 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158492    4864 flags.go:64] FLAG: --container-log-max-files="5"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158498    4864 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158503    4864 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158508    4864 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158513    4864 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158517    4864 flags.go:64] FLAG: --contention-profiling="false"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158532    4864 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158538    4864 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158543    4864 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158547    4864 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158554    4864 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158560    4864 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158564    4864 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158570    4864 flags.go:64] FLAG: --enable-load-reader="false"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158575    4864 flags.go:64] FLAG: --enable-server="true"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158580    4864 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158588    4864 flags.go:64] FLAG: --event-burst="100"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158593    4864 flags.go:64] FLAG: --event-qps="50"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158597    4864 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158602    4864 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158606    4864 flags.go:64] FLAG: --eviction-hard=""
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158614    4864 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 01 05:03:03.166873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158619    4864 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158624    4864 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158629    4864 flags.go:64] FLAG: --eviction-soft=""
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158634    4864 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158639    4864 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158643    4864 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158657    4864 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158662    4864 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158667    4864 flags.go:64] FLAG: --fail-swap-on="true"
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158672    4864 flags.go:64] FLAG: --feature-gates=""
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158684    4864 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158689    4864 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158694    4864 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158699    4864 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158703    4864 flags.go:64] FLAG: --healthz-port="10248"
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158708    4864 flags.go:64] FLAG: --help="false"
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158715    4864 flags.go:64] FLAG: --hostname-override="ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158720    4864 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158725    4864 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158729    4864 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158735    4864 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158741    4864 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158745    4864 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 01 05:03:03.167507 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158750    4864 flags.go:64] FLAG: --image-service-endpoint=""
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158754    4864 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158759    4864 flags.go:64] FLAG: --kube-api-burst="100"
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158764    4864 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158769    4864 flags.go:64] FLAG: --kube-api-qps="50"
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158773    4864 flags.go:64] FLAG: --kube-reserved=""
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158778    4864 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158800    4864 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158807    4864 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158812    4864 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158816    4864 flags.go:64] FLAG: --lock-file=""
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158821    4864 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158826    4864 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158831    4864 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158840    4864 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158844    4864 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158850    4864 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158854    4864 flags.go:64] FLAG: --logging-format="text"
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158859    4864 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158865    4864 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158870    4864 flags.go:64] FLAG: --manifest-url=""
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158874    4864 flags.go:64] FLAG: --manifest-url-header=""
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158881    4864 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158886    4864 flags.go:64] FLAG: --max-open-files="1000000"
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158892    4864 flags.go:64] FLAG: --max-pods="110"
Nov 01 05:03:03.168140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158899    4864 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158903    4864 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158908    4864 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158914    4864 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158919    4864 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158923    4864 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158928    4864 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158939    4864 flags.go:64] FLAG: --node-status-max-images="50"
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158944    4864 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158949    4864 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158954    4864 flags.go:64] FLAG: --pod-cidr=""
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158959    4864 flags.go:64] FLAG: --pod-manifest-path=""
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158963    4864 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158968    4864 flags.go:64] FLAG: --pods-per-core="0"
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158973    4864 flags.go:64] FLAG: --port="10250"
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158979    4864 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158983    4864 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-061474926bbfd0d9a"
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158988    4864 flags.go:64] FLAG: --qos-reserved=""
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158993    4864 flags.go:64] FLAG: --read-only-port="10255"
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.158998    4864 flags.go:64] FLAG: --register-node="true"
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159003    4864 flags.go:64] FLAG: --register-with-taints=""
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159009    4864 flags.go:64] FLAG: --registry-burst="10"
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159013    4864 flags.go:64] FLAG: --registry-qps="5"
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159018    4864 flags.go:64] FLAG: --reserved-cpus=""
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159024    4864 flags.go:64] FLAG: --reserved-memory=""
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159030    4864 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 01 05:03:03.168743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159035    4864 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159039    4864 flags.go:64] FLAG: --rotate-certificates="false"
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159044    4864 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159049    4864 flags.go:64] FLAG: --runonce="false"
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159053    4864 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159058    4864 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159065    4864 flags.go:64] FLAG: --seccomp-default="false"
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159069    4864 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159074    4864 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159080    4864 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159085    4864 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159089    4864 flags.go:64] FLAG: --storage-driver-password="root"
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159094    4864 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159098    4864 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159103    4864 flags.go:64] FLAG: --storage-driver-user="root"
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159107    4864 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159112    4864 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159118    4864 flags.go:64] FLAG: --system-cgroups=""
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159123    4864 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159131    4864 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159136    4864 flags.go:64] FLAG: --tls-cert-file=""
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159142    4864 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159149    4864 flags.go:64] FLAG: --tls-min-version=""
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159153    4864 flags.go:64] FLAG: --tls-private-key-file=""
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159157    4864 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 01 05:03:03.169397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159162    4864 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 01 05:03:03.170084 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159167    4864 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 01 05:03:03.170084 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159171    4864 flags.go:64] FLAG: --v="2"
Nov 01 05:03:03.170084 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159178    4864 flags.go:64] FLAG: --version="false"
Nov 01 05:03:03.170084 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159184    4864 flags.go:64] FLAG: --vmodule=""
Nov 01 05:03:03.170084 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159191    4864 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 01 05:03:03.170084 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159196    4864 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 01 05:03:03.170084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159409    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 05:03:03.170084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159416    4864 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 05:03:03.170084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159421    4864 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 05:03:03.170084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159426    4864 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 05:03:03.170084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159431    4864 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 05:03:03.170084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159435    4864 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 05:03:03.170084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159442    4864 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 05:03:03.170084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159446    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 05:03:03.170084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159450    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 05:03:03.170084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159454    4864 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 05:03:03.170084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159459    4864 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 05:03:03.170084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159464    4864 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 05:03:03.170084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159468    4864 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 05:03:03.170084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159472    4864 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 05:03:03.170084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159476    4864 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 05:03:03.170610 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159480    4864 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 05:03:03.170610 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159485    4864 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 05:03:03.170610 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159489    4864 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 05:03:03.170610 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159493    4864 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 05:03:03.170610 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159497    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 05:03:03.170610 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159501    4864 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 05:03:03.170610 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159506    4864 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 05:03:03.170610 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159511    4864 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 05:03:03.170610 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159515    4864 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 05:03:03.170610 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159519    4864 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 05:03:03.170610 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159523    4864 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 05:03:03.170610 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159527    4864 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 05:03:03.170610 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159531    4864 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 05:03:03.170610 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159535    4864 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 05:03:03.170610 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159540    4864 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 05:03:03.170610 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159544    4864 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 05:03:03.170610 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159548    4864 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 05:03:03.170610 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159553    4864 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 05:03:03.170610 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159557    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 05:03:03.170610 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159563    4864 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 05:03:03.171146 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159570    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 05:03:03.171146 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159576    4864 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 05:03:03.171146 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159581    4864 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 05:03:03.171146 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159587    4864 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 05:03:03.171146 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159591    4864 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 05:03:03.171146 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159595    4864 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 05:03:03.171146 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159600    4864 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 05:03:03.171146 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159604    4864 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 05:03:03.171146 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159608    4864 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 05:03:03.171146 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159613    4864 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 05:03:03.171146 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159617    4864 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 05:03:03.171146 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159621    4864 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 05:03:03.171146 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159625    4864 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 05:03:03.171146 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159629    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 05:03:03.171146 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159633    4864 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 05:03:03.171146 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159638    4864 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 05:03:03.171146 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159642    4864 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 05:03:03.171146 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159646    4864 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 05:03:03.171146 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159650    4864 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 05:03:03.171146 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159654    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 05:03:03.171714 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159659    4864 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 05:03:03.171714 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159663    4864 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 05:03:03.171714 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159667    4864 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 05:03:03.171714 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159671    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 05:03:03.171714 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159675    4864 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 05:03:03.171714 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159680    4864 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 05:03:03.171714 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159684    4864 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 05:03:03.171714 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159688    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 05:03:03.171714 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159692    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 05:03:03.171714 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159696    4864 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 05:03:03.171714 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159701    4864 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 05:03:03.171714 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159705    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 05:03:03.171714 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159709    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 05:03:03.171714 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159714    4864 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 05:03:03.171714 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159718    4864 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 05:03:03.171714 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159724    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 05:03:03.171714 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159729    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 05:03:03.171714 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159733    4864 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 05:03:03.171714 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159737    4864 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 05:03:03.171714 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159741    4864 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 05:03:03.172280 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159745    4864 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 05:03:03.172280 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159749    4864 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 05:03:03.172280 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159754    4864 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 05:03:03.172280 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159758    4864 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 05:03:03.172280 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159762    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 05:03:03.172280 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159766    4864 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 05:03:03.172280 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159770    4864 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 05:03:03.172280 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159775    4864 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 05:03:03.172280 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159778    4864 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 05:03:03.172280 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159802    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 05:03:03.172280 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159806    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 05:03:03.172280 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159810    4864 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 05:03:03.172280 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159815    4864 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 05:03:03.172280 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159819    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 05:03:03.172280 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159823    4864 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 05:03:03.172280 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159826    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 05:03:03.172280 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159830    4864 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 05:03:03.172280 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159837    4864 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 05:03:03.172280 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159844    4864 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 05:03:03.172798 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159849    4864 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 05:03:03.172798 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159853    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 05:03:03.172798 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159857    4864 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 05:03:03.172798 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159861    4864 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 05:03:03.172798 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159866    4864 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 05:03:03.172798 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159871    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 05:03:03.172798 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159875    4864 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 05:03:03.172798 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159879    4864 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 05:03:03.172798 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159886    4864 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 05:03:03.172798 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159890    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 05:03:03.172798 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159895    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 05:03:03.172798 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159899    4864 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 05:03:03.172798 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.159941    4864 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 01 05:03:03.172798 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.159999    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 05:03:03.172798 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160007    4864 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 05:03:03.172798 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160012    4864 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 05:03:03.173208 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160016    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 05:03:03.173208 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160020    4864 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 05:03:03.173208 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160024    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 05:03:03.173208 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160029    4864 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 05:03:03.173208 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160035    4864 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 05:03:03.173208 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160042    4864 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 05:03:03.173208 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160047    4864 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 05:03:03.173208 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160052    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 05:03:03.173208 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160057    4864 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 05:03:03.173208 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160062    4864 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 05:03:03.173208 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160067    4864 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 05:03:03.173208 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160071    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 05:03:03.173208 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160075    4864 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 05:03:03.173208 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160079    4864 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 05:03:03.173208 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160083    4864 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 05:03:03.173208 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160088    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 05:03:03.173208 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160092    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 05:03:03.173208 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160097    4864 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 05:03:03.173208 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160101    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 05:03:03.173797 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160105    4864 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 05:03:03.173797 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160110    4864 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 05:03:03.173797 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160115    4864 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 05:03:03.173797 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160119    4864 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 05:03:03.173797 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160124    4864 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 05:03:03.173797 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160129    4864 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 05:03:03.173797 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160135    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 05:03:03.173797 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160140    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 05:03:03.173797 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160144    4864 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 05:03:03.173797 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160148    4864 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 05:03:03.173797 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160152    4864 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 05:03:03.173797 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160156    4864 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 05:03:03.173797 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160160    4864 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 05:03:03.173797 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160165    4864 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 05:03:03.173797 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160169    4864 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 05:03:03.173797 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160173    4864 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 05:03:03.173797 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160178    4864 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 05:03:03.173797 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160182    4864 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 05:03:03.173797 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160186    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 05:03:03.173797 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160190    4864 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 05:03:03.174465 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160194    4864 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 05:03:03.174465 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160199    4864 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 05:03:03.174465 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160203    4864 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 05:03:03.174465 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160207    4864 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 05:03:03.174465 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160212    4864 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 05:03:03.174465 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160216    4864 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 05:03:03.174465 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160220    4864 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 05:03:03.174465 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160224    4864 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 05:03:03.174465 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160229    4864 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 05:03:03.174465 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160233    4864 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 05:03:03.174465 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160237    4864 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 05:03:03.174465 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160241    4864 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 05:03:03.174465 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160245    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 05:03:03.174465 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160252    4864 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 05:03:03.174465 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160258    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 05:03:03.174465 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160263    4864 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 05:03:03.174465 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160268    4864 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 05:03:03.174465 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160274    4864 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 05:03:03.174465 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160280    4864 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 05:03:03.175048 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160284    4864 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 05:03:03.175048 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160289    4864 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 05:03:03.175048 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160293    4864 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 05:03:03.175048 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160298    4864 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 05:03:03.175048 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160302    4864 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 05:03:03.175048 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160305    4864 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 05:03:03.175048 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160310    4864 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 05:03:03.175048 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160314    4864 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 05:03:03.175048 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160318    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 05:03:03.175048 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160322    4864 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 05:03:03.175048 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160327    4864 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 05:03:03.175048 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160331    4864 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 05:03:03.175048 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160335    4864 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 05:03:03.175048 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160339    4864 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 05:03:03.175048 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160344    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 05:03:03.175048 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160348    4864 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 05:03:03.175048 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160353    4864 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 05:03:03.175048 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160357    4864 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 05:03:03.175048 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160361    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 05:03:03.175048 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160365    4864 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 05:03:03.175539 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160369    4864 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 05:03:03.175539 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160373    4864 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 05:03:03.175539 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160377    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 05:03:03.175539 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160382    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 05:03:03.175539 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160386    4864 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 05:03:03.175539 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160390    4864 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 05:03:03.175539 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160394    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 05:03:03.175539 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160398    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 05:03:03.175539 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160403    4864 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 05:03:03.175539 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160410    4864 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 05:03:03.175539 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160416    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 05:03:03.175539 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160422    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 05:03:03.175539 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160426    4864 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 05:03:03.175539 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160431    4864 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 05:03:03.175539 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160435    4864 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 05:03:03.175539 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160439    4864 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 05:03:03.175539 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160443    4864 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 05:03:03.175539 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160448    4864 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 05:03:03.175539 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160452    4864 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 05:03:03.175539 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160456    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 05:03:03.176052 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160460    4864 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 05:03:03.176052 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160464    4864 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 05:03:03.176052 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160468    4864 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 05:03:03.176052 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160472    4864 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 05:03:03.176052 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.160477    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 05:03:03.176052 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.164852    4864 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 01 05:03:03.176052 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.164869    4864 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 01 05:03:03.176052 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.164887    4864 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 01 05:03:03.176052 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.164892    4864 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 01 05:03:03.176052 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.164951    4864 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 05:03:03.176052 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.164957    4864 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 05:03:03.176052 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.164960    4864 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 05:03:03.176052 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.164963    4864 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 05:03:03.176052 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.164966    4864 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 05:03:03.176052 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.164968    4864 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 05:03:03.176052 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.164970    4864 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 05:03:03.176052 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.164973    4864 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 05:03:03.176052 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.164976    4864 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 05:03:03.176052 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.164979    4864 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 05:03:03.176052 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.164981    4864 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 05:03:03.176540 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.164984    4864 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 05:03:03.176540 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.164987    4864 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 05:03:03.176540 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.164989    4864 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 05:03:03.176540 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.164992    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 05:03:03.176540 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.164994    4864 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 05:03:03.176540 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.164997    4864 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 05:03:03.176540 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.164999    4864 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 05:03:03.176540 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165002    4864 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 05:03:03.176540 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165005    4864 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 05:03:03.176540 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165007    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 05:03:03.176540 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165009    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 05:03:03.176540 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165012    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 05:03:03.176540 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165015    4864 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 05:03:03.176540 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165017    4864 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 05:03:03.176540 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165021    4864 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 05:03:03.176540 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165023    4864 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 05:03:03.176540 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165026    4864 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 05:03:03.176540 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165028    4864 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 05:03:03.176540 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165036    4864 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 05:03:03.177139 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165039    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 05:03:03.177139 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165042    4864 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 05:03:03.177139 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165044    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 05:03:03.177139 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165046    4864 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 05:03:03.177139 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165049    4864 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 05:03:03.177139 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165051    4864 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 05:03:03.177139 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165055    4864 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 05:03:03.177139 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165059    4864 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 05:03:03.177139 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165061    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 05:03:03.177139 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165064    4864 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 05:03:03.177139 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165067    4864 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 05:03:03.177139 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165069    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 05:03:03.177139 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165072    4864 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 05:03:03.177139 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165074    4864 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 05:03:03.177139 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165077    4864 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 05:03:03.177139 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165079    4864 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 05:03:03.177139 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165082    4864 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 05:03:03.177139 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165084    4864 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 05:03:03.177139 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165087    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 05:03:03.177594 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165090    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 05:03:03.177594 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165093    4864 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 05:03:03.177594 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165096    4864 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 05:03:03.177594 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165098    4864 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 05:03:03.177594 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165100    4864 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 05:03:03.177594 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165103    4864 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 05:03:03.177594 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165105    4864 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 05:03:03.177594 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165108    4864 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 05:03:03.177594 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165110    4864 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 05:03:03.177594 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165113    4864 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 05:03:03.177594 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165115    4864 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 05:03:03.177594 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165118    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 05:03:03.177594 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165120    4864 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 05:03:03.177594 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165128    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 05:03:03.177594 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165131    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 05:03:03.177594 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165133    4864 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 05:03:03.177594 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165136    4864 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 05:03:03.177594 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165138    4864 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 05:03:03.177594 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165141    4864 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 05:03:03.177594 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165143    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 05:03:03.178084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165145    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 05:03:03.178084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165148    4864 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 05:03:03.178084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165150    4864 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 05:03:03.178084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165153    4864 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 05:03:03.178084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165155    4864 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 05:03:03.178084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165158    4864 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 05:03:03.178084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165160    4864 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 05:03:03.178084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165162    4864 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 05:03:03.178084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165165    4864 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 05:03:03.178084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165167    4864 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 05:03:03.178084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165169    4864 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 05:03:03.178084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165173    4864 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 05:03:03.178084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165175    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 05:03:03.178084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165178    4864 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 05:03:03.178084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165180    4864 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 05:03:03.178084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165183    4864 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 05:03:03.178084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165185    4864 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 05:03:03.178084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165188    4864 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 05:03:03.178084 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165190    4864 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 05:03:03.178552 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165194    4864 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 05:03:03.178552 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165197    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 05:03:03.178552 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165199    4864 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 05:03:03.178552 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165202    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 05:03:03.178552 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165204    4864 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 05:03:03.178552 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165206    4864 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 05:03:03.178552 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165209    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 05:03:03.178552 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165248    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 05:03:03.178552 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165590    4864 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 05:03:03.178552 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165601    4864 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 05:03:03.178552 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165607    4864 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 05:03:03.178552 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165613    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 05:03:03.178552 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165617    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 05:03:03.178552 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165622    4864 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 05:03:03.178552 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165627    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 05:03:03.178552 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165632    4864 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 05:03:03.178552 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165638    4864 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 05:03:03.178552 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.165642    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 05:03:03.179017 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.165728    4864 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 01 05:03:03.179017 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166872    4864 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 05:03:03.179017 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166882    4864 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 05:03:03.179017 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166885    4864 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 05:03:03.179017 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166888    4864 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 05:03:03.179017 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166891    4864 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 05:03:03.179017 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166895    4864 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 05:03:03.179017 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166898    4864 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 05:03:03.179017 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166901    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 05:03:03.179017 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166904    4864 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 05:03:03.179017 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166906    4864 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 05:03:03.179017 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166909    4864 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 05:03:03.179017 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166911    4864 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 05:03:03.179017 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166914    4864 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 05:03:03.179017 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166916    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 05:03:03.179017 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166919    4864 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 05:03:03.179409 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166922    4864 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 05:03:03.179409 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166924    4864 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 05:03:03.179409 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166927    4864 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 05:03:03.179409 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166929    4864 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 05:03:03.179409 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166932    4864 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 05:03:03.179409 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166935    4864 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 05:03:03.179409 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166938    4864 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 05:03:03.179409 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166941    4864 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 05:03:03.179409 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166943    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 05:03:03.179409 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166946    4864 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 05:03:03.179409 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166948    4864 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 05:03:03.179409 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166951    4864 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 05:03:03.179409 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166954    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 05:03:03.179409 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166958    4864 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 05:03:03.179409 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166962    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 05:03:03.179409 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166965    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 05:03:03.179409 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166968    4864 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 05:03:03.179409 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166970    4864 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 05:03:03.179409 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166973    4864 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 05:03:03.179409 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166975    4864 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 05:03:03.179949 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166978    4864 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 05:03:03.179949 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166980    4864 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 05:03:03.179949 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166983    4864 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 05:03:03.179949 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166986    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 05:03:03.179949 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166988    4864 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 05:03:03.179949 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166991    4864 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 05:03:03.179949 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166994    4864 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 05:03:03.179949 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166996    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 05:03:03.179949 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.166998    4864 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 05:03:03.179949 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167001    4864 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 05:03:03.179949 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167003    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 05:03:03.179949 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167006    4864 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 05:03:03.179949 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167008    4864 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 05:03:03.179949 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167011    4864 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 05:03:03.179949 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167013    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 05:03:03.179949 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167016    4864 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 05:03:03.179949 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167018    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 05:03:03.179949 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167021    4864 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 05:03:03.179949 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167023    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 05:03:03.180427 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167026    4864 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 05:03:03.180427 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167028    4864 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 05:03:03.180427 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167031    4864 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 05:03:03.180427 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167033    4864 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 05:03:03.180427 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167036    4864 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 05:03:03.180427 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167038    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 05:03:03.180427 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167040    4864 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 05:03:03.180427 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167043    4864 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 05:03:03.180427 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167045    4864 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 05:03:03.180427 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167048    4864 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 05:03:03.180427 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167050    4864 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 05:03:03.180427 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167053    4864 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 05:03:03.180427 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167055    4864 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 05:03:03.180427 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167058    4864 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 05:03:03.180427 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167060    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 05:03:03.180427 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167063    4864 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 05:03:03.180427 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167065    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 05:03:03.180427 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167068    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 05:03:03.180427 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167071    4864 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 05:03:03.181072 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167073    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 05:03:03.181072 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167076    4864 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 05:03:03.181072 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167078    4864 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 05:03:03.181072 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167081    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 05:03:03.181072 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167083    4864 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 05:03:03.181072 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167086    4864 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 05:03:03.181072 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167088    4864 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 05:03:03.181072 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167091    4864 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 05:03:03.181072 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167093    4864 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 05:03:03.181072 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167096    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 05:03:03.181072 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167098    4864 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 05:03:03.181072 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167100    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 05:03:03.181072 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167103    4864 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 05:03:03.181072 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167105    4864 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 05:03:03.181072 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167108    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 05:03:03.181072 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167121    4864 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 05:03:03.181072 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167125    4864 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 05:03:03.181072 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167128    4864 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 05:03:03.181072 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167131    4864 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 05:03:03.181579 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167133    4864 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 05:03:03.181579 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167136    4864 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 05:03:03.181579 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167138    4864 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 05:03:03.181579 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167141    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 05:03:03.181579 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167143    4864 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 05:03:03.181579 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167146    4864 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 05:03:03.181579 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167148    4864 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 05:03:03.181579 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167151    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 05:03:03.181579 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167153    4864 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 05:03:03.181579 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167156    4864 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 05:03:03.181579 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167158    4864 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 05:03:03.181579 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167161    4864 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 05:03:03.181579 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167164    4864 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 05:03:03.181579 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167166    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 05:03:03.181579 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.167195    4864 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 01 05:03:03.181579 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167244    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 05:03:03.182002 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167250    4864 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 05:03:03.182002 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167254    4864 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 05:03:03.182002 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167257    4864 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 05:03:03.182002 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167259    4864 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 05:03:03.182002 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167262    4864 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 05:03:03.182002 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167265    4864 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 05:03:03.182002 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167268    4864 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 05:03:03.182002 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167271    4864 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 05:03:03.182002 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167273    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 05:03:03.182002 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167276    4864 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 05:03:03.182002 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167278    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 05:03:03.182002 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167280    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 05:03:03.182002 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167283    4864 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 05:03:03.182002 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167286    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 05:03:03.182002 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167289    4864 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 05:03:03.182002 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167291    4864 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 05:03:03.182002 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167294    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 05:03:03.182002 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167297    4864 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 05:03:03.182002 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167299    4864 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 05:03:03.182477 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167301    4864 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 05:03:03.182477 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167304    4864 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 05:03:03.182477 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167306    4864 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 05:03:03.182477 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167308    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 05:03:03.182477 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167311    4864 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 05:03:03.182477 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167313    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 05:03:03.182477 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167315    4864 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 05:03:03.182477 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167318    4864 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 05:03:03.182477 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167320    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 05:03:03.182477 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167323    4864 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 05:03:03.182477 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167327    4864 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 05:03:03.182477 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167331    4864 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 05:03:03.182477 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167333    4864 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 05:03:03.182477 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167336    4864 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 05:03:03.182477 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167338    4864 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 05:03:03.182477 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167341    4864 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 05:03:03.182477 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167343    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 05:03:03.182477 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167345    4864 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 05:03:03.182477 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167348    4864 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 05:03:03.182477 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167350    4864 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 05:03:03.182991 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167352    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 05:03:03.182991 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167355    4864 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 05:03:03.182991 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167357    4864 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 05:03:03.182991 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167360    4864 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 05:03:03.182991 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167362    4864 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 05:03:03.182991 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167365    4864 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 05:03:03.182991 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167367    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 05:03:03.182991 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167370    4864 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 05:03:03.182991 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167372    4864 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 05:03:03.182991 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167375    4864 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 05:03:03.182991 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167377    4864 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 05:03:03.182991 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167380    4864 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 05:03:03.182991 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167382    4864 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 05:03:03.182991 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167385    4864 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 05:03:03.182991 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167388    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 05:03:03.182991 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167390    4864 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 05:03:03.182991 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167392    4864 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 05:03:03.182991 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167395    4864 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 05:03:03.182991 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167397    4864 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 05:03:03.182991 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167400    4864 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 05:03:03.183554 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167405    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 05:03:03.183554 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167408    4864 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 05:03:03.183554 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167410    4864 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 05:03:03.183554 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167413    4864 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 05:03:03.183554 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167415    4864 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 05:03:03.183554 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167418    4864 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 05:03:03.183554 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167420    4864 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 05:03:03.183554 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167423    4864 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 05:03:03.183554 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167426    4864 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 05:03:03.183554 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167429    4864 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 05:03:03.183554 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167431    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 05:03:03.183554 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167434    4864 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 05:03:03.183554 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167436    4864 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 05:03:03.183554 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167439    4864 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 05:03:03.183554 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167441    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 05:03:03.183554 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167445    4864 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 05:03:03.183554 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167448    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 05:03:03.183554 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167450    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 05:03:03.183554 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167453    4864 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 05:03:03.183554 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167455    4864 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 05:03:03.184389 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167458    4864 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 05:03:03.184389 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167460    4864 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 05:03:03.184389 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167463    4864 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 05:03:03.184389 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167465    4864 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 05:03:03.184389 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167468    4864 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 05:03:03.184389 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167470    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 05:03:03.184389 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167473    4864 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 05:03:03.184389 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167475    4864 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 05:03:03.184389 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167479    4864 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 05:03:03.184389 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167481    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 05:03:03.184389 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167484    4864 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 05:03:03.184389 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167486    4864 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 05:03:03.184389 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167490    4864 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 05:03:03.184389 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167492    4864 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 05:03:03.184389 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167495    4864 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 05:03:03.184389 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167497    4864 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 05:03:03.184389 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167500    4864 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 05:03:03.184389 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167503    4864 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 05:03:03.184389 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167505    4864 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 05:03:03.184389 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167508    4864 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 05:03:03.184935 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167510    4864 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 05:03:03.184935 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167513    4864 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 05:03:03.184935 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167516    4864 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 05:03:03.184935 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167519    4864 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 05:03:03.184935 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167521    4864 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 05:03:03.184935 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:03.167524    4864 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 05:03:03.184935 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.167578    4864 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 01 05:03:03.184935 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.168587    4864 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 01 05:03:03.184935 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.168643    4864 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 01 05:03:03.184935 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.168850    4864 server.go:1014] "Starting client certificate rotation"
Nov 01 05:03:03.184935 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.168981    4864 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 01 05:03:03.184935 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.169021    4864 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-31 04:57:21 +0000 UTC" deadline="2028-06-15 20:14:01.004934639 +0000 UTC"
Nov 01 05:03:03.184935 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.169049    4864 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14223h10m57.835888024s"
Nov 01 05:03:03.184935 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.169308    4864 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 01 05:03:03.184935 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.170370    4864 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 01 05:03:03.184935 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.171828    4864 log.go:25] "Validated CRI v1 runtime API"
Nov 01 05:03:03.185365 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.173445    4864 log.go:25] "Validated CRI v1 image API"
Nov 01 05:03:03.185365 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.174205    4864 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 01 05:03:03.185365 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.176112    4864 fs.go:135] Filesystem UUIDs: map[569cc989-5283-41df-9103-52866cfe9d73:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e1eb79c9-a3af-4e8d-8cea-fd91056eada3:/dev/nvme0n1p3]
Nov 01 05:03:03.185463 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.176135    4864 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/256bbfe43b8fe4de7449f137b92cb1b8759725d201896ca0c5d890f4850969c5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/256bbfe43b8fe4de7449f137b92cb1b8759725d201896ca0c5d890f4850969c5/userdata/shm major:0 minor:290 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/273a50863469fb5a4e75ef299b4cdbe92adb5a2508cc1fa9e2e393bd457b5a4d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/273a50863469fb5a4e75ef299b4cdbe92adb5a2508cc1fa9e2e393bd457b5a4d/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3c1331112ec45d56fd63f15fd4809c720cc0ee0bd39f24754eef6fe229b37653/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3c1331112ec45d56fd63f15fd4809c720cc0ee0bd39f24754eef6fe229b37653/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3cb3ab0bf1302a3e326f502441612a54ec24b932a304b4484211e0d8adbf31c1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3cb3ab0bf1302a3e326f502441612a54ec24b932a304b4484211e0d8adbf31c1/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/410d7603e73aa5da9a5b4a98d3484392406cb6c36f6d22b85cd73dfab13fe0c0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/410d7603e73aa5da9a5b4a98d3484392406cb6c36f6d22b85cd73dfab13fe0c0/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5c9e60c7c7a0e875b7ac9eca1c09f7ae1992f2153833836e1d22d79bf5e54c8e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5c9e60c7c7a0e875b7ac9eca1c09f7ae1992f2153833836e1d22d79bf5e54c8e/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5fcb2a03ebf60efd422b67e506e0739906758c878f7bf46ee7ce4f792bd0a10b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5fcb2a03ebf60efd422b67e506e0739906758c878f7bf46ee7ce4f792bd0a10b/userdata/shm major:0 minor:70 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7bfde12401734403cb360b41d973d858fe8124d5920b6c4f2023738afd03bf5d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7bfde12401734403cb360b41d973d858fe8124d5920b6c4f2023738afd03bf5d/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/89381bfb4116e1748e052dd89cce07fe6c629b844d963fb75c618944a05a1707/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/89381bfb4116e1748e052dd89cce07fe6c629b844d963fb75c618944a05a1707/userdata/shm major:0 minor:232 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/914e556f074a17e41fd70606c911cc4933035569ca01d47ea3cdf87f5b697df1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/914e556f074a17e41fd70606c911cc4933035569ca01d47ea3cdf87f5b697df1/userdata/shm major:0 minor:229 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/94ecffcf8dc81a911dd5e6c02fac43e13e1ab76ddd815f0a6deff64dcc59a08a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/94ecffcf8dc81a911dd5e6c02fac43e13e1ab76ddd815f0a6deff64dcc59a08a/userdata/shm major:0 minor:75 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/996b756136bcfaab4303a0eeb394dd84c47b6b8ad7c5830fff57e8cfb084ec6a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/996b756136bcfaab4303a0eeb394dd84c47b6b8ad7c5830fff57e8cfb084ec6a/userdata/shm major:0 minor:210 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9cb17b91c845339f4cc640538f1e369ed1363d614799a796d296164bf9b93658/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9cb17b91c845339f4cc640538f1e369ed1363d614799a796d296164bf9b93658/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a34c6abf742f8f286b936d09b642c7ad10b290cc9e7f39dfbfc46d59a9b0e9b6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a34c6abf742f8f286b936d09b642c7ad10b290cc9e7f39dfbfc46d59a9b0e9b6/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bca57b561e663cb1f3a26e52878ee097de3b83d75b9c9b60932ccbfe1bb2b4e1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bca57b561e663cb1f3a26e52878ee097de3b83d75b9c9b60932ccbfe1bb2b4e1/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c6b287a466534cdc830fed20da4cf5655443d448ebdb9857c93bd0bc6c5cb3b8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c6b287a466534cdc830fed20da4cf5655443d448ebdb9857c93bd0bc6c5cb3b8/userdata/shm major:0 minor:91 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cb074bfe629a379116a5c363d223c78cd86d750828f0543b494ad362282da496/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cb074bfe629a379116a5c363d223c78cd86d750828f0543b494ad362282da496/userdata/shm major:0 minor:287 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d017aa4053940b857f08931573d7b582976c3cdb191c0ab67f7b8c6858b5c33f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d017aa4053940b857f08931573d7b582976c3cdb191c0ab67f7b8c6858b5c33f/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/de33dda5ec106d52d85fa7e3acb23fe0845dd765a8b3e7a9e2e274f6da780bb5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/de33dda5ec106d52d85fa7e3acb23fe0845dd765a8b3e7a9e2e274f6da780bb5/userdata/shm major:0 minor:226 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f3fd3df15230b3254b41078ef4220d57100eb071bc9a1b029a7e3da10c795e43/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f3fd3df15230b3254b41078ef4220d57100eb071bc9a1b029a7e3da10c795e43/userdata/shm major:0 minor:218 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fa241532a7ae7a09f847da65293301bad95da33ebfb7b6ff170ce467ce27815a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fa241532a7ae7a09f847da65293301bad95da33ebfb7b6ff170ce467ce27815a/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fc7aa10b2d58b36d8ae5ee178b2b253e0f9e24d5684ced6ee4ea7636a3c5cd36/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fc7aa10b2d58b36d8ae5ee178b2b253e0f9e24d5684ced6ee4ea7636a3c5cd36/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0197ee7e-0d4a-4e9c-8057-26356d6f34b3/volumes/kubernetes.io~projected/kube-api-access-xmbj9:{mountpoint:/var/lib/kubelet/pods/0197ee7e-0d4a-4e9c-8057-26356d6f34b3/volumes/kubernetes.io~projected/kube-api-access-xmbj9 major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/037937cc-75c9-4416-bd8f-63589df05625/volumes/kubernetes.io~projected/kube-api-access-w4qrj:{mountpoint:/var/lib/kubelet/pods/037937cc-75c9-4416-bd8f-63589df05625/volumes/kubernetes.io~projected/kube-api-access-w4qrj major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0bdedda0-279a-41fd-a7cc-300c816985df/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/0bdedda0-279a-41fd-a7cc-300c816985df/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0bdedda0-279a-41fd-a7cc-300c816985df/volumes/kubernetes.io~projected/kube-api-access-2nh4k:{mountpoint:/var/lib/kubelet/pods/0bdedda0-279a-41fd-a7cc-300c816985df/volumes/kubernetes.io~projected/kube-api-access-2nh4k major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0bdedda0-279a-41fd-a7cc-300c816985df/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/0bdedda0-279a-41fd-a7cc-300c816985df/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1967227b-a4ea-4ee6-945d-efb0b5040a13/volumes/kubernetes.io~projected/kube-api-access-xfldn:{mountpoint:/var/lib/kubelet/pods/1967227b-a4ea-4ee6-945d-efb0b5040a13/volumes/kubernetes.io~projected/kube-api-access-xfldn major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1967227b-a4ea-4ee6-945d-efb0b5040a13/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/1967227b-a4ea-4ee6-945d-efb0b5040a13/volumes/kubernetes.io~secret/serving-cert major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/36a4e628-5f12-4849-9d77-deaa3e93b190/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/36a4e628-5f12-4849-9d77-deaa3e93b190/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:282 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/45a40b3e-ab72-4b18-933f-f52272a6742c/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/45a40b3e-ab72-4b18-933f-f52272a6742c/volumes/kubernetes.io~secret/agent-certs major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/48c05e4f-a673-4f5f-8c21-8dc1ff716683/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/48c05e4f-a673-4f5f-8c21-8dc1ff716683/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/48c05e4f-a673-4f5f-8c21-8dc1ff716683/volumes/kubernetes.io~projected/kube-api-access-rdspf:{mountpoint:/var/lib/kubelet/pods/48c05e4f-a673-4f5f-8c21-8dc1ff716683/volumes/kubernetes.io~projected/kube-api-access-rdspf major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/48c05e4f-a673-4f5f-8c21-8dc1ff716683/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/48c05e4f-a673-4f5f-8c21-8dc1ff716683/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/48c05e4f-a673-4f5f-8c21-8dc1ff716683/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/48c05e4f-a673-4f5f-8c21-8dc1ff716683/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/57fa3738-c3e8-4c18-9582-f6e8377a041e/volumes/kubernetes.io~projected/kube-api-access-p9lkh:{mountpoint:/var/lib/kubelet/pods/57fa3738-c3e8-4c18-9582-f6e8377a041e/volumes/kubernetes.io~projected/kube-api-access-p9lkh major:0 minor:239 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5dc708b0-6c35-4f79-a415-e7533381b95b/volumes/kubernetes.io~projected/kube-api-access-wblsz:{mountpoint:/var/lib/kubelet/pods/5dc708b0-6c35-4f79-a415-e7533381b95b/volumes/kubernetes.io~projected/kube-api-access-wblsz major:0 minor:236 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/62652d56-1be6-47cc-a774-77d4fa9d206a/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/62652d56-1be6-47cc-a774-77d4fa9d206a/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/62652d56-1be6-47cc-a774-77d4fa9d206a/volumes/kubernetes.io~projected/kube-api-access-9n4xr:{mountpoint:/var/lib/kubelet/pods/62652d56-1be6-47cc-a774-77d4fa9d206a/volumes/kubernetes.io~projected/kube-api-access-9n4xr major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/62652d56-1be6-47cc-a774-77d4fa9d206a/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/62652d56-1be6-47cc-a774-77d4fa9d206a/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/62652d56-1be6-47cc-a774-77d4fa9d206a/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/62652d56-1be6-47cc-a774-77d4fa9d206a/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/636cccf5-2397-4dca-899a-3598a7b8096b/volumes/kubernetes.io~projected/kube-api-access-5pj75:{mountpoint:/var/lib/kubelet/pods/636cccf5-2397-4dca-899a-3598a7b8096b/volumes/kubernetes.io~projected/kube-api-access-5pj75 major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/636cccf5-2397-4dca-899a-3598a7b8096b/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/636cccf5-2397-4dca-899a-3598a7b8096b/volumes/kubernetes.io~secret/serving-cert major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6a747e88-6f89-4f3a-ba71-c97babf998f6/volumes/kubernetes.io~projected/kube-api-access-njxgf:{mountpoint:/var/lib/kubelet/pods/6a747e88-6f89-4f3a-ba71-c97babf998f6/volumes/kubernetes.io~projected/kube-api-access-njxgf major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c9eec03-4159-4a5e-a284-174d634ae0c9/volumes/kubernetes.io~projected/kube-api-access-rzrjv:{mountpoint:/var/lib/kubelet/pods/7c9eec03-4159-4a5e-a284-174d634ae0c9/volumes/kubernetes.io~projected/kube-api-access-rzrjv major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7fb489a8-8a4f-45f2-977b-594b6126281b/volumes/kubernetes.io~projected/kube-api-access-hlhpw:{mountpoint:/var/lib/kubelet/pods/7fb489a8-8a4f-45f2-977b-594b6126281b/volumes/kubernetes.io~projected/kube-api-access-hlhpw major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/82335b9e-5584-4afc-b1fc-cc003dbcad4b/volumes/kubernetes.io~projected/kube-api-access-d8l8g:{mountpoint:/var/lib/kubelet/pods/82335b9e-5584-4afc-b1fc-cc003dbcad4b/volumes/kubernetes.io~projected/kube-api-access-d8l8g major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/82335b9e-5584-4afc-b1fc-cc003dbcad4b/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/82335b9e-5584-4afc-b1fc-cc003dbcad4b/volumes/kubernetes.io~secret/default-certificate major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/82335b9e-5584-4afc-b1fc-cc003dbcad4b/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/82335b9e-5584-4afc-b1fc-cc003dbcad4b/volumes/kubernetes.io~secret/stats-auth major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/878896d2-fa82-46ef-af5e-0bf491216b03/volumes/kubernetes.io~projected/kube-api-access-qjg2z:{mountpoint:/var/lib/kubelet/pods/878896d2-fa82-46ef-af5e-0bf491216b03/volumes/kubernetes.io~projected/kube-api-access-qjg2z major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/878896d2-fa82-46ef-af5e-0bf491216b03/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/878896d2-fa82-46ef-af5e-0bf491216b03/volumes/kubernetes.io~secret/serving-cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a01a018f-2c6e-413f-b88f-37b4e91699b1/volumes/kubernetes.io~projected/kube-api-access-v6b98:{mountpoint:/var/lib/kubelet/pods/a01a018f-2c6e-413f-b88f-37b4e91699b1/volumes/kubernetes.io~projected/kube-api-access-v6b98 major:0 minor:238 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a5b30117-7cb8-43ec-92a8-4e326b48c22d/volumes/kubernetes.io~projected/kube-api-access-bzgwz:{mountpoint:/var/lib/kubelet/pods/a5b30117-7cb8-43ec-92a8-4e326b48c22d/volumes/kubernetes.io~projected/kube-api-access-bzgwz major:0 minor:289 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b53e86a9-ca94-4753-b57b-47eb94bbd879/volumes/kubernetes.io~projected/kube-api-access-bj67m:{mountpoint:/var/lib/kubelet/pods/b53e86a9-ca94-4753-b57b-47eb94bbd879/volumes/kubernetes.io~projected/kube-api-access-bj67m major:0 minor:237 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b53e86a9-ca94-4753-b57b-47eb94bbd879/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/b53e86a9-ca94-4753-b57b-47eb94bbd879/volumes/kubernetes.io~secret/cert major:0 minor:231 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b91ced44-8be6-47c8-9229-549400076eb7/volumes/kubernetes.io~projected/kube-api-access-hvgdb:{mountpoint:/var/lib/kubelet/pods/b91ced44-8be6-47c8-9229-549400076eb7/volumes/kubernetes.io~projected/kube-api-access-hvgdb major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bb889bee-f31e-47f1-8617-6a27bc1eebf4/volumes/kubernetes.io~projected/kube-api-access-bxknn:{mountpoint:/var/lib/kubelet/pods/bb889bee-f31e-47f1-8617-6a27bc1eebf4/volumes/kubernetes.io~projected/kube-api-access-bxknn major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bce36747-2159-4b72-8c0e-d98226588d18/volumes/kubernetes.io~projected/kube-api-access-hkbbr:{mountpoint:/var/lib/kubelet/pods/bce36747-2159-4b72-8c0e-d98226588d18/volumes/kubernetes.io~projected/kube-api-access-hkbbr major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bce36747-2159-4b72-8c0e-d98226588d18/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/bce36747-2159-4b72-8c0e-d98226588d18/volumes/kubernetes.io~secret/serving-cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47/volumes/kubernetes.io~projected/kube-api-access-h6qrt:{mountpoint:/var/lib/kubelet/pods/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47/volumes/kubernetes.io~projected/kube-api-access-h6qrt major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d09454b6-6ac7-46dc-83ee-a3c60dfdf262/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/d09454b6-6ac7-46dc-83ee-a3c60dfdf262/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d09454b6-6ac7-46dc-83ee-a3c60dfdf262/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/d09454b6-6ac7-46dc-83ee-a3c60dfdf262/volumes/kubernetes.io~empty-dir/tmp major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d09454b6-6ac7-46dc-83ee-a3c60dfdf262/volumes/kubernetes.io~projected/kube-api-access-zdwk6:{mountpoint:/var/lib/kubelet/pods/d09454b6-6ac7-46dc-83ee-a3c60dfdf262/volumes/kubernetes.io~projected/kube-api-access-zdwk6 major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d1f4c040-7436-4df1-8462-6a38acee2c5a/volumes/kubernetes.io~projected/kube-api-access-h6vvf:{mountpoint:/var/lib/kubelet/pods/d1f4c040-7436-4df1-8462-6a38acee2c5a/volumes/kubernetes.io~projected/kube-api-access-h6vvf major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/df7d0108-685b-4c27-8d0f-bccf4ca5dbbe/volumes/kubernetes.io~projected/kube-api-access-z9hd4:{mountpoint:/var/lib/kubelet/pods/df7d0108-685b-4c27-8d0f-bccf4ca5dbbe/volumes/kubernetes.io~projected/kube-api-access-z9hd4 major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ee3cc716-4716-4207-a493-9eeee140a6fe/volumes/kubernetes.io~projected/kube-api-access-tw4b5:{mountpoint:/var/lib/kubelet/pods/ee3cc716-4716-4207-a493-9eeee140a6fe/volumes/kubernetes.io~projected/kube-api-access-tw4b5 major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ef46e568-5c98-4520-99e5-c49c4c3aca2a/volumes/kubernetes.io~projected/kube-api-access-2j8z2:{mountpoint:/var/lib/kubelet/pods/ef46e568-5c98-4520-99e5-c49c4c3aca2a/volumes/kubernetes.io~projected/kube-api-access-2j8z2 major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb78a5ab-3a2d-45fa-abaa-7b26c33ec929/volumes/kubernetes.io~projected/kube-api-access-dhm7l:{mountpoint:/var/lib/kubelet/pods/fb78a5ab-3a2d-45fa-abaa-7b26c33ec929/volumes/kubernetes.io~projected/kube-api-access-dhm7l major:0 minor:205 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/b5459756dd8a63fd3dfff8f8752b84aa57c6fcfe576a220fcf226f17e835226c/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/ee5cc98b16dee8982a52be6a73ee0c16e68181966930c513fe52d5455d9c605a/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/25d3651d48246a82c3084eebe130d5f7c8b9c077e58b281cec18b4b31c4f7d75/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-105:{mountpoint:/var/lib/containers/storage/overlay/0b52fa4738c1df8f9779f5e634110f355f70083d0445cfa32caa0bb9ce1cda17/merged major:0 minor:105 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/d9e2abee82448b87b5a4c1d7f843e41756cbaed6acd134b15977e908be02812f/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/1bf4578d33b43e7fae40ffa607080c5e344fe8f6d30f072ef26c4a856e64a2c9/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/3b282cd88adfbb68ac8a437e171d1dd1bbafd1aa47fa1c9771f4cbdd0df17129/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/6ddceb2ae2d1188ec2b43a84792fde17333a730d73477f1ee051159d97dccf09/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/4a04c0f9bf87221dabe822fbf8b1b916d545d378fe54061f1571bb75e603b60d/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/d2a3359ff8936f63abe4dc4e2b5b910d8e736ab42e5d4c917af2b7befd73b7db/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/3c0fd9dcca2973f0b4f3f58b71b1506949f00773b98cbb02922e572884ffb18e/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/1a85f66cabdd645d167a4d4aeecf6cdb75bcf282619a90a590ec6c91b25b74c2/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-149:{mountpoint:/var/lib/containers/storage/overlay/c311bdb1d9ea8e773d5d1cbeaec0376189fae845b8f5351ce3fc334c58b0b17f/merged major:0 minor:149 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/e566e2e6ba22ceccda5994d35b2616547c822de8d66b1aa19e0698a459dfdb61/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/f2dc43d0737d7ad50a18165632d614a10f833c6090159a18d4ec84abb236dc15/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/c807ce16493ed683b043ed420a265a3b0f9265c19aae3daea47e9e7fdd5f9852/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/69b329762a09f05ce76a4911982217bda92d8bc59d3fef42391e77e978f3a1cb/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/4240e54fb1ceecf77fcfb1c921d309385c7294f4b0dfd7c75e5693a9c0da9532/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/0d25f40b83e4fe980aebf320f914c6eca55e764acf358010dc162296f34a5164/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/16e2407578e5a07d46dd698c54708562b1273db7f02229f7de6f1e1c82311ee9/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/2c650d20ff7311f8a60c1ff4f49bc2b9ec8d9b9cfc458729f381cda2beee0921/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/156750de930ce4adbe1b701165bc4bfcc4c94e000d55e857b6708d442f7d954a/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/9032da8563354919fce878c2f8ba6e40e4f8eda6c6fdab40c0d828cee54e9ebb/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/ade6f754b4c83dc1ea00a3254369b2f3dfaca88a290bc262225224c809d362c6/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/8560815ee55dbd3f1966498788f9fe60fcf555073fab1d96642addc99b3e180f/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/4cb94c475ff5fa4e0e7b5c300b373f5b119ca8459276811e10cb5ea248c4f7c1/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/31b146482a4cbda00ce9637f8bc85ddd4c1d911726c230a1c24600f8ff1bb979/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/0c40efb617d45c863f8733d7a697b2920e4d9474b41021e89f4126e646063527/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/fe0342f7704fa97268594c09a75cbf3769b6c46855749cf0136216a04da9b9eb/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-272:{mountpoint:/var/lib/containers/storage/overlay/389f2880464943a98ecb753d4694752342f18098e3a5fd742a85c315cfe942ac/merged major:0 minor:272 fsType:overlay blockSize:0} overlay_0-274:{mountpoint:/var/lib/containers/storage/overlay/adad1ced64924b9d7546ea72066b1136cebe7da25e4074d08129cca544fa5c53/merged major:0 minor:274 fsType:overlay blockSize:0} overlay_0-276:{mountpoint:/var/lib/containers/storage/overlay/1d792a8aa5d2b0ea4e7b8306ab2da2395e66128d6d80c89fb6328230f2b8588a/merged major:0 minor:276 fsType:overlay blockSize:0} overlay_0-278:{mountpoint:/var/lib/containers/storage/overlay/14bac6573546169dee07596fc3be7f2929c9a4ade2a60c62cd711f122921e46c/merged major:0 minor:278 fsType:overlay blockSize:0} overlay_0-280:{mountpoint:/var/lib/containers/storage/overlay/df8b6f96023ec0e2f06f0f3c499ad4e6c86fae35ce8ffb9988835ef38bf27bb9/merged major:0 minor:280 fsType:overlay blockSize:0} overlay_0-292:{mountpoint:/var/lib/containers/storage/overlay/3688c86bbd18f34f85283c4a9bef8db1c478c3a1b20b59cdffd4ff7a9de8aef4/merged major:0 minor:292 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/739fd183f87d5bed1ba294168fad0e6de308b36d042f208b07b608c2e5ba8167/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-296:{mountpoint:/var/lib/containers/storage/overlay/ecb1bc53d167bc10ca88acd560dea4aa5ad7aebbbcbbc4bc26a0e2c3f42b9c65/merged major:0 minor:296 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/846cb3235610793b3271911eda419d348430c69c15b384f58758840f5ab9cab1/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/a22f8f8a0917baddab6e1f1b53cf13217dfdac903862744a1720f91f78974736/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-306:{mountpoint:/var/lib/containers/storage/overlay/5167832fe9d7e0e077923176cec04ab27098518ec77aa9535e81dcf17982efaf/merged major:0 minor:306 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/b2b9b184dafd96367acdf6d749651f0430eb90bc2afe86c7a5e92068e6005d27/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-327:{mountpoint:/var/lib/containers/storage/overlay/64ede9257b82ba922468f2fc3b21192e2e3704e7df6779c9300b4a7306ce3794/merged major:0 minor:327 fsType:overlay blockSize:0} overlay_0-329:{mountpoint:/var/lib/containers/storage/overlay/3ed06c9f44c24c8d4975ddc7d27ae78fe7250cfed035e72b9588b26e6079830d/merged major:0 minor:329 fsType:overlay blockSize:0} overlay_0-335:{mountpoint:/var/lib/containers/storage/overlay/72fbfed3ee9edea98b0402c87b3b27a64ebdd55fb002617993ccd2d486021838/merged major:0 minor:335 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/5b4bd6555ad75aafce7cf413d6ed0ee17872008296c42d39b840d9c33cd6cebd/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/ad8afb88e1986cd2995fae8e774894b6ca4aa2ffdf3ae2838a5b4241ceaf6715/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/3ab3da491dfcaa1f67d80c0b8ac6495804d8baa85353c6a7a789166e0fb85688/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/37a5b9b24826b702291bb8550eef58da50c4537b21dacc7dd48055b19d6650c8/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/10b4fa646fdab94cc43fc08ce53ff2a36a7e6aba0e2a74649a60acd65812524b/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/efafe4334c20e5b957e1fd051399ce960ca2d521d7f212277c4ecbd05f525a54/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/8d86ca2144326936ccdf97465d28f6d233fa035323fc4a20c09d903c457c4754/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/4a2d753e5e08543a6b744de2569dd500791ebfe23d046a97ae42f3630dd56881/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/a1712ab0387deb9cdfc027ce572dee36ddfb1ad4fbe498ecfdd220fd1138ae6b/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/fa1cc5c187e35dcb852b4f8817ebd0753da13470db33bf4fe83d051ce58e3d58/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 01 05:03:03.186203 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.185896    4864 manager.go:217] Machine: {Timestamp:2026-11-01 05:03:03.18542243 +0000 UTC m=+0.075798397 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3135440 MemoryCapacity:32812011520 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec273bbdaeb0eb3fbec17f1db8c05b76 SystemUUID:ec273bbd-aeb0-eb3f-bec1-7f1db8c05b76 BootID:572b1f06-7e71-47d8-8d18-ff5273b0e0d1 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/878896d2-fa82-46ef-af5e-0bf491216b03/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:201 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-272 DeviceMajor:0 DeviceMinor:272 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/48c05e4f-a673-4f5f-8c21-8dc1ff716683/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:202 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/89381bfb4116e1748e052dd89cce07fe6c629b844d963fb75c618944a05a1707/userdata/shm DeviceMajor:0 DeviceMinor:232 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/5dc708b0-6c35-4f79-a415-e7533381b95b/volumes/kubernetes.io~projected/kube-api-access-wblsz DeviceMajor:0 DeviceMinor:236 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/48c05e4f-a673-4f5f-8c21-8dc1ff716683/volumes/kubernetes.io~projected/kube-api-access-rdspf DeviceMajor:0 DeviceMinor:208 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0197ee7e-0d4a-4e9c-8057-26356d6f34b3/volumes/kubernetes.io~projected/kube-api-access-xmbj9 DeviceMajor:0 DeviceMinor:67 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ef46e568-5c98-4520-99e5-c49c4c3aca2a/volumes/kubernetes.io~projected/kube-api-access-2j8z2 DeviceMajor:0 DeviceMinor:62 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/d1f4c040-7436-4df1-8462-6a38acee2c5a/volumes/kubernetes.io~projected/kube-api-access-h6vvf DeviceMajor:0 DeviceMinor:65 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f3fd3df15230b3254b41078ef4220d57100eb071bc9a1b029a7e3da10c795e43/userdata/shm DeviceMajor:0 DeviceMinor:218 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/94ecffcf8dc81a911dd5e6c02fac43e13e1ab76ddd815f0a6deff64dcc59a08a/userdata/shm DeviceMajor:0 DeviceMinor:75 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c6b287a466534cdc830fed20da4cf5655443d448ebdb9857c93bd0bc6c5cb3b8/userdata/shm DeviceMajor:0 DeviceMinor:91 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-292 DeviceMajor:0 DeviceMinor:292 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/410d7603e73aa5da9a5b4a98d3484392406cb6c36f6d22b85cd73dfab13fe0c0/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-276 DeviceMajor:0 DeviceMinor:276 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bb889bee-f31e-47f1-8617-6a27bc1eebf4/volumes/kubernetes.io~projected/kube-api-access-bxknn DeviceMajor:0 DeviceMinor:58 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-306 DeviceMajor:0 DeviceMinor:306 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fc7aa10b2d58b36d8ae5ee178b2b253e0f9e24d5684ced6ee4ea7636a3c5cd36/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/62652d56-1be6-47cc-a774-77d4fa9d206a/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:217 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/1967227b-a4ea-4ee6-945d-efb0b5040a13/volumes/kubernetes.io~projected/kube-api-access-xfldn DeviceMajor:0 DeviceMinor:223 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/636cccf5-2397-4dca-899a-3598a7b8096b/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:212 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d09454b6-6ac7-46dc-83ee-a3c60dfdf262/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:55 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3c1331112ec45d56fd63f15fd4809c720cc0ee0bd39f24754eef6fe229b37653/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/996b756136bcfaab4303a0eeb394dd84c47b6b8ad7c5830fff57e8cfb084ec6a/userdata/shm DeviceMajor:0 DeviceMinor:210 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-296 DeviceMajor:0 DeviceMinor:296 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0bdedda0-279a-41fd-a7cc-300c816985df/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:57 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/7c9eec03-4159-4a5e-a284-174d634ae0c9/volumes/kubernetes.io~projected/kube-api-access-rzrjv DeviceMajor:0 DeviceMinor:322 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/62652d56-1be6-47cc-a774-77d4fa9d206a/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:214 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/b53e86a9-ca94-4753-b57b-47eb94bbd879/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:231 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3cb3ab0bf1302a3e326f502441612a54ec24b932a304b4484211e0d8adbf31c1/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/636cccf5-2397-4dca-899a-3598a7b8096b/volumes/kubernetes.io~projected/kube-api-access-5pj75 DeviceMajor:0 DeviceMinor:221 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/62652d56-1be6-47cc-a774-77d4fa9d206a/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:224 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/62652d56-1be6-47cc-a774-77d4fa9d206a/volumes/kubernetes.io~projected/kube-api-access-9n4xr DeviceMajor:0 DeviceMinor:227 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/de33dda5ec106d52d85fa7e3acb23fe0845dd765a8b3e7a9e2e274f6da780bb5/userdata/shm DeviceMajor:0 DeviceMinor:226 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/b53e86a9-ca94-4753-b57b-47eb94bbd879/volumes/kubernetes.io~projected/kube-api-access-bj67m DeviceMajor:0 DeviceMinor:237 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-280 DeviceMajor:0 DeviceMinor:280 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/45a40b3e-ab72-4b18-933f-f52272a6742c/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:64 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/0bdedda0-279a-41fd-a7cc-300c816985df/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/1967227b-a4ea-4ee6-945d-efb0b5040a13/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:213 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/82335b9e-5584-4afc-b1fc-cc003dbcad4b/volumes/kubernetes.io~projected/kube-api-access-d8l8g DeviceMajor:0 DeviceMinor:222 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/6a747e88-6f89-4f3a-ba71-c97babf998f6/volumes/kubernetes.io~projected/kube-api-access-njxgf DeviceMajor:0 DeviceMinor:66 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9cb17b91c845339f4cc640538f1e369ed1363d614799a796d296164bf9b93658/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/bce36747-2159-4b72-8c0e-d98226588d18/volumes/kubernetes.io~projected/kube-api-access-hkbbr DeviceMajor:0 DeviceMinor:209 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/82335b9e-5584-4afc-b1fc-cc003dbcad4b/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:216 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/256bbfe43b8fe4de7449f137b92cb1b8759725d201896ca0c5d890f4850969c5/userdata/shm DeviceMajor:0 DeviceMinor:290 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/d09454b6-6ac7-46dc-83ee-a3c60dfdf262/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:56 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7bfde12401734403cb360b41d973d858fe8124d5920b6c4f2023738afd03bf5d/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-274 DeviceMajor:0 DeviceMinor:274 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0bdedda0-279a-41fd-a7cc-300c816985df/volumes/kubernetes.io~projected/kube-api-access-2nh4k DeviceMajor:0 DeviceMinor:59 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/ee3cc716-4716-4207-a493-9eeee140a6fe/volumes/kubernetes.io~projected/kube-api-access-tw4b5 DeviceMajor:0 DeviceMinor:60 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/d09454b6-6ac7-46dc-83ee-a3c60dfdf262/volumes/kubernetes.io~projected/kube-api-access-zdwk6 DeviceMajor:0 DeviceMinor:63 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406003712 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5fcb2a03ebf60efd422b67e506e0739906758c878f7bf46ee7ce4f792bd0a10b/userdata/shm DeviceMajor:0 DeviceMinor:70 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fa241532a7ae7a09f847da65293301bad95da33ebfb7b6ff170ce467ce27815a/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/48c05e4f-a673-4f5f-8c21-8dc1ff716683/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:197 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/bce36747-2159-4b72-8c0e-d98226588d18/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:45 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/914e556f074a17e41fd70606c911cc4933035569ca01d47ea3cdf87f5b697df1/userdata/shm DeviceMajor:0 DeviceMinor:229 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-278 DeviceMajor:0 DeviceMinor:278 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-335 DeviceMajor:0 DeviceMinor:335 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/273a50863469fb5a4e75ef299b4cdbe92adb5a2508cc1fa9e2e393bd457b5a4d/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5c9e60c7c7a0e875b7ac9eca1c09f7ae1992f2153833836e1d22d79bf5e54c8e/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/b91ced44-8be6-47c8-9229-549400076eb7/volumes/kubernetes.io~projected/kube-api-access-hvgdb DeviceMajor:0 DeviceMinor:206 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/48c05e4f-a673-4f5f-8c21-8dc1ff716683/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:207 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/82335b9e-5584-4afc-b1fc-cc003dbcad4b/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:215 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-149 DeviceMajor:0 DeviceMinor:149 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a34c6abf742f8f286b936d09b642c7ad10b290cc9e7f39dfbfc46d59a9b0e9b6/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/037937cc-75c9-4416-bd8f-63589df05625/volumes/kubernetes.io~projected/kube-api-access-w4qrj DeviceMajor:0 DeviceMinor:61 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fb78a5ab-3a2d-45fa-abaa-7b26c33ec929/volumes/kubernetes.io~projected/kube-api-access-dhm7l DeviceMajor:0 DeviceMinor:205 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bca57b561e663cb1f3a26e52878ee097de3b83d75b9c9b60932ccbfe1bb2b4e1/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-329 DeviceMajor:0 DeviceMinor:329 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-105 DeviceMajor:0 DeviceMinor:105 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/df7d0108-685b-4c27-8d0f-bccf4ca5dbbe/volumes/kubernetes.io~projected/kube-api-access-z9hd4 DeviceMajor:0 DeviceMinor:203 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/36a4e628-5f12-4849-9d77-deaa3e93b190/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:282 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7fb489a8-8a4f-45f2-977b-594b6126281b/volumes/kubernetes.io~projected/kube-api-access-hlhpw DeviceMajor:0 DeviceMinor:225 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/a01a018f-2c6e-413f-b88f-37b4e91699b1/volumes/kubernetes.io~projected/kube-api-access-v6b98 DeviceMajor:0 DeviceMinor:238 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/57fa3738-c3e8-4c18-9582-f6e8377a041e/volumes/kubernetes.io~projected/kube-api-access-p9lkh DeviceMajor:0 DeviceMinor:239 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cb074bfe629a379116a5c363d223c78cd86d750828f0543b494ad362282da496/userdata/shm DeviceMajor:0 DeviceMinor:287 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/878896d2-fa82-46ef-af5e-0bf491216b03/volumes/kubernetes.io~projected/kube-api-access-qjg2z DeviceMajor:0 DeviceMinor:204 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a5b30117-7cb8-43ec-92a8-4e326b48c22d/volumes/kubernetes.io~projected/kube-api-access-bzgwz DeviceMajor:0 DeviceMinor:289 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47/volumes/kubernetes.io~projected/kube-api-access-h6qrt DeviceMajor:0 DeviceMinor:220 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d017aa4053940b857f08931573d7b582976c3cdb191c0ab67f7b8c6858b5c33f/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-327 DeviceMajor:0 DeviceMinor:327 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:256bbfe43b8fe4d MacAddress:c2:01:25:06:fd:bf Speed:10000 Mtu:8901} {Name:410d7603e73aa5d MacAddress:1e:cb:34:6d:ad:43 Speed:10000 Mtu:8901} {Name:7bfde1240173440 MacAddress:0a:a4:a9:e9:2d:67 Speed:10000 Mtu:8901} {Name:89381bfb4116e17 MacAddress:de:89:1a:0f:cc:6d Speed:10000 Mtu:8901} {Name:914e556f074a17e MacAddress:d2:05:56:d2:82:33 Speed:10000 Mtu:8901} {Name:996b756136bcfaa MacAddress:5a:5e:83:99:21:c4 Speed:10000 Mtu:8901} {Name:bca57b561e663cb MacAddress:96:e1:6e:11:45:62 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:11:14:e0:c2:03 Speed:0 Mtu:9001} {Name:br-int MacAddress:0e:cf:94:41:95:25 Speed:0 Mtu:8901} {Name:cb074bfe629a379 MacAddress:ee:dd:85:29:4d:78 Speed:10000 Mtu:8901} {Name:d017aa4053940b8 MacAddress:ba:54:0d:da:2e:f2 Speed:10000 Mtu:8901} {Name:de33dda5ec106d5 MacAddress:b2:0b:1c:04:ef:89 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:11:14:e0:c2:03 Speed:0 Mtu:9001} {Name:f3fd3df15230b32 MacAddress:d2:d8:1c:24:b7:fc Speed:10000 Mtu:8901} {Name:fc7aa10b2d58b36 MacAddress:da:0f:69:85:04:a4 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:a2:23:c6:0f:c8:61 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:5e:ba:41:8e:09:32 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812011520 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 01 05:03:03.186389 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.186206    4864 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 01 05:03:03.186389 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.186249    4864 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 01 05:03:03.186583 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.186553    4864 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 01 05:03:03.186707 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.186585    4864 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-253.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 01 05:03:03.186752 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.186710    4864 topology_manager.go:143] "Creating topology manager with none policy"
Nov 01 05:03:03.186752 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.186718    4864 container_manager_linux.go:311] "Creating device plugin manager"
Nov 01 05:03:03.186752 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.186728    4864 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 01 05:03:03.186752 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.186748    4864 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 01 05:03:03.186882 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.186862    4864 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 01 05:03:03.187027 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.187017    4864 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 01 05:03:03.187137 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.187128    4864 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 01 05:03:03.187169 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.187164    4864 kubelet.go:493] "Attempting to sync node with API server"
Nov 01 05:03:03.187196 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.187177    4864 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 01 05:03:03.187196 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.187186    4864 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 01 05:03:03.187196 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.187195    4864 kubelet.go:405] "Adding apiserver pod source"
Nov 01 05:03:03.187296 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.187203    4864 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 01 05:03:03.187909 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.187895    4864 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 01 05:03:03.187909 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.187908    4864 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 01 05:03:03.188366 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.188351    4864 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 01 05:03:03.190433 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.190418    4864 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 01 05:03:03.190433 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.190432    4864 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 01 05:03:03.190556 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.190546    4864 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 01 05:03:03.190745 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.190735    4864 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 01 05:03:03.190776 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.190765    4864 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 01 05:03:03.190922 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.190908    4864 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 01 05:03:03.190957 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.190936    4864 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 01 05:03:03.190957 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.190951    4864 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 01 05:03:03.191011 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.190963    4864 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 01 05:03:03.191011 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.190976    4864 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 01 05:03:03.191011 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.190990    4864 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 01 05:03:03.191011 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.191002    4864 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 01 05:03:03.191125 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.191014    4864 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 01 05:03:03.191125 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.191029    4864 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 01 05:03:03.191125 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.191042    4864 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 01 05:03:03.191125 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.191059    4864 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 01 05:03:03.191125 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.191077    4864 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 01 05:03:03.191125 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.191123    4864 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 01 05:03:03.191369 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.191136    4864 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 01 05:03:03.194626 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.194613    4864 server.go:1263] "Started kubelet"
Nov 01 05:03:03.194722 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.194695    4864 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 01 05:03:03.194801 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.194710    4864 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 01 05:03:03.194873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.194858    4864 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 01 05:03:03.195172 ip-10-0-134-253 systemd[1]: Started Kubernetes Kubelet.
Nov 01 05:03:03.195383 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.195261    4864 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 01 05:03:03.196143 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.196127    4864 server.go:317] "Adding debug handlers to kubelet server"
Nov 01 05:03:03.205456 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.205437    4864 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 01 05:03:03.205579 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.205528    4864 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 01 05:03:03.205810 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.205703    4864 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-31 04:57:23 +0000 UTC" deadline="2028-05-05 20:59:00.496784039 +0000 UTC"
Nov 01 05:03:03.205810 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.205733    4864 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13239h55m57.291054388s"
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:03.205938    4864 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-253.us-west-1.compute.internal\" not found"
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.206483    4864 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.206608    4864 factory.go:55] Registering systemd factory
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.206620    4864 factory.go:223] Registration of the systemd container factory successfully
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.206830    4864 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.206946    4864 factory.go:153] Registering CRI-O factory
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.206965    4864 factory.go:223] Registration of the crio container factory successfully
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.206992    4864 factory.go:103] Registering Raw factory
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.207012    4864 manager.go:1199] Started watching for new ooms in manager
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.207468    4864 manager.go:319] Starting recovery of all containers
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.207487    4864 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.207507    4864 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210366    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="45a40b3e-ab72-4b18-933f-f52272a6742c" volumeName="kubernetes.io/configmap/45a40b3e-ab72-4b18-933f-f52272a6742c-konnectivity-ca" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210403    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62652d56-1be6-47cc-a774-77d4fa9d206a" volumeName="kubernetes.io/empty-dir/62652d56-1be6-47cc-a774-77d4fa9d206a-ca-trust-extracted" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210419    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62652d56-1be6-47cc-a774-77d4fa9d206a" volumeName="kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-bound-sa-token" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210463    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62652d56-1be6-47cc-a774-77d4fa9d206a" volumeName="kubernetes.io/secret/62652d56-1be6-47cc-a774-77d4fa9d206a-installation-pull-secrets" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210477    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ee3cc716-4716-4207-a493-9eeee140a6fe" volumeName="kubernetes.io/configmap/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-daemon-config" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210496    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="45a40b3e-ab72-4b18-933f-f52272a6742c" volumeName="kubernetes.io/secret/45a40b3e-ab72-4b18-933f-f52272a6742c-agent-certs" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210509    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5dc708b0-6c35-4f79-a415-e7533381b95b" volumeName="kubernetes.io/empty-dir/5dc708b0-6c35-4f79-a415-e7533381b95b-tmp-dir" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210521    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7fb489a8-8a4f-45f2-977b-594b6126281b" volumeName="kubernetes.io/projected/7fb489a8-8a4f-45f2-977b-594b6126281b-kube-api-access-hlhpw" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210544    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ee3cc716-4716-4207-a493-9eeee140a6fe" volumeName="kubernetes.io/configmap/ee3cc716-4716-4207-a493-9eeee140a6fe-cni-binary-copy" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210557    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="037937cc-75c9-4416-bd8f-63589df05625" volumeName="kubernetes.io/configmap/037937cc-75c9-4416-bd8f-63589df05625-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210569    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1967227b-a4ea-4ee6-945d-efb0b5040a13" volumeName="kubernetes.io/projected/1967227b-a4ea-4ee6-945d-efb0b5040a13-kube-api-access-xfldn" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210583    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9eec03-4159-4a5e-a284-174d634ae0c9" volumeName="kubernetes.io/empty-dir/7c9eec03-4159-4a5e-a284-174d634ae0c9-data-volume" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210596    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="82335b9e-5584-4afc-b1fc-cc003dbcad4b" volumeName="kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-stats-auth" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210607    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a5b30117-7cb8-43ec-92a8-4e326b48c22d" volumeName="kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210622    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bce36747-2159-4b72-8c0e-d98226588d18" volumeName="kubernetes.io/secret/bce36747-2159-4b72-8c0e-d98226588d18-serving-cert" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210635    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1967227b-a4ea-4ee6-945d-efb0b5040a13" volumeName="kubernetes.io/empty-dir/1967227b-a4ea-4ee6-945d-efb0b5040a13-tmp" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210646    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="878896d2-fa82-46ef-af5e-0bf491216b03" volumeName="kubernetes.io/configmap/878896d2-fa82-46ef-af5e-0bf491216b03-config" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210662    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="878896d2-fa82-46ef-af5e-0bf491216b03" volumeName="kubernetes.io/projected/878896d2-fa82-46ef-af5e-0bf491216b03-kube-api-access-qjg2z" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210675    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b53e86a9-ca94-4753-b57b-47eb94bbd879" volumeName="kubernetes.io/secret/b53e86a9-ca94-4753-b57b-47eb94bbd879-cert" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210688    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bce36747-2159-4b72-8c0e-d98226588d18" volumeName="kubernetes.io/configmap/bce36747-2159-4b72-8c0e-d98226588d18-config" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210701    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="037937cc-75c9-4416-bd8f-63589df05625" volumeName="kubernetes.io/configmap/037937cc-75c9-4416-bd8f-63589df05625-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210717    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1967227b-a4ea-4ee6-945d-efb0b5040a13" volumeName="kubernetes.io/configmap/1967227b-a4ea-4ee6-945d-efb0b5040a13-service-ca-bundle" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210730    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62652d56-1be6-47cc-a774-77d4fa9d206a" volumeName="kubernetes.io/configmap/62652d56-1be6-47cc-a774-77d4fa9d206a-trusted-ca" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210744    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b91ced44-8be6-47c8-9229-549400076eb7" volumeName="kubernetes.io/projected/b91ced44-8be6-47c8-9229-549400076eb7-kube-api-access-hvgdb" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210758    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47" volumeName="kubernetes.io/projected/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-kube-api-access-h6qrt" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210807    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5dc708b0-6c35-4f79-a415-e7533381b95b" volumeName="kubernetes.io/configmap/5dc708b0-6c35-4f79-a415-e7533381b95b-config-volume" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210835    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6a747e88-6f89-4f3a-ba71-c97babf998f6" volumeName="kubernetes.io/projected/6a747e88-6f89-4f3a-ba71-c97babf998f6-kube-api-access-njxgf" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210848    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b53e86a9-ca94-4753-b57b-47eb94bbd879" volumeName="kubernetes.io/projected/b53e86a9-ca94-4753-b57b-47eb94bbd879-kube-api-access-bj67m" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210865    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bdedda0-279a-41fd-a7cc-300c816985df" volumeName="kubernetes.io/secret/0bdedda0-279a-41fd-a7cc-300c816985df-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210879    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5dc708b0-6c35-4f79-a415-e7533381b95b" volumeName="kubernetes.io/projected/5dc708b0-6c35-4f79-a415-e7533381b95b-kube-api-access-wblsz" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210891    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="878896d2-fa82-46ef-af5e-0bf491216b03" volumeName="kubernetes.io/secret/878896d2-fa82-46ef-af5e-0bf491216b03-serving-cert" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210904    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bce36747-2159-4b72-8c0e-d98226588d18" volumeName="kubernetes.io/projected/bce36747-2159-4b72-8c0e-d98226588d18-kube-api-access-hkbbr" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210915    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb78a5ab-3a2d-45fa-abaa-7b26c33ec929" volumeName="kubernetes.io/projected/fb78a5ab-3a2d-45fa-abaa-7b26c33ec929-kube-api-access-dhm7l" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210929    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a01a018f-2c6e-413f-b88f-37b4e91699b1" volumeName="kubernetes.io/projected/a01a018f-2c6e-413f-b88f-37b4e91699b1-kube-api-access-v6b98" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210944    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bdedda0-279a-41fd-a7cc-300c816985df" volumeName="kubernetes.io/configmap/0bdedda0-279a-41fd-a7cc-300c816985df-ovnkube-script-lib" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210961    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="48c05e4f-a673-4f5f-8c21-8dc1ff716683" volumeName="kubernetes.io/empty-dir/48c05e4f-a673-4f5f-8c21-8dc1ff716683-ca-trust-extracted" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210973    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="48c05e4f-a673-4f5f-8c21-8dc1ff716683" volumeName="kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-bound-sa-token" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210986    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="48c05e4f-a673-4f5f-8c21-8dc1ff716683" volumeName="kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-kube-api-access-rdspf" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.210999    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62652d56-1be6-47cc-a774-77d4fa9d206a" volumeName="kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-kube-api-access-9n4xr" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211012    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9eec03-4159-4a5e-a284-174d634ae0c9" volumeName="kubernetes.io/projected/7c9eec03-4159-4a5e-a284-174d634ae0c9-kube-api-access-rzrjv" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211024    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bdedda0-279a-41fd-a7cc-300c816985df" volumeName="kubernetes.io/projected/0bdedda0-279a-41fd-a7cc-300c816985df-kube-api-access-2nh4k" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211038    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="878896d2-fa82-46ef-af5e-0bf491216b03" volumeName="kubernetes.io/configmap/878896d2-fa82-46ef-af5e-0bf491216b03-trusted-ca" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211050    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb889bee-f31e-47f1-8617-6a27bc1eebf4" volumeName="kubernetes.io/projected/bb889bee-f31e-47f1-8617-6a27bc1eebf4-kube-api-access-bxknn" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211063    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1f4c040-7436-4df1-8462-6a38acee2c5a" volumeName="kubernetes.io/configmap/d1f4c040-7436-4df1-8462-6a38acee2c5a-serviceca" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211076    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="037937cc-75c9-4416-bd8f-63589df05625" volumeName="kubernetes.io/configmap/037937cc-75c9-4416-bd8f-63589df05625-cni-binary-copy" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211088    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1967227b-a4ea-4ee6-945d-efb0b5040a13" volumeName="kubernetes.io/secret/1967227b-a4ea-4ee6-945d-efb0b5040a13-serving-cert" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211104    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="57fa3738-c3e8-4c18-9582-f6e8377a041e" volumeName="kubernetes.io/projected/57fa3738-c3e8-4c18-9582-f6e8377a041e-kube-api-access-p9lkh" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211120    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62652d56-1be6-47cc-a774-77d4fa9d206a" volumeName="kubernetes.io/configmap/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-certificates" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211135    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7fb489a8-8a4f-45f2-977b-594b6126281b" volumeName="kubernetes.io/configmap/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-config" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211150    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="82335b9e-5584-4afc-b1fc-cc003dbcad4b" volumeName="kubernetes.io/projected/82335b9e-5584-4afc-b1fc-cc003dbcad4b-kube-api-access-d8l8g" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211165    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47" volumeName="kubernetes.io/configmap/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-telemetry-config" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211179    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d09454b6-6ac7-46dc-83ee-a3c60dfdf262" volumeName="kubernetes.io/empty-dir/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-tuned" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211192    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bdedda0-279a-41fd-a7cc-300c816985df" volumeName="kubernetes.io/configmap/0bdedda0-279a-41fd-a7cc-300c816985df-env-overrides" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211210    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="48c05e4f-a673-4f5f-8c21-8dc1ff716683" volumeName="kubernetes.io/configmap/48c05e4f-a673-4f5f-8c21-8dc1ff716683-trusted-ca" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211227    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="48c05e4f-a673-4f5f-8c21-8dc1ff716683" volumeName="kubernetes.io/configmap/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-certificates" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211240    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="48c05e4f-a673-4f5f-8c21-8dc1ff716683" volumeName="kubernetes.io/secret/48c05e4f-a673-4f5f-8c21-8dc1ff716683-image-registry-private-configuration" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211261    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ee3cc716-4716-4207-a493-9eeee140a6fe" volumeName="kubernetes.io/projected/ee3cc716-4716-4207-a493-9eeee140a6fe-kube-api-access-tw4b5" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211273    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef46e568-5c98-4520-99e5-c49c4c3aca2a" volumeName="kubernetes.io/projected/ef46e568-5c98-4520-99e5-c49c4c3aca2a-kube-api-access-2j8z2" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211287    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0197ee7e-0d4a-4e9c-8057-26356d6f34b3" volumeName="kubernetes.io/empty-dir/0197ee7e-0d4a-4e9c-8057-26356d6f34b3-tmp-dir" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211300    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="037937cc-75c9-4416-bd8f-63589df05625" volumeName="kubernetes.io/projected/037937cc-75c9-4416-bd8f-63589df05625-kube-api-access-w4qrj" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211312    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bdedda0-279a-41fd-a7cc-300c816985df" volumeName="kubernetes.io/configmap/0bdedda0-279a-41fd-a7cc-300c816985df-ovnkube-config" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211325    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="636cccf5-2397-4dca-899a-3598a7b8096b" volumeName="kubernetes.io/projected/636cccf5-2397-4dca-899a-3598a7b8096b-kube-api-access-5pj75" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211338    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="636cccf5-2397-4dca-899a-3598a7b8096b" volumeName="kubernetes.io/secret/636cccf5-2397-4dca-899a-3598a7b8096b-serving-cert" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211352    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="82335b9e-5584-4afc-b1fc-cc003dbcad4b" volumeName="kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-default-certificate" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211367    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0197ee7e-0d4a-4e9c-8057-26356d6f34b3" volumeName="kubernetes.io/projected/0197ee7e-0d4a-4e9c-8057-26356d6f34b3-kube-api-access-xmbj9" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211382    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="36a4e628-5f12-4849-9d77-deaa3e93b190" volumeName="kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211396    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d09454b6-6ac7-46dc-83ee-a3c60dfdf262" volumeName="kubernetes.io/empty-dir/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-tmp" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211408    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1967227b-a4ea-4ee6-945d-efb0b5040a13" volumeName="kubernetes.io/empty-dir/1967227b-a4ea-4ee6-945d-efb0b5040a13-snapshots" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211421    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="48c05e4f-a673-4f5f-8c21-8dc1ff716683" volumeName="kubernetes.io/secret/48c05e4f-a673-4f5f-8c21-8dc1ff716683-installation-pull-secrets" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211434    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="636cccf5-2397-4dca-899a-3598a7b8096b" volumeName="kubernetes.io/configmap/636cccf5-2397-4dca-899a-3598a7b8096b-config" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211447    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="df7d0108-685b-4c27-8d0f-bccf4ca5dbbe" volumeName="kubernetes.io/projected/df7d0108-685b-4c27-8d0f-bccf4ca5dbbe-kube-api-access-z9hd4" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211459    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1967227b-a4ea-4ee6-945d-efb0b5040a13" volumeName="kubernetes.io/configmap/1967227b-a4ea-4ee6-945d-efb0b5040a13-trusted-ca-bundle" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211472    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62652d56-1be6-47cc-a774-77d4fa9d206a" volumeName="kubernetes.io/secret/62652d56-1be6-47cc-a774-77d4fa9d206a-image-registry-private-configuration" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211484    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb889bee-f31e-47f1-8617-6a27bc1eebf4" volumeName="kubernetes.io/configmap/bb889bee-f31e-47f1-8617-6a27bc1eebf4-iptables-alerter-script" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211498    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d09454b6-6ac7-46dc-83ee-a3c60dfdf262" volumeName="kubernetes.io/projected/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-kube-api-access-zdwk6" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211516    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1f4c040-7436-4df1-8462-6a38acee2c5a" volumeName="kubernetes.io/projected/d1f4c040-7436-4df1-8462-6a38acee2c5a-kube-api-access-h6vvf" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211530    4864 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6185c12-8b19-47d8-b0b3-01aa0f1192a7" volumeName="kubernetes.io/configmap/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-nginx-conf" seLinuxMountContext=""
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211547    4864 reconstruct.go:97] "Volume reconstruction finished"
Nov 01 05:03:03.212805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.211555    4864 reconciler.go:29] "Reconciler: start to sync state"
Nov 01 05:03:03.222580 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.222556    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/1.log"
Nov 01 05:03:03.223045 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.223027    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/1.log"
Nov 01 05:03:03.231852 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.231817    4864 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 01 05:03:03.239639 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.239610    4864 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 01 05:03:03.239639 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.239637    4864 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 01 05:03:03.239877 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.239658    4864 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 01 05:03:03.239877 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:03.239700    4864 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 01 05:03:03.249850 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.249810    4864 generic.go:356] "Generic (PLEG): container finished" podID="f72249826b80cce6b307e39e964ca081" containerID="15a12c78f0fbd68eec29581885426179bf7003ab69bf4e55a0c903c0d34c6249" exitCode=0
Nov 01 05:03:03.257587 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.257560    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/1.log"
Nov 01 05:03:03.257711 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.257595    4864 generic.go:356] "Generic (PLEG): container finished" podID="878896d2-fa82-46ef-af5e-0bf491216b03" containerID="2bfdcabf49fcf5bed5a8daab44aa1f9b67a4866acfc31c80777617c2ac716e5f" exitCode=255
Nov 01 05:03:03.278478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.278446    4864 generic.go:356] "Generic (PLEG): container finished" podID="037937cc-75c9-4416-bd8f-63589df05625" containerID="9c39cf5c3fd64c0d3a409845499f530bcbc26eeb6b80ff85a77a1576f14136d7" exitCode=0
Nov 01 05:03:03.278478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.278474    4864 generic.go:356] "Generic (PLEG): container finished" podID="037937cc-75c9-4416-bd8f-63589df05625" containerID="743ede4a72caa98cdee31e3610e9fa408b72535a4e830add8d8a0c61c4b17333" exitCode=0
Nov 01 05:03:03.278478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.278481    4864 generic.go:356] "Generic (PLEG): container finished" podID="037937cc-75c9-4416-bd8f-63589df05625" containerID="82c0241cd126dd94e764d8579f64fc37c278283e9297961f82a58be0f0db15b4" exitCode=0
Nov 01 05:03:03.278766 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.278487    4864 generic.go:356] "Generic (PLEG): container finished" podID="037937cc-75c9-4416-bd8f-63589df05625" containerID="830433732fb06158fc96420a38f6905d39231e1fe4d7f66adb013cb454906191" exitCode=0
Nov 01 05:03:03.278766 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.278494    4864 generic.go:356] "Generic (PLEG): container finished" podID="037937cc-75c9-4416-bd8f-63589df05625" containerID="17a7da1fd3d42b0cfc389fc7eca716b4f6a24b066b4eeef259416e1e97b82928" exitCode=0
Nov 01 05:03:03.278766 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.278502    4864 generic.go:356] "Generic (PLEG): container finished" podID="037937cc-75c9-4416-bd8f-63589df05625" containerID="cc33c90d2bcb9d8ac3e212ae4edb2dd664a5216ff84b08c87c28b8ed90772039" exitCode=0
Nov 01 05:03:03.301659 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.301598    4864 manager.go:324] Recovery completed
Nov 01 05:03:03.306039 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:03.306020    4864 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-253.us-west-1.compute.internal\" not found"
Nov 01 05:03:03.318343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.318323    4864 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 05:03:03.319584 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.319564    4864 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 05:03:03.319661 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.319605    4864 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 05:03:03.319661 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.319621    4864 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 05:03:03.324377 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.324354    4864 cpu_manager.go:226] "Starting" policy="none"
Nov 01 05:03:03.324377 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.324381    4864 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 01 05:03:03.324533 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.324384    4864 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 05:03:03.324533 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.324403    4864 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 01 05:03:03.324855 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.324836    4864 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 01 05:03:03.324921 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.324898    4864 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 01 05:03:03.324959 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.324920    4864 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 01 05:03:03.324959 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.324926    4864 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 01 05:03:03.324959 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.324933    4864 policy_none.go:50] "Start"
Nov 01 05:03:03.324959 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.324942    4864 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 01 05:03:03.324959 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.324956    4864 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 01 05:03:03.325657 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.325632    4864 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 05:03:03.325736 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.325675    4864 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 05:03:03.325736 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.325693    4864 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 05:03:03.339694 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.339660    4864 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 01 05:03:03.339694 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.339692    4864 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 01 05:03:03.339694 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.339700    4864 policy_none.go:44] "Start"
Nov 01 05:03:03.339919 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:03.339833    4864 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Nov 01 05:03:03.344199 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.344166    4864 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 01 05:03:03.346813 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.346799    4864 manager.go:341] "Starting Device Plugin manager"
Nov 01 05:03:03.346970 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:03.346830    4864 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 01 05:03:03.346970 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.346840    4864 server.go:87] "Starting device plugin registration server"
Nov 01 05:03:03.347168 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.347155    4864 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 01 05:03:03.347219 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.347170    4864 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 01 05:03:03.347314 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.347294    4864 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 01 05:03:03.347436 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.347402    4864 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 01 05:03:03.347483 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.347442    4864 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 01 05:03:03.347483 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.347448    4864 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 01 05:03:03.347728 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.347503    4864 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-01T05:03:03.347431394Z","UUID":"d6d1c18d-162e-4061-a78f-c23cec0ddf5f","Handler":null,"Name":"","Endpoint":""}
Nov 01 05:03:03.348941 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:03.348919    4864 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 01 05:03:03.349056 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:03.348983    4864 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-253.us-west-1.compute.internal\" not found"
Nov 01 05:03:03.349741 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.349720    4864 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 01 05:03:03.349841 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.349771    4864 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 01 05:03:03.398155 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.398118    4864 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 05:03:03.398321 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.398175    4864 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 05:03:03.398366 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.398319    4864 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 01 05:03:03.398863 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.398849    4864 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 05:03:03.447394 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.447368    4864 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 05:03:03.448529 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.448501    4864 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 05:03:03.448661 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.448541    4864 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 05:03:03.448661 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.448560    4864 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-253.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 05:03:03.448661 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.448612    4864 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 05:03:03.448661 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.448618    4864 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:03:03.540987 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.540915    4864 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal"]
Nov 01 05:03:03.541616 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.541572    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal" event={"ID":"f72249826b80cce6b307e39e964ca081","Type":"ContainerStarted","Data":"fce34e94868ccc120ea970a54bf3762aec3b5601f8d380c8506b8addd40d7584"}
Nov 01 05:03:03.541654 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.541632    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal" event={"ID":"f72249826b80cce6b307e39e964ca081","Type":"ContainerDied","Data":"15a12c78f0fbd68eec29581885426179bf7003ab69bf4e55a0c903c0d34c6249"}
Nov 01 05:03:03.541683 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.541654    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal" event={"ID":"f72249826b80cce6b307e39e964ca081","Type":"ContainerStarted","Data":"3c1331112ec45d56fd63f15fd4809c720cc0ee0bd39f24754eef6fe229b37653"}
Nov 01 05:03:03.541683 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.541669    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal" event={"ID":"1c1427a4a27caa7c2e9df3986def0a2a","Type":"ContainerStarted","Data":"5c64cedd44856c8229fa05011eb151f97f1bee4f59accbad7f01e94c871d24bd"}
Nov 01 05:03:03.541683 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.541681    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal" event={"ID":"1c1427a4a27caa7c2e9df3986def0a2a","Type":"ContainerStarted","Data":"a34c6abf742f8f286b936d09b642c7ad10b290cc9e7f39dfbfc46d59a9b0e9b6"}
Nov 01 05:03:03.541833 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.541775    4864 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:03:03.541871 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.541858    4864 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:03:03.557873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.557849    4864 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:03:03.558007 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.557914    4864 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:03:03.604039 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.604009    4864 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 01 05:03:03.604186 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:03.604072    4864 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:03:03.604317 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.604302    4864 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 01 05:03:03.604371 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:03.604340    4864 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:03:03.700183 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.700146    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c1427a4a27caa7c2e9df3986def0a2a-config\") pod \"kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal\" (UID: \"1c1427a4a27caa7c2e9df3986def0a2a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:03:03.700183 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.700186    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f72249826b80cce6b307e39e964ca081-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal\" (UID: \"f72249826b80cce6b307e39e964ca081\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:03:03.700387 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.700203    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f72249826b80cce6b307e39e964ca081-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal\" (UID: \"f72249826b80cce6b307e39e964ca081\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:03:03.800850 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.800809    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f72249826b80cce6b307e39e964ca081-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal\" (UID: \"f72249826b80cce6b307e39e964ca081\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:03:03.800850 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.800825    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f72249826b80cce6b307e39e964ca081-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal\" (UID: \"f72249826b80cce6b307e39e964ca081\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:03:03.801055 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.800866    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c1427a4a27caa7c2e9df3986def0a2a-config\") pod \"kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal\" (UID: \"1c1427a4a27caa7c2e9df3986def0a2a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:03:03.801055 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.800900    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f72249826b80cce6b307e39e964ca081-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal\" (UID: \"f72249826b80cce6b307e39e964ca081\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:03:03.801055 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.800905    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c1427a4a27caa7c2e9df3986def0a2a-config\") pod \"kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal\" (UID: \"1c1427a4a27caa7c2e9df3986def0a2a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:03:03.801055 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:03.800926    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f72249826b80cce6b307e39e964ca081-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal\" (UID: \"f72249826b80cce6b307e39e964ca081\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:03:04.187565 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.187536    4864 apiserver.go:52] "Watching apiserver"
Nov 01 05:03:04.295337 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.295314    4864 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:03:04.295482 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.295420    4864 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:03:04.339887 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.339862    4864 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 01 05:03:04.340605 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.340579    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-qszhw","openshift-cluster-node-tuning-operator/tuned-m8d7h","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j67hv","kube-system/kas-connection-checker-685895c7c9-qb994","openshift-image-registry/node-ca-m9ckd","openshift-network-operator/iptables-alerter-nk4wf","openshift-ovn-kubernetes/ovnkube-node-89wdz","openshift-multus/multus-additional-cni-plugins-h6ht6","openshift-image-registry/image-registry-6969bc8dd6-85rsc","openshift-insights/insights-operator-7c4986d449-cvn58","openshift-network-console/networking-console-plugin-86c478bb88-ht4j4","openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27","openshift-dns/node-resolver-85452","kube-system/kas-connection-checker-685895c7c9-d2cm4","kube-system/konnectivity-agent-jswq7","openshift-multus/network-metrics-daemon-rvg8t","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal","kube-system/global-pull-secret-syncer-q92dh","openshift-dns/dns-default-wcrwt","openshift-multus/multus-gwbm4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd","openshift-network-diagnostics/network-check-target-tl74d","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz","openshift-ingress/router-default-74dd76c985-n79rt","kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal","openshift-ingress-canary/ingress-canary-n8nnj","openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9","openshift-console-operator/console-operator-7cd56b8dbc-6twtw","openshift-image-registry/image-registry-79659c8779-dnfbf","openshift-insights/insights-runtime-extractor-b6sl7","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk","openshift-network-diagnostics/network-check-source-6f78d59f99-srv7d"]
Nov 01 05:03:04.342489 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.342465    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:03:04.343403 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.343387    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:03:04.344561 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.344541    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:04.345346 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.345329    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:04.345813 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.345774    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"
Nov 01 05:03:04.346936 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.346919    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:03:04.355073 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.355029    4864 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 01 05:03:04.355346 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.355330    4864 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 01 05:03:04.355387 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.355356    4864 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:03:04.355387 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.355378    4864 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-253.us-west-1.compute.internal"
Nov 01 05:03:04.355558 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.355542    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:04.356974 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.356959    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:03:04.357427 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.357411    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:04.393090 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.393069    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 01 05:03:04.393190 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.393140    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 01 05:03:04.393317 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.393300    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 01 05:03:04.394715 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.394701    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 01 05:03:04.395157 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.395142    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 01 05:03:04.395157 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.395152    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 01 05:03:04.395254 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.395223    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 01 05:03:04.395314 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.395268    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 01 05:03:04.395314 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.395289    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 01 05:03:04.396040 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.396017    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 01 05:03:04.396040 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.396037    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 01 05:03:04.396420 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.396402    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 01 05:03:04.396493 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.396461    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 01 05:03:04.396544 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.396489    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 01 05:03:04.396678 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.396664    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 01 05:03:04.396725 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.396689    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 01 05:03:04.396725 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.396699    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 01 05:03:04.396821 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.396759    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 01 05:03:04.396821 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.396762    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 01 05:03:04.396821 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.396770    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 01 05:03:04.396933 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.396828    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 01 05:03:04.396933 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.396839    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 01 05:03:04.396996 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.396936    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 01 05:03:04.397484 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.397465    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 01 05:03:04.397573 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.397500    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 01 05:03:04.397573 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.397509    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 01 05:03:04.398618 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.398600    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-n5nx4\""
Nov 01 05:03:04.398658 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.398588    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 01 05:03:04.398658 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.398648    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 01 05:03:04.398730 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.398659    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 01 05:03:04.398730 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.398671    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-hjhjw\""
Nov 01 05:03:04.409274 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.409258    4864 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 01 05:03:04.442062 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.441997    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 01 05:03:04.442062 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.441997    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 01 05:03:04.442062 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.441998    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 01 05:03:04.442062 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.442035    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 01 05:03:04.442241 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.442078    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-mk4l6\""
Nov 01 05:03:04.442241 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.442086    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-7fb86\""
Nov 01 05:03:04.442446 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.442429    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 01 05:03:04.442515 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.442471    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 01 05:03:04.442585 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.442514    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 01 05:03:04.442585 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.442519    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 01 05:03:04.442585 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.442520    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 01 05:03:04.442585 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.442557    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 01 05:03:04.442585 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.442567    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 01 05:03:04.442819 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.442589    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 01 05:03:04.442819 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.442711    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 01 05:03:04.442819 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.442716    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 01 05:03:04.442819 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.442743    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 01 05:03:04.444746 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.444731    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 01 05:03:04.446017 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446001    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 01 05:03:04.446239 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446222    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 01 05:03:04.446239 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446224    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 01 05:03:04.446377 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446228    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 01 05:03:04.446377 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446249    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 01 05:03:04.446540 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446517    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 01 05:03:04.446646 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446603    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 01 05:03:04.446716 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446603    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 01 05:03:04.446716 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446619    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 01 05:03:04.446716 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446620    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 01 05:03:04.446716 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446620    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 01 05:03:04.446716 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446637    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4vjvv\""
Nov 01 05:03:04.446957 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446728    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6k7br\""
Nov 01 05:03:04.446957 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446747    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 01 05:03:04.446957 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446861    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 01 05:03:04.446957 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446881    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 01 05:03:04.446957 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446899    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 01 05:03:04.446957 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446934    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-7kc5j\""
Nov 01 05:03:04.446957 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446939    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 01 05:03:04.446957 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446946    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 01 05:03:04.446957 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446940    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446982    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.446986    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-b9csn\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447001    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-fhbr8\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447036    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fbhl7\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447040    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-nkwq4\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447055    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447064    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447056    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xvz28\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447073    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447113    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-q6tm4\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447122    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447152    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447161    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447168    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447174    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447179    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-g7ktw\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447195    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447206    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447196    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6ndpw\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447230    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447240    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gnn2d\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447254    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-27c4k\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447258    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447281    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447289    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 01 05:03:04.447343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447358    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 01 05:03:04.448058 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447433    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-tw68j\""
Nov 01 05:03:04.448058 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447463    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-7m49c\""
Nov 01 05:03:04.448058 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.447515    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 01 05:03:04.506650 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.506620    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-etc-kubernetes\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.506650 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.506653    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bb889bee-f31e-47f1-8617-6a27bc1eebf4-iptables-alerter-script\") pod \"iptables-alerter-nk4wf\" (UID: \"bb889bee-f31e-47f1-8617-6a27bc1eebf4\") " pod="openshift-network-operator/iptables-alerter-nk4wf"
Nov 01 05:03:04.506857 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.506677    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wblsz\" (UniqueName: \"kubernetes.io/projected/5dc708b0-6c35-4f79-a415-e7533381b95b-kube-api-access-wblsz\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:03:04.506857 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.506701    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1f4c040-7436-4df1-8462-6a38acee2c5a-host\") pod \"node-ca-m9ckd\" (UID: \"d1f4c040-7436-4df1-8462-6a38acee2c5a\") " pod="openshift-image-registry/node-ca-m9ckd"
Nov 01 05:03:04.506857 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.506742    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9hd4\" (UniqueName: \"kubernetes.io/projected/df7d0108-685b-4c27-8d0f-bccf4ca5dbbe-kube-api-access-z9hd4\") pod \"kas-connection-checker-685895c7c9-qb994\" (UID: \"df7d0108-685b-4c27-8d0f-bccf4ca5dbbe\") " pod="kube-system/kas-connection-checker-685895c7c9-qb994"
Nov 01 05:03:04.506857 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.506827    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:03:04.507009 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.506936    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-socket-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:03:04.507009 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.506983    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nh4k\" (UniqueName: \"kubernetes.io/projected/0bdedda0-279a-41fd-a7cc-300c816985df-kube-api-access-2nh4k\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.507081 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507045    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48c05e4f-a673-4f5f-8c21-8dc1ff716683-image-registry-private-configuration\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:04.507135 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507080    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/62652d56-1be6-47cc-a774-77d4fa9d206a-trusted-ca\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:04.507135 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507110    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-bound-sa-token\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:04.507135 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507125    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bb889bee-f31e-47f1-8617-6a27bc1eebf4-iptables-alerter-script\") pod \"iptables-alerter-nk4wf\" (UID: \"bb889bee-f31e-47f1-8617-6a27bc1eebf4\") " pod="openshift-network-operator/iptables-alerter-nk4wf"
Nov 01 05:03:04.507277 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507136    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjg2z\" (UniqueName: \"kubernetes.io/projected/878896d2-fa82-46ef-af5e-0bf491216b03-kube-api-access-qjg2z\") pod \"console-operator-7cd56b8dbc-6twtw\" (UID: \"878896d2-fa82-46ef-af5e-0bf491216b03\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:03:04.507277 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507163    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1967227b-a4ea-4ee6-945d-efb0b5040a13-snapshots\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:03:04.507277 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507190    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-conf-dir\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.507277 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507213    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-run-ovn\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.507277 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507238    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bce36747-2159-4b72-8c0e-d98226588d18-serving-cert\") pod \"service-ca-operator-544d749c8c-97k27\" (UID: \"bce36747-2159-4b72-8c0e-d98226588d18\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27"
Nov 01 05:03:04.507277 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507255    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1967227b-a4ea-4ee6-945d-efb0b5040a13-snapshots\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:03:04.507520 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507285    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njxgf\" (UniqueName: \"kubernetes.io/projected/6a747e88-6f89-4f3a-ba71-c97babf998f6-kube-api-access-njxgf\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:03:04.507520 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507292    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48c05e4f-a673-4f5f-8c21-8dc1ff716683-image-registry-private-configuration\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:04.507520 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507406    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-bound-sa-token\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:04.507520 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507442    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bce36747-2159-4b72-8c0e-d98226588d18-serving-cert\") pod \"service-ca-operator-544d749c8c-97k27\" (UID: \"bce36747-2159-4b72-8c0e-d98226588d18\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27"
Nov 01 05:03:04.507520 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507482    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/62652d56-1be6-47cc-a774-77d4fa9d206a-trusted-ca\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:04.507520 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507489    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:04.507780 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507561    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7c9eec03-4159-4a5e-a284-174d634ae0c9-crio-socket\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:04.507780 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507612    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-slash\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.507780 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507640    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bdedda0-279a-41fd-a7cc-300c816985df-ovnkube-config\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.507780 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507655    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bdedda0-279a-41fd-a7cc-300c816985df-ovn-node-metrics-cert\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.507780 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507741    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-sysctl-d\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.508005 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507821    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-etc-openvswitch\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.508005 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507854    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmbj9\" (UniqueName: \"kubernetes.io/projected/0197ee7e-0d4a-4e9c-8057-26356d6f34b3-kube-api-access-xmbj9\") pod \"node-resolver-85452\" (UID: \"0197ee7e-0d4a-4e9c-8057-26356d6f34b3\") " pod="openshift-dns/node-resolver-85452"
Nov 01 05:03:04.508005 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507884    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-default-certificate\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:04.508005 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507889    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bdedda0-279a-41fd-a7cc-300c816985df-ovn-node-metrics-cert\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.508005 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507913    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0197ee7e-0d4a-4e9c-8057-26356d6f34b3-tmp-dir\") pod \"node-resolver-85452\" (UID: \"0197ee7e-0d4a-4e9c-8057-26356d6f34b3\") " pod="openshift-dns/node-resolver-85452"
Nov 01 05:03:04.508005 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.507933    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bdedda0-279a-41fd-a7cc-300c816985df-ovnkube-config\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.508178 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508012    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-sys\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.508178 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508031    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:03:04.508178 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508059    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw4b5\" (UniqueName: \"kubernetes.io/projected/ee3cc716-4716-4207-a493-9eeee140a6fe-kube-api-access-tw4b5\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.508178 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508081    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bdedda0-279a-41fd-a7cc-300c816985df-ovnkube-script-lib\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.508178 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508097    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0197ee7e-0d4a-4e9c-8057-26356d6f34b3-hosts-file\") pod \"node-resolver-85452\" (UID: \"0197ee7e-0d4a-4e9c-8057-26356d6f34b3\") " pod="openshift-dns/node-resolver-85452"
Nov 01 05:03:04.508178 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508114    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkbbr\" (UniqueName: \"kubernetes.io/projected/bce36747-2159-4b72-8c0e-d98226588d18-kube-api-access-hkbbr\") pod \"service-ca-operator-544d749c8c-97k27\" (UID: \"bce36747-2159-4b72-8c0e-d98226588d18\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27"
Nov 01 05:03:04.508178 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508135    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-daemon-config\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.508446 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508304    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0197ee7e-0d4a-4e9c-8057-26356d6f34b3-tmp-dir\") pod \"node-resolver-85452\" (UID: \"0197ee7e-0d4a-4e9c-8057-26356d6f34b3\") " pod="openshift-dns/node-resolver-85452"
Nov 01 05:03:04.508446 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508335    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-run-netns\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.508446 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508365    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48c05e4f-a673-4f5f-8c21-8dc1ff716683-ca-trust-extracted\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:04.508592 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508482    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4qrj\" (UniqueName: \"kubernetes.io/projected/037937cc-75c9-4416-bd8f-63589df05625-kube-api-access-w4qrj\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:03:04.508592 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508486    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:03:04.508592 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508498    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-daemon-config\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.508592 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508530    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bdedda0-279a-41fd-a7cc-300c816985df-ovnkube-script-lib\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.508592 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508550    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48c05e4f-a673-4f5f-8c21-8dc1ff716683-ca-trust-extracted\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:04.508753 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508588    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-nginx-conf\") pod \"networking-console-plugin-86c478bb88-ht4j4\" (UID: \"f6185c12-8b19-47d8-b0b3-01aa0f1192a7\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"
Nov 01 05:03:04.508753 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508728    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-system-cni-dir\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.508753 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508712    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-default-certificate\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:04.508866 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508760    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8l8g\" (UniqueName: \"kubernetes.io/projected/82335b9e-5584-4afc-b1fc-cc003dbcad4b-kube-api-access-d8l8g\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:04.508866 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508778    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-run-openvswitch\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.508866 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508821    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bdedda0-279a-41fd-a7cc-300c816985df-env-overrides\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.508866 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508843    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-nginx-conf\") pod \"networking-console-plugin-86c478bb88-ht4j4\" (UID: \"f6185c12-8b19-47d8-b0b3-01aa0f1192a7\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"
Nov 01 05:03:04.508973 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508855    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48c05e4f-a673-4f5f-8c21-8dc1ff716683-installation-pull-secrets\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:04.508973 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508891    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dc708b0-6c35-4f79-a415-e7533381b95b-tmp-dir\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:03:04.508973 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508917    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-var-lib-openvswitch\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.508973 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508960    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/62652d56-1be6-47cc-a774-77d4fa9d206a-ca-trust-extracted\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:04.509093 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508973    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bdedda0-279a-41fd-a7cc-300c816985df-env-overrides\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.509093 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.508988    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj67m\" (UniqueName: \"kubernetes.io/projected/b53e86a9-ca94-4753-b57b-47eb94bbd879-kube-api-access-bj67m\") pod \"ingress-canary-n8nnj\" (UID: \"b53e86a9-ca94-4753-b57b-47eb94bbd879\") " pod="openshift-ingress-canary/ingress-canary-n8nnj"
Nov 01 05:03:04.509093 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.509001    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dc708b0-6c35-4f79-a415-e7533381b95b-tmp-dir\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:03:04.509093 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.509020    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6qrt\" (UniqueName: \"kubernetes.io/projected/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-kube-api-access-h6qrt\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:03:04.509093 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.509046    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/62652d56-1be6-47cc-a774-77d4fa9d206a-ca-trust-extracted\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:04.509093 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.509048    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48c05e4f-a673-4f5f-8c21-8dc1ff716683-installation-pull-secrets\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:04.509093 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.509054    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:04.509093 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.509087    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee3cc716-4716-4207-a493-9eeee140a6fe-cni-binary-copy\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.509310 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.509222    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-certificates\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:04.509310 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.509243    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfldn\" (UniqueName: \"kubernetes.io/projected/1967227b-a4ea-4ee6-945d-efb0b5040a13-kube-api-access-xfldn\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:03:04.509310 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.509260    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-tuned\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.509310 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.509282    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee3cc716-4716-4207-a493-9eeee140a6fe-cni-binary-copy\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.509310 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.509284    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.509459 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.509339    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-certificates\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:04.509669 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.509338    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-tuned\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.509778 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.509755    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n4xr\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-kube-api-access-9n4xr\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:04.509863 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.509827    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/037937cc-75c9-4416-bd8f-63589df05625-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:03:04.509910 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.509864    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhm7l\" (UniqueName: \"kubernetes.io/projected/fb78a5ab-3a2d-45fa-abaa-7b26c33ec929-kube-api-access-dhm7l\") pod \"kas-connection-checker-685895c7c9-d2cm4\" (UID: \"fb78a5ab-3a2d-45fa-abaa-7b26c33ec929\") " pod="kube-system/kas-connection-checker-685895c7c9-d2cm4"
Nov 01 05:03:04.509957 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.509898    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb889bee-f31e-47f1-8617-6a27bc1eebf4-host-slash\") pod \"iptables-alerter-nk4wf\" (UID: \"bb889bee-f31e-47f1-8617-6a27bc1eebf4\") " pod="openshift-network-operator/iptables-alerter-nk4wf"
Nov 01 05:03:04.509957 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.509949    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-lib-modules\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.510098 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510072    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-tmp\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.510232 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510115    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-tmp\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.510232 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510194    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-etc-selinux\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:03:04.510334 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510243    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1967227b-a4ea-4ee6-945d-efb0b5040a13-tmp\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:03:04.510334 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510287    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/878896d2-fa82-46ef-af5e-0bf491216b03-trusted-ca\") pod \"console-operator-7cd56b8dbc-6twtw\" (UID: \"878896d2-fa82-46ef-af5e-0bf491216b03\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:03:04.510435 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510385    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1967227b-a4ea-4ee6-945d-efb0b5040a13-tmp\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:03:04.510509 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510490    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-hostroot\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.510568 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510544    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlhpw\" (UniqueName: \"kubernetes.io/projected/7fb489a8-8a4f-45f2-977b-594b6126281b-kube-api-access-hlhpw\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:03:04.510615 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510577    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs\") pod \"network-metrics-daemon-rvg8t\" (UID: \"ef46e568-5c98-4520-99e5-c49c4c3aca2a\") " pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:03:04.510615 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510601    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:03:04.510750 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510633    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/037937cc-75c9-4416-bd8f-63589df05625-system-cni-dir\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:03:04.510750 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510663    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:04.510750 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510690    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-run-systemd\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.510750 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510721    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/62652d56-1be6-47cc-a774-77d4fa9d206a-installation-pull-secrets\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:04.510750 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510746    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-var-lib-kubelet\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.511058 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510776    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdwk6\" (UniqueName: \"kubernetes.io/projected/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-kube-api-access-zdwk6\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.511058 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510837    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j8z2\" (UniqueName: \"kubernetes.io/projected/ef46e568-5c98-4520-99e5-c49c4c3aca2a-kube-api-access-2j8z2\") pod \"network-metrics-daemon-rvg8t\" (UID: \"ef46e568-5c98-4520-99e5-c49c4c3aca2a\") " pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:03:04.511058 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510866    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1967227b-a4ea-4ee6-945d-efb0b5040a13-serving-cert\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:03:04.511058 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510898    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-run-multus-certs\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.511058 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510923    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-modprobe-d\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.511058 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510951    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-systemd\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.511058 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.510983    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/037937cc-75c9-4416-bd8f-63589df05625-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:03:04.511058 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.511018    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/878896d2-fa82-46ef-af5e-0bf491216b03-serving-cert\") pod \"console-operator-7cd56b8dbc-6twtw\" (UID: \"878896d2-fa82-46ef-af5e-0bf491216b03\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:03:04.511058 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.511046    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-cni-dir\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.511407 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.511078    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxknn\" (UniqueName: \"kubernetes.io/projected/bb889bee-f31e-47f1-8617-6a27bc1eebf4-kube-api-access-bxknn\") pod \"iptables-alerter-nk4wf\" (UID: \"bb889bee-f31e-47f1-8617-6a27bc1eebf4\") " pod="openshift-network-operator/iptables-alerter-nk4wf"
Nov 01 05:03:04.511407 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.511104    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdspf\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-kube-api-access-rdspf\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:04.511407 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.511134    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/636cccf5-2397-4dca-899a-3598a7b8096b-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz\" (UID: \"636cccf5-2397-4dca-899a-3598a7b8096b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz"
Nov 01 05:03:04.511407 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.511166    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/45a40b3e-ab72-4b18-933f-f52272a6742c-agent-certs\") pod \"konnectivity-agent-jswq7\" (UID: \"45a40b3e-ab72-4b18-933f-f52272a6742c\") " pod="kube-system/konnectivity-agent-jswq7"
Nov 01 05:03:04.511407 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.511197    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7c9eec03-4159-4a5e-a284-174d634ae0c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:04.511407 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.511226    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-log-socket\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.511407 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.511251    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-sys-fs\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:03:04.511407 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.511286    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6vvf\" (UniqueName: \"kubernetes.io/projected/d1f4c040-7436-4df1-8462-6a38acee2c5a-kube-api-access-h6vvf\") pod \"node-ca-m9ckd\" (UID: \"d1f4c040-7436-4df1-8462-6a38acee2c5a\") " pod="openshift-image-registry/node-ca-m9ckd"
Nov 01 05:03:04.511407 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.511318    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-sysctl-conf\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.511407 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.511347    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/36a4e628-5f12-4849-9d77-deaa3e93b190-dbus\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:03:04.511407 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.511349    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/878896d2-fa82-46ef-af5e-0bf491216b03-trusted-ca\") pod \"console-operator-7cd56b8dbc-6twtw\" (UID: \"878896d2-fa82-46ef-af5e-0bf491216b03\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:03:04.511407 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.511381    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-certificates\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:04.511918 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.511712    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-cnibin\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.511968 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.511957    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1967227b-a4ea-4ee6-945d-efb0b5040a13-serving-cert\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:03:04.512122 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.512102    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/878896d2-fa82-46ef-af5e-0bf491216b03-serving-cert\") pod \"console-operator-7cd56b8dbc-6twtw\" (UID: \"878896d2-fa82-46ef-af5e-0bf491216b03\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:03:04.512221 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.512201    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/636cccf5-2397-4dca-899a-3598a7b8096b-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz\" (UID: \"636cccf5-2397-4dca-899a-3598a7b8096b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz"
Nov 01 05:03:04.512275 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.512259    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-certificates\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:04.512599 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.512578    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/45a40b3e-ab72-4b18-933f-f52272a6742c-agent-certs\") pod \"konnectivity-agent-jswq7\" (UID: \"45a40b3e-ab72-4b18-933f-f52272a6742c\") " pod="kube-system/konnectivity-agent-jswq7"
Nov 01 05:03:04.512654 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.512611    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/037937cc-75c9-4416-bd8f-63589df05625-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:03:04.514002 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.512816    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-var-lib-cni-multus\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.514002 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.512857    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5dc708b0-6c35-4f79-a415-e7533381b95b-config-volume\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:03:04.514002 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.512916    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzgwz\" (UniqueName: \"kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz\") pod \"network-check-target-tl74d\" (UID: \"a5b30117-7cb8-43ec-92a8-4e326b48c22d\") " pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:03:04.514002 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.512957    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:03:04.514002 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.513014    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-run\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.514002 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.513016    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/62652d56-1be6-47cc-a774-77d4fa9d206a-installation-pull-secrets\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:04.514002 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.513047    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:03:04.514002 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.513085    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5dc708b0-6c35-4f79-a415-e7533381b95b-config-volume\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:03:04.514002 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.513085    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/037937cc-75c9-4416-bd8f-63589df05625-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:03:04.514002 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.513341    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:03:04.514002 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.513583    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/36a4e628-5f12-4849-9d77-deaa3e93b190-original-pull-secret\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:03:04.514002 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.513640    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/878896d2-fa82-46ef-af5e-0bf491216b03-config\") pod \"console-operator-7cd56b8dbc-6twtw\" (UID: \"878896d2-fa82-46ef-af5e-0bf491216b03\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:03:04.514002 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.513657    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/037937cc-75c9-4416-bd8f-63589df05625-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:03:04.514002 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.513700    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9lkh\" (UniqueName: \"kubernetes.io/projected/57fa3738-c3e8-4c18-9582-f6e8377a041e-kube-api-access-p9lkh\") pod \"network-check-source-6f78d59f99-srv7d\" (UID: \"57fa3738-c3e8-4c18-9582-f6e8377a041e\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-srv7d"
Nov 01 05:03:04.514002 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.513883    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/878896d2-fa82-46ef-af5e-0bf491216b03-config\") pod \"console-operator-7cd56b8dbc-6twtw\" (UID: \"878896d2-fa82-46ef-af5e-0bf491216b03\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:03:04.514002 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.513930    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pj75\" (UniqueName: \"kubernetes.io/projected/636cccf5-2397-4dca-899a-3598a7b8096b-kube-api-access-5pj75\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz\" (UID: \"636cccf5-2397-4dca-899a-3598a7b8096b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz"
Nov 01 05:03:04.514002 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.513965    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d1f4c040-7436-4df1-8462-6a38acee2c5a-serviceca\") pod \"node-ca-m9ckd\" (UID: \"d1f4c040-7436-4df1-8462-6a38acee2c5a\") " pod="openshift-image-registry/node-ca-m9ckd"
Nov 01 05:03:04.514665 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.514016    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-ht4j4\" (UID: \"f6185c12-8b19-47d8-b0b3-01aa0f1192a7\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"
Nov 01 05:03:04.514665 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.514163    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c9eec03-4159-4a5e-a284-174d634ae0c9-data-volume\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:04.514665 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.514223    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-kubelet\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.514665 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.514256    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d1f4c040-7436-4df1-8462-6a38acee2c5a-serviceca\") pod \"node-ca-m9ckd\" (UID: \"d1f4c040-7436-4df1-8462-6a38acee2c5a\") " pod="openshift-image-registry/node-ca-m9ckd"
Nov 01 05:03:04.514665 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.514256    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-run-ovn-kubernetes\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.514665 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.514333    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c9eec03-4159-4a5e-a284-174d634ae0c9-data-volume\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:04.514665 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.514335    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-cni-bin\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.514665 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.514407    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bce36747-2159-4b72-8c0e-d98226588d18-config\") pod \"service-ca-operator-544d749c8c-97k27\" (UID: \"bce36747-2159-4b72-8c0e-d98226588d18\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27"
Nov 01 05:03:04.515000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.514772    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-host\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.515000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.514844    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-device-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:03:04.515000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.514854    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bce36747-2159-4b72-8c0e-d98226588d18-config\") pod \"service-ca-operator-544d749c8c-97k27\" (UID: \"bce36747-2159-4b72-8c0e-d98226588d18\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27"
Nov 01 05:03:04.515000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.514894    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-var-lib-cni-bin\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.515000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.514927    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-cni-netd\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.515000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.514956    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1967227b-a4ea-4ee6-945d-efb0b5040a13-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:03:04.515000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.514995    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-socket-dir-parent\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.515316 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515028    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzrjv\" (UniqueName: \"kubernetes.io/projected/7c9eec03-4159-4a5e-a284-174d634ae0c9-kube-api-access-rzrjv\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:04.515316 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515095    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48c05e4f-a673-4f5f-8c21-8dc1ff716683-trusted-ca\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:04.515316 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515141    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/62652d56-1be6-47cc-a774-77d4fa9d206a-image-registry-private-configuration\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:04.515316 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515209    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b53e86a9-ca94-4753-b57b-47eb94bbd879-cert\") pod \"ingress-canary-n8nnj\" (UID: \"b53e86a9-ca94-4753-b57b-47eb94bbd879\") " pod="openshift-ingress-canary/ingress-canary-n8nnj"
Nov 01 05:03:04.515316 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515309    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1967227b-a4ea-4ee6-945d-efb0b5040a13-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:03:04.515557 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515333    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-run-k8s-cni-cncf-io\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.515557 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515357    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/62652d56-1be6-47cc-a774-77d4fa9d206a-image-registry-private-configuration\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:04.515557 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515390    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/636cccf5-2397-4dca-899a-3598a7b8096b-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz\" (UID: \"636cccf5-2397-4dca-899a-3598a7b8096b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz"
Nov 01 05:03:04.515557 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515421    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-systemd-units\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.515557 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515472    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-node-log\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.515557 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515485    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b53e86a9-ca94-4753-b57b-47eb94bbd879-cert\") pod \"ingress-canary-n8nnj\" (UID: \"b53e86a9-ca94-4753-b57b-47eb94bbd879\") " pod="openshift-ingress-canary/ingress-canary-n8nnj"
Nov 01 05:03:04.515557 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515511    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/636cccf5-2397-4dca-899a-3598a7b8096b-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz\" (UID: \"636cccf5-2397-4dca-899a-3598a7b8096b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz"
Nov 01 05:03:04.515557 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515513    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:03:04.515557 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515543    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:03:04.515557 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515544    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48c05e4f-a673-4f5f-8c21-8dc1ff716683-trusted-ca\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:04.515883 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515566    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-sysconfig\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.515883 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515589    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-kubernetes\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.515883 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515606    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-registration-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:03:04.515883 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515629    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/037937cc-75c9-4416-bd8f-63589df05625-cni-binary-copy\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:03:04.515883 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515652    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1967227b-a4ea-4ee6-945d-efb0b5040a13-service-ca-bundle\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:03:04.515883 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515699    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-var-lib-kubelet\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.515883 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515740    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6b98\" (UniqueName: \"kubernetes.io/projected/a01a018f-2c6e-413f-b88f-37b4e91699b1-kube-api-access-v6b98\") pod \"volume-data-source-validator-58c9f5f46d-j67hv\" (UID: \"a01a018f-2c6e-413f-b88f-37b4e91699b1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j67hv"
Nov 01 05:03:04.515883 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515800    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/037937cc-75c9-4416-bd8f-63589df05625-cni-binary-copy\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:03:04.515883 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515805    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/45a40b3e-ab72-4b18-933f-f52272a6742c-konnectivity-ca\") pod \"konnectivity-agent-jswq7\" (UID: \"45a40b3e-ab72-4b18-933f-f52272a6742c\") " pod="kube-system/konnectivity-agent-jswq7"
Nov 01 05:03:04.515883 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515833    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1967227b-a4ea-4ee6-945d-efb0b5040a13-service-ca-bundle\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:03:04.516270 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515897    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-run-netns\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.516270 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515979    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/45a40b3e-ab72-4b18-933f-f52272a6742c-konnectivity-ca\") pod \"konnectivity-agent-jswq7\" (UID: \"45a40b3e-ab72-4b18-933f-f52272a6742c\") " pod="kube-system/konnectivity-agent-jswq7"
Nov 01 05:03:04.516270 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.515984    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvgdb\" (UniqueName: \"kubernetes.io/projected/b91ced44-8be6-47c8-9229-549400076eb7-kube-api-access-hvgdb\") pod \"kas-connection-checker-685895c7c9-qszhw\" (UID: \"b91ced44-8be6-47c8-9229-549400076eb7\") " pod="kube-system/kas-connection-checker-685895c7c9-qszhw"
Nov 01 05:03:04.516270 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.516001    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:04.516270 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.516020    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:04.516270 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.516087    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-stats-auth\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:04.516270 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.516114    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/36a4e628-5f12-4849-9d77-deaa3e93b190-kubelet-config\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:03:04.516270 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.516130    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/037937cc-75c9-4416-bd8f-63589df05625-cnibin\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:03:04.516512 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.516310    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-stats-auth\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:04.571099 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.571065    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njxgf\" (UniqueName: \"kubernetes.io/projected/6a747e88-6f89-4f3a-ba71-c97babf998f6-kube-api-access-njxgf\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:03:04.571486 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.571466    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjg2z\" (UniqueName: \"kubernetes.io/projected/878896d2-fa82-46ef-af5e-0bf491216b03-kube-api-access-qjg2z\") pod \"console-operator-7cd56b8dbc-6twtw\" (UID: \"878896d2-fa82-46ef-af5e-0bf491216b03\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:03:04.573318 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.573293    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9hd4\" (UniqueName: \"kubernetes.io/projected/df7d0108-685b-4c27-8d0f-bccf4ca5dbbe-kube-api-access-z9hd4\") pod \"kas-connection-checker-685895c7c9-qb994\" (UID: \"df7d0108-685b-4c27-8d0f-bccf4ca5dbbe\") " pod="kube-system/kas-connection-checker-685895c7c9-qb994"
Nov 01 05:03:04.577974 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.577955    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj67m\" (UniqueName: \"kubernetes.io/projected/b53e86a9-ca94-4753-b57b-47eb94bbd879-kube-api-access-bj67m\") pod \"ingress-canary-n8nnj\" (UID: \"b53e86a9-ca94-4753-b57b-47eb94bbd879\") " pod="openshift-ingress-canary/ingress-canary-n8nnj"
Nov 01 05:03:04.578821 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.578679    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8l8g\" (UniqueName: \"kubernetes.io/projected/82335b9e-5584-4afc-b1fc-cc003dbcad4b-kube-api-access-d8l8g\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:04.578821 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.578687    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n4xr\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-kube-api-access-9n4xr\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:04.578821 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.578728    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6vvf\" (UniqueName: \"kubernetes.io/projected/d1f4c040-7436-4df1-8462-6a38acee2c5a-kube-api-access-h6vvf\") pod \"node-ca-m9ckd\" (UID: \"d1f4c040-7436-4df1-8462-6a38acee2c5a\") " pod="openshift-image-registry/node-ca-m9ckd"
Nov 01 05:03:04.578821 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.578751    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhm7l\" (UniqueName: \"kubernetes.io/projected/fb78a5ab-3a2d-45fa-abaa-7b26c33ec929-kube-api-access-dhm7l\") pod \"kas-connection-checker-685895c7c9-d2cm4\" (UID: \"fb78a5ab-3a2d-45fa-abaa-7b26c33ec929\") " pod="kube-system/kas-connection-checker-685895c7c9-d2cm4"
Nov 01 05:03:04.578821 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.578777    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j8z2\" (UniqueName: \"kubernetes.io/projected/ef46e568-5c98-4520-99e5-c49c4c3aca2a-kube-api-access-2j8z2\") pod \"network-metrics-daemon-rvg8t\" (UID: \"ef46e568-5c98-4520-99e5-c49c4c3aca2a\") " pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:03:04.578821 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.578813    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfldn\" (UniqueName: \"kubernetes.io/projected/1967227b-a4ea-4ee6-945d-efb0b5040a13-kube-api-access-xfldn\") pod \"insights-operator-7c4986d449-cvn58\" (UID: \"1967227b-a4ea-4ee6-945d-efb0b5040a13\") " pod="openshift-insights/insights-operator-7c4986d449-cvn58"
Nov 01 05:03:04.579063 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.578951    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9lkh\" (UniqueName: \"kubernetes.io/projected/57fa3738-c3e8-4c18-9582-f6e8377a041e-kube-api-access-p9lkh\") pod \"network-check-source-6f78d59f99-srv7d\" (UID: \"57fa3738-c3e8-4c18-9582-f6e8377a041e\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-srv7d"
Nov 01 05:03:04.579221 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.579198    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6qrt\" (UniqueName: \"kubernetes.io/projected/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-kube-api-access-h6qrt\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:03:04.579265 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.579232    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxknn\" (UniqueName: \"kubernetes.io/projected/bb889bee-f31e-47f1-8617-6a27bc1eebf4-kube-api-access-bxknn\") pod \"iptables-alerter-nk4wf\" (UID: \"bb889bee-f31e-47f1-8617-6a27bc1eebf4\") " pod="openshift-network-operator/iptables-alerter-nk4wf"
Nov 01 05:03:04.579265 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.579237    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzgwz\" (UniqueName: \"kubernetes.io/projected/a5b30117-7cb8-43ec-92a8-4e326b48c22d-kube-api-access-bzgwz\") pod \"network-check-target-tl74d\" (UID: \"a5b30117-7cb8-43ec-92a8-4e326b48c22d\") " pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:03:04.584071 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.584047    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmbj9\" (UniqueName: \"kubernetes.io/projected/0197ee7e-0d4a-4e9c-8057-26356d6f34b3-kube-api-access-xmbj9\") pod \"node-resolver-85452\" (UID: \"0197ee7e-0d4a-4e9c-8057-26356d6f34b3\") " pod="openshift-dns/node-resolver-85452"
Nov 01 05:03:04.584216 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.584167    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdspf\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-kube-api-access-rdspf\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:04.584216 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.584191    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4qrj\" (UniqueName: \"kubernetes.io/projected/037937cc-75c9-4416-bd8f-63589df05625-kube-api-access-w4qrj\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:03:04.584216 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.584212    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-bound-sa-token\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:04.584456 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.584436    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdwk6\" (UniqueName: \"kubernetes.io/projected/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-kube-api-access-zdwk6\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.584535 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.584522    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wblsz\" (UniqueName: \"kubernetes.io/projected/5dc708b0-6c35-4f79-a415-e7533381b95b-kube-api-access-wblsz\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:03:04.587517 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.587500    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6b98\" (UniqueName: \"kubernetes.io/projected/a01a018f-2c6e-413f-b88f-37b4e91699b1-kube-api-access-v6b98\") pod \"volume-data-source-validator-58c9f5f46d-j67hv\" (UID: \"a01a018f-2c6e-413f-b88f-37b4e91699b1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-j67hv"
Nov 01 05:03:04.589230 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.589197    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzrjv\" (UniqueName: \"kubernetes.io/projected/7c9eec03-4159-4a5e-a284-174d634ae0c9-kube-api-access-rzrjv\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:04.589230 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.589204    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pj75\" (UniqueName: \"kubernetes.io/projected/636cccf5-2397-4dca-899a-3598a7b8096b-kube-api-access-5pj75\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz\" (UID: \"636cccf5-2397-4dca-899a-3598a7b8096b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz"
Nov 01 05:03:04.589361 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.589313    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkbbr\" (UniqueName: \"kubernetes.io/projected/bce36747-2159-4b72-8c0e-d98226588d18-kube-api-access-hkbbr\") pod \"service-ca-operator-544d749c8c-97k27\" (UID: \"bce36747-2159-4b72-8c0e-d98226588d18\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27"
Nov 01 05:03:04.589361 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.589321    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlhpw\" (UniqueName: \"kubernetes.io/projected/7fb489a8-8a4f-45f2-977b-594b6126281b-kube-api-access-hlhpw\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:03:04.589432 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.589378    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvgdb\" (UniqueName: \"kubernetes.io/projected/b91ced44-8be6-47c8-9229-549400076eb7-kube-api-access-hvgdb\") pod \"kas-connection-checker-685895c7c9-qszhw\" (UID: \"b91ced44-8be6-47c8-9229-549400076eb7\") " pod="kube-system/kas-connection-checker-685895c7c9-qszhw"
Nov 01 05:03:04.589888 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.589870    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nh4k\" (UniqueName: \"kubernetes.io/projected/0bdedda0-279a-41fd-a7cc-300c816985df-kube-api-access-2nh4k\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.589922 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.589910    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw4b5\" (UniqueName: \"kubernetes.io/projected/ee3cc716-4716-4207-a493-9eeee140a6fe-kube-api-access-tw4b5\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.593142 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.593126    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-bound-sa-token\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:04.616435 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616410    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-sys\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.616517 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616439    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0197ee7e-0d4a-4e9c-8057-26356d6f34b3-hosts-file\") pod \"node-resolver-85452\" (UID: \"0197ee7e-0d4a-4e9c-8057-26356d6f34b3\") " pod="openshift-dns/node-resolver-85452"
Nov 01 05:03:04.616517 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616458    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-run-netns\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.616517 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616476    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-system-cni-dir\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.616517 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616501    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-run-openvswitch\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.616687 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616521    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-sys\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.616687 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616543    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0197ee7e-0d4a-4e9c-8057-26356d6f34b3-hosts-file\") pod \"node-resolver-85452\" (UID: \"0197ee7e-0d4a-4e9c-8057-26356d6f34b3\") " pod="openshift-dns/node-resolver-85452"
Nov 01 05:03:04.616687 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616548    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-var-lib-openvswitch\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.616687 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616577    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-run-netns\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.616687 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616586    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-var-lib-openvswitch\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.616687 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616594    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:04.616687 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616589    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-run-openvswitch\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.616687 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616583    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-system-cni-dir\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.616687 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616635    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.616687 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616653    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/037937cc-75c9-4416-bd8f-63589df05625-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:03:04.616687 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616660    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.616687 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616669    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb889bee-f31e-47f1-8617-6a27bc1eebf4-host-slash\") pod \"iptables-alerter-nk4wf\" (UID: \"bb889bee-f31e-47f1-8617-6a27bc1eebf4\") " pod="openshift-network-operator/iptables-alerter-nk4wf"
Nov 01 05:03:04.616687 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.616687    4864 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.616700    4864 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6969bc8dd6-85rsc: secret "image-registry-tls" not found
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616701    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/037937cc-75c9-4416-bd8f-63589df05625-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616737    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb889bee-f31e-47f1-8617-6a27bc1eebf4-host-slash\") pod \"iptables-alerter-nk4wf\" (UID: \"bb889bee-f31e-47f1-8617-6a27bc1eebf4\") " pod="openshift-network-operator/iptables-alerter-nk4wf"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616691    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-lib-modules\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616777    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-lib-modules\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.616807    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls podName:48c05e4f-a673-4f5f-8c21-8dc1ff716683 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:05.116772265 +0000 UTC m=+2.007148238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls") pod "image-registry-6969bc8dd6-85rsc" (UID: "48c05e4f-a673-4f5f-8c21-8dc1ff716683") : secret "image-registry-tls" not found
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616850    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-etc-selinux\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616880    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-hostroot\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616904    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs\") pod \"network-metrics-daemon-rvg8t\" (UID: \"ef46e568-5c98-4520-99e5-c49c4c3aca2a\") " pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616916    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-etc-selinux\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616929    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616952    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/037937cc-75c9-4416-bd8f-63589df05625-system-cni-dir\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616962    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-hostroot\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.616977    4864 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.616976    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.617007    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs podName:ef46e568-5c98-4520-99e5-c49c4c3aca2a nodeName:}" failed. No retries permitted until 2026-11-01 05:03:05.116998433 +0000 UTC m=+2.007374421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs") pod "network-metrics-daemon-rvg8t" (UID: "ef46e568-5c98-4520-99e5-c49c4c3aca2a") : secret "metrics-daemon-secret" not found
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617018    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617024    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/037937cc-75c9-4416-bd8f-63589df05625-system-cni-dir\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617033    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-run-systemd\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.617048    4864 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617077    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-var-lib-kubelet\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617052    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-var-lib-kubelet\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617094    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-run-systemd\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.617105    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs podName:82335b9e-5584-4afc-b1fc-cc003dbcad4b nodeName:}" failed. No retries permitted until 2026-11-01 05:03:05.117089217 +0000 UTC m=+2.007465184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs") pod "router-default-74dd76c985-n79rt" (UID: "82335b9e-5584-4afc-b1fc-cc003dbcad4b") : secret "router-metrics-certs-default" not found
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617129    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-run-multus-certs\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617146    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-modprobe-d\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617176    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-systemd\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617192    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-run-multus-certs\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617201    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-cni-dir\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.617214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617226    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7c9eec03-4159-4a5e-a284-174d634ae0c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617263    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-systemd\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617264    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-log-socket\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617285    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-cni-dir\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617299    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-sys-fs\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617310    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-log-socket\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617322    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-sysctl-conf\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617338    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/36a4e628-5f12-4849-9d77-deaa3e93b190-dbus\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617347    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-modprobe-d\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617359    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-sys-fs\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617375    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-cnibin\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617378    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/36a4e628-5f12-4849-9d77-deaa3e93b190-dbus\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617420    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-var-lib-cni-multus\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617432    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-cnibin\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617433    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-sysctl-conf\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617458    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-run\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617480    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-ht4j4\" (UID: \"f6185c12-8b19-47d8-b0b3-01aa0f1192a7\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617498    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-kubelet\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617457    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-var-lib-cni-multus\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617520    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-run-ovn-kubernetes\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617539    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-run\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617545    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-kubelet\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617543    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-cni-bin\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.617567    4864 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617574    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-cni-bin\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617568    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-run-ovn-kubernetes\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617594    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-host\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617568    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-host\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.617607    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert podName:f6185c12-8b19-47d8-b0b3-01aa0f1192a7 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:05.117593631 +0000 UTC m=+2.007969584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-ht4j4" (UID: "f6185c12-8b19-47d8-b0b3-01aa0f1192a7") : secret "networking-console-plugin-cert" not found
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617634    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-device-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617649    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-var-lib-cni-bin\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617665    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-cni-netd\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617689    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-socket-dir-parent\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617696    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-cni-netd\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617716    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-device-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617721    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-run-k8s-cni-cncf-io\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617737    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-socket-dir-parent\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617718    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-var-lib-cni-bin\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617753    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-systemd-units\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617775    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-node-log\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617754    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-run-k8s-cni-cncf-io\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617804    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-systemd-units\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617812    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617829    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617837    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-node-log\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617845    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-sysconfig\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617859    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-kubernetes\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617825    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7c9eec03-4159-4a5e-a284-174d634ae0c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617873    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-registration-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617890    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-var-lib-kubelet\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.617895    4864 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617903    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-sysconfig\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.617909    4864 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617915    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-run-netns\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617941    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-var-lib-kubelet\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617923    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-kubernetes\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.617955    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls podName:5dc708b0-6c35-4f79-a415-e7533381b95b nodeName:}" failed. No retries permitted until 2026-11-01 05:03:05.117927984 +0000 UTC m=+2.008303945 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls") pod "dns-default-wcrwt" (UID: "5dc708b0-6c35-4f79-a415-e7533381b95b") : secret "dns-default-metrics-tls" not found
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617957    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-host-run-netns\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.617941    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-registration-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.617982    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls podName:c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:05.11797573 +0000 UTC m=+2.008351684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-mtms9" (UID: "c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47") : secret "cluster-monitoring-operator-tls" not found
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618009    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618045    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618074    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/36a4e628-5f12-4849-9d77-deaa3e93b190-kubelet-config\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.618087    4864 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618098    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/037937cc-75c9-4416-bd8f-63589df05625-cnibin\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618124    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-etc-kubernetes\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.618138    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls podName:7c9eec03-4159-4a5e-a284-174d634ae0c9 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:05.118123144 +0000 UTC m=+2.008499111 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b6sl7" (UID: "7c9eec03-4159-4a5e-a284-174d634ae0c9") : secret "insights-runtime-extractor-tls" not found
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618149    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/36a4e628-5f12-4849-9d77-deaa3e93b190-kubelet-config\") pod \"global-pull-secret-syncer-q92dh\" (UID: \"36a4e628-5f12-4849-9d77-deaa3e93b190\") " pod="kube-system/global-pull-secret-syncer-q92dh"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618157    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-etc-kubernetes\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618152    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/037937cc-75c9-4416-bd8f-63589df05625-cnibin\") pod \"multus-additional-cni-plugins-h6ht6\" (UID: \"037937cc-75c9-4416-bd8f-63589df05625\") " pod="openshift-multus/multus-additional-cni-plugins-h6ht6"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618164    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1f4c040-7436-4df1-8462-6a38acee2c5a-host\") pod \"node-ca-m9ckd\" (UID: \"d1f4c040-7436-4df1-8462-6a38acee2c5a\") " pod="openshift-image-registry/node-ca-m9ckd"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.618178    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle podName:82335b9e-5584-4afc-b1fc-cc003dbcad4b nodeName:}" failed. No retries permitted until 2026-11-01 05:03:05.118166297 +0000 UTC m=+2.008542287 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle") pod "router-default-74dd76c985-n79rt" (UID: "82335b9e-5584-4afc-b1fc-cc003dbcad4b") : configmap references non-existent config key: service-ca.crt
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618196    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1f4c040-7436-4df1-8462-6a38acee2c5a-host\") pod \"node-ca-m9ckd\" (UID: \"d1f4c040-7436-4df1-8462-6a38acee2c5a\") " pod="openshift-image-registry/node-ca-m9ckd"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618216    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618232    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-socket-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618251    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-conf-dir\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618270    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-run-ovn\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.618281    4864 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618297    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618322    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7c9eec03-4159-4a5e-a284-174d634ae0c9-crio-socket\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.618329    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls podName:7fb489a8-8a4f-45f2-977b-594b6126281b nodeName:}" failed. No retries permitted until 2026-11-01 05:03:05.118317503 +0000 UTC m=+2.008693470 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-d8pkk" (UID: "7fb489a8-8a4f-45f2-977b-594b6126281b") : secret "samples-operator-tls" not found
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618328    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6a747e88-6f89-4f3a-ba71-c97babf998f6-socket-dir\") pod \"aws-ebs-csi-driver-node-qfsxd\" (UID: \"6a747e88-6f89-4f3a-ba71-c97babf998f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qfsxd"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618339    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-run-ovn\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618327    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee3cc716-4716-4207-a493-9eeee140a6fe-multus-conf-dir\") pod \"multus-gwbm4\" (UID: \"ee3cc716-4716-4207-a493-9eeee140a6fe\") " pod="openshift-multus/multus-gwbm4"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618357    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-slash\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618376    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-host-slash\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.618382    4864 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618386    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-sysctl-d\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.618333 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.618396    4864 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79659c8779-dnfbf: secret "image-registry-tls" not found
Nov 01 05:03:04.620860 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618360    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7c9eec03-4159-4a5e-a284-174d634ae0c9-crio-socket\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:04.620860 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618411    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-etc-openvswitch\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.620860 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:04.618441    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls podName:62652d56-1be6-47cc-a774-77d4fa9d206a nodeName:}" failed. No retries permitted until 2026-11-01 05:03:05.118428519 +0000 UTC m=+2.008804488 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls") pod "image-registry-79659c8779-dnfbf" (UID: "62652d56-1be6-47cc-a774-77d4fa9d206a") : secret "image-registry-tls" not found
Nov 01 05:03:04.620860 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618450    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bdedda0-279a-41fd-a7cc-300c816985df-etc-openvswitch\") pod \"ovnkube-node-89wdz\" (UID: \"0bdedda0-279a-41fd-a7cc-300c816985df\") " pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:04.620860 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.618532    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d09454b6-6ac7-46dc-83ee-a3c60dfdf262-etc-sysctl-d\") pod \"tuned-m8d7h\" (UID: \"d09454b6-6ac7-46dc-83ee-a3c60dfdf262\") " pod="openshift-cluster-node-tuning-operator/tuned-m8d7h"
Nov 01 05:03:04.642589 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:04.642569    4864 scope.go:122] "RemoveContainer" containerID="2bfdcabf49fcf5bed5a8daab44aa1f9b67a4866acfc31c80777617c2ac716e5f"
Nov 01 05:03:05.122445 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:05.122409    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:03:05.122445 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:05.122449    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:05.122652 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.122549    4864 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 05:03:05.122652 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.122560    4864 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79659c8779-dnfbf: secret "image-registry-tls" not found
Nov 01 05:03:05.122652 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.122563    4864 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 01 05:03:05.122652 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:05.122579    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:05.122652 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.122601    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls podName:62652d56-1be6-47cc-a774-77d4fa9d206a nodeName:}" failed. No retries permitted until 2026-11-01 05:03:06.122588835 +0000 UTC m=+3.012964790 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls") pod "image-registry-79659c8779-dnfbf" (UID: "62652d56-1be6-47cc-a774-77d4fa9d206a") : secret "image-registry-tls" not found
Nov 01 05:03:05.122652 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.122617    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls podName:7fb489a8-8a4f-45f2-977b-594b6126281b nodeName:}" failed. No retries permitted until 2026-11-01 05:03:06.122610493 +0000 UTC m=+3.012986447 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-d8pkk" (UID: "7fb489a8-8a4f-45f2-977b-594b6126281b") : secret "samples-operator-tls" not found
Nov 01 05:03:05.122652 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:05.122641    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs\") pod \"network-metrics-daemon-rvg8t\" (UID: \"ef46e568-5c98-4520-99e5-c49c4c3aca2a\") " pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:03:05.122652 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.122644    4864 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 05:03:05.122652 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.122654    4864 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6969bc8dd6-85rsc: secret "image-registry-tls" not found
Nov 01 05:03:05.123022 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:05.122660    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:05.123022 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.122685    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls podName:48c05e4f-a673-4f5f-8c21-8dc1ff716683 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:06.122674287 +0000 UTC m=+3.013050258 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls") pod "image-registry-6969bc8dd6-85rsc" (UID: "48c05e4f-a673-4f5f-8c21-8dc1ff716683") : secret "image-registry-tls" not found
Nov 01 05:03:05.123022 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.122706    4864 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 01 05:03:05.123022 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:05.122720    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-ht4j4\" (UID: \"f6185c12-8b19-47d8-b0b3-01aa0f1192a7\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"
Nov 01 05:03:05.123022 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.122731    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs podName:82335b9e-5584-4afc-b1fc-cc003dbcad4b nodeName:}" failed. No retries permitted until 2026-11-01 05:03:06.12272019 +0000 UTC m=+3.013096146 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs") pod "router-default-74dd76c985-n79rt" (UID: "82335b9e-5584-4afc-b1fc-cc003dbcad4b") : secret "router-metrics-certs-default" not found
Nov 01 05:03:05.123022 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.122752    4864 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 01 05:03:05.123022 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.122760    4864 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 01 05:03:05.123022 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:05.122762    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:03:05.123022 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:05.122780    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:03:05.123022 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.122809    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert podName:f6185c12-8b19-47d8-b0b3-01aa0f1192a7 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:06.122777248 +0000 UTC m=+3.013153235 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-ht4j4" (UID: "f6185c12-8b19-47d8-b0b3-01aa0f1192a7") : secret "networking-console-plugin-cert" not found
Nov 01 05:03:05.123022 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.122831    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs podName:ef46e568-5c98-4520-99e5-c49c4c3aca2a nodeName:}" failed. No retries permitted until 2026-11-01 05:03:06.12282522 +0000 UTC m=+3.013201174 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs") pod "network-metrics-daemon-rvg8t" (UID: "ef46e568-5c98-4520-99e5-c49c4c3aca2a") : secret "metrics-daemon-secret" not found
Nov 01 05:03:05.123022 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.122855    4864 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 01 05:03:05.123022 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.122856    4864 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 01 05:03:05.123022 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:05.122858    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:05.123022 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.122886    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls podName:c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:06.12287654 +0000 UTC m=+3.013252497 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-mtms9" (UID: "c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47") : secret "cluster-monitoring-operator-tls" not found
Nov 01 05:03:05.123022 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.122895    4864 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 01 05:03:05.123022 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.122915    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls podName:5dc708b0-6c35-4f79-a415-e7533381b95b nodeName:}" failed. No retries permitted until 2026-11-01 05:03:06.122909354 +0000 UTC m=+3.013285308 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls") pod "dns-default-wcrwt" (UID: "5dc708b0-6c35-4f79-a415-e7533381b95b") : secret "dns-default-metrics-tls" not found
Nov 01 05:03:05.123022 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:05.122928    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:05.123022 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.122971    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls podName:7c9eec03-4159-4a5e-a284-174d634ae0c9 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:06.122964834 +0000 UTC m=+3.013340788 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b6sl7" (UID: "7c9eec03-4159-4a5e-a284-174d634ae0c9") : secret "insights-runtime-extractor-tls" not found
Nov 01 05:03:05.123022 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.122982    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle podName:82335b9e-5584-4afc-b1fc-cc003dbcad4b nodeName:}" failed. No retries permitted until 2026-11-01 05:03:06.122976517 +0000 UTC m=+3.013352471 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle") pod "router-default-74dd76c985-n79rt" (UID: "82335b9e-5584-4afc-b1fc-cc003dbcad4b") : configmap references non-existent config key: service-ca.crt
Nov 01 05:03:05.299694 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:05.299666    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:03:05.300109 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:05.300071    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/1.log"
Nov 01 05:03:05.300144 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:05.300104    4864 generic.go:356] "Generic (PLEG): container finished" podID="878896d2-fa82-46ef-af5e-0bf491216b03" containerID="c08dd451c0d57973f398a0c2fe171502d897ada6b44aa99cab4ecf5eb7cbdc17" exitCode=255
Nov 01 05:03:05.300204 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:05.300185    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw" event={"ID":"878896d2-fa82-46ef-af5e-0bf491216b03","Type":"ContainerDied","Data":"c08dd451c0d57973f398a0c2fe171502d897ada6b44aa99cab4ecf5eb7cbdc17"}
Nov 01 05:03:05.300253 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:05.300241    4864 scope.go:122] "RemoveContainer" containerID="2bfdcabf49fcf5bed5a8daab44aa1f9b67a4866acfc31c80777617c2ac716e5f"
Nov 01 05:03:05.300740 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:05.300726    4864 scope.go:122] "RemoveContainer" containerID="c08dd451c0d57973f398a0c2fe171502d897ada6b44aa99cab4ecf5eb7cbdc17"
Nov 01 05:03:05.300930 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:05.300913    4864 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-6twtw_openshift-console-operator(878896d2-fa82-46ef-af5e-0bf491216b03)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw" podUID="878896d2-fa82-46ef-af5e-0bf491216b03"
Nov 01 05:03:05.387568 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:05.387498    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:06.132871 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:06.132830    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:06.132871 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:06.132870    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:06.133101 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:06.132897    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:03:06.133101 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:06.132920    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:06.133101 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:06.132953    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:06.133101 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:06.132988    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs\") pod \"network-metrics-daemon-rvg8t\" (UID: \"ef46e568-5c98-4520-99e5-c49c4c3aca2a\") " pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:03:06.133101 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.132994    4864 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 01 05:03:06.133101 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.133020    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle podName:82335b9e-5584-4afc-b1fc-cc003dbcad4b nodeName:}" failed. No retries permitted until 2026-11-01 05:03:08.132998389 +0000 UTC m=+5.023374359 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle") pod "router-default-74dd76c985-n79rt" (UID: "82335b9e-5584-4afc-b1fc-cc003dbcad4b") : configmap references non-existent config key: service-ca.crt
Nov 01 05:03:06.133101 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.133051    4864 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 01 05:03:06.133101 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.133059    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls podName:7c9eec03-4159-4a5e-a284-174d634ae0c9 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:08.133048915 +0000 UTC m=+5.023424877 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b6sl7" (UID: "7c9eec03-4159-4a5e-a284-174d634ae0c9") : secret "insights-runtime-extractor-tls" not found
Nov 01 05:03:06.133101 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.133059    4864 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 05:03:06.133101 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.133077    4864 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79659c8779-dnfbf: secret "image-registry-tls" not found
Nov 01 05:03:06.133101 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.133083    4864 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 05:03:06.133101 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.133095    4864 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6969bc8dd6-85rsc: secret "image-registry-tls" not found
Nov 01 05:03:06.133101 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.133099    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs podName:ef46e568-5c98-4520-99e5-c49c4c3aca2a nodeName:}" failed. No retries permitted until 2026-11-01 05:03:08.133083842 +0000 UTC m=+5.023459816 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs") pod "network-metrics-daemon-rvg8t" (UID: "ef46e568-5c98-4520-99e5-c49c4c3aca2a") : secret "metrics-daemon-secret" not found
Nov 01 05:03:06.133585 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.133118    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls podName:62652d56-1be6-47cc-a774-77d4fa9d206a nodeName:}" failed. No retries permitted until 2026-11-01 05:03:08.133106898 +0000 UTC m=+5.023482854 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls") pod "image-registry-79659c8779-dnfbf" (UID: "62652d56-1be6-47cc-a774-77d4fa9d206a") : secret "image-registry-tls" not found
Nov 01 05:03:06.133585 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:06.133079    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:06.133585 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.133131    4864 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 01 05:03:06.133585 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.133135    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls podName:48c05e4f-a673-4f5f-8c21-8dc1ff716683 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:08.13312669 +0000 UTC m=+5.023502643 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls") pod "image-registry-6969bc8dd6-85rsc" (UID: "48c05e4f-a673-4f5f-8c21-8dc1ff716683") : secret "image-registry-tls" not found
Nov 01 05:03:06.133585 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.133165    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs podName:82335b9e-5584-4afc-b1fc-cc003dbcad4b nodeName:}" failed. No retries permitted until 2026-11-01 05:03:08.133154183 +0000 UTC m=+5.023530142 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs") pod "router-default-74dd76c985-n79rt" (UID: "82335b9e-5584-4afc-b1fc-cc003dbcad4b") : secret "router-metrics-certs-default" not found
Nov 01 05:03:06.133585 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:06.133199    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-ht4j4\" (UID: \"f6185c12-8b19-47d8-b0b3-01aa0f1192a7\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"
Nov 01 05:03:06.133585 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:06.133238    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:03:06.133585 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:06.133255    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:03:06.133585 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.133056    4864 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 01 05:03:06.133585 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.133328    4864 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 01 05:03:06.133585 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.133336    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls podName:7fb489a8-8a4f-45f2-977b-594b6126281b nodeName:}" failed. No retries permitted until 2026-11-01 05:03:08.133315118 +0000 UTC m=+5.023691074 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-d8pkk" (UID: "7fb489a8-8a4f-45f2-977b-594b6126281b") : secret "samples-operator-tls" not found
Nov 01 05:03:06.133585 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.133339    4864 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 01 05:03:06.133585 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.133353    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert podName:f6185c12-8b19-47d8-b0b3-01aa0f1192a7 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:08.133347086 +0000 UTC m=+5.023723041 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-ht4j4" (UID: "f6185c12-8b19-47d8-b0b3-01aa0f1192a7") : secret "networking-console-plugin-cert" not found
Nov 01 05:03:06.133585 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.133369    4864 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 01 05:03:06.133585 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.133376    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls podName:c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:08.133365578 +0000 UTC m=+5.023741561 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-mtms9" (UID: "c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47") : secret "cluster-monitoring-operator-tls" not found
Nov 01 05:03:06.133585 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.133418    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls podName:5dc708b0-6c35-4f79-a415-e7533381b95b nodeName:}" failed. No retries permitted until 2026-11-01 05:03:08.133408301 +0000 UTC m=+5.023784255 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls") pod "dns-default-wcrwt" (UID: "5dc708b0-6c35-4f79-a415-e7533381b95b") : secret "dns-default-metrics-tls" not found
Nov 01 05:03:06.285121 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:06.285089    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:03:06.304757 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:06.304729    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:03:06.305122 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:06.305064    4864 scope.go:122] "RemoveContainer" containerID="c08dd451c0d57973f398a0c2fe171502d897ada6b44aa99cab4ecf5eb7cbdc17"
Nov 01 05:03:06.305260 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:06.305242    4864 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-6twtw_openshift-console-operator(878896d2-fa82-46ef-af5e-0bf491216b03)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw" podUID="878896d2-fa82-46ef-af5e-0bf491216b03"
Nov 01 05:03:08.150981 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:08.150941    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs\") pod \"network-metrics-daemon-rvg8t\" (UID: \"ef46e568-5c98-4520-99e5-c49c4c3aca2a\") " pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:03:08.150981 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:08.150982    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:08.151541 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:08.151021    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-ht4j4\" (UID: \"f6185c12-8b19-47d8-b0b3-01aa0f1192a7\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"
Nov 01 05:03:08.151541 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:08.151062    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:03:08.151541 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:08.151089    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:03:08.151541 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:08.151121    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:08.151541 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:08.151121    4864 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 01 05:03:08.151541 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:08.151188    4864 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 01 05:03:08.151541 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:08.151202    4864 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 01 05:03:08.151541 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:08.151220    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs podName:ef46e568-5c98-4520-99e5-c49c4c3aca2a nodeName:}" failed. No retries permitted until 2026-11-01 05:03:12.151175007 +0000 UTC m=+9.041550975 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs") pod "network-metrics-daemon-rvg8t" (UID: "ef46e568-5c98-4520-99e5-c49c4c3aca2a") : secret "metrics-daemon-secret" not found
Nov 01 05:03:08.151541 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:08.151234    4864 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 01 05:03:08.151541 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:08.151252    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls podName:c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:12.151234106 +0000 UTC m=+9.041610080 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-mtms9" (UID: "c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47") : secret "cluster-monitoring-operator-tls" not found
Nov 01 05:03:08.151541 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:08.151273    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls podName:5dc708b0-6c35-4f79-a415-e7533381b95b nodeName:}" failed. No retries permitted until 2026-11-01 05:03:12.151262329 +0000 UTC m=+9.041638283 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls") pod "dns-default-wcrwt" (UID: "5dc708b0-6c35-4f79-a415-e7533381b95b") : secret "dns-default-metrics-tls" not found
Nov 01 05:03:08.151541 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:08.151297    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:08.151541 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:08.151344    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:03:08.151541 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:08.151357    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls podName:7c9eec03-4159-4a5e-a284-174d634ae0c9 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:12.151338666 +0000 UTC m=+9.041714631 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b6sl7" (UID: "7c9eec03-4159-4a5e-a284-174d634ae0c9") : secret "insights-runtime-extractor-tls" not found
Nov 01 05:03:08.151541 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:08.151402    4864 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 01 05:03:08.151541 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:08.151419    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:08.151541 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:08.151435    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls podName:7fb489a8-8a4f-45f2-977b-594b6126281b nodeName:}" failed. No retries permitted until 2026-11-01 05:03:12.151425333 +0000 UTC m=+9.041801310 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-d8pkk" (UID: "7fb489a8-8a4f-45f2-977b-594b6126281b") : secret "samples-operator-tls" not found
Nov 01 05:03:08.151541 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:08.151420    4864 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 01 05:03:08.151541 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:08.151490    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:08.151541 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:08.151493    4864 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 05:03:08.152214 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:08.151560    4864 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 05:03:08.152214 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:08.151567    4864 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79659c8779-dnfbf: secret "image-registry-tls" not found
Nov 01 05:03:08.152214 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:08.151574    4864 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6969bc8dd6-85rsc: secret "image-registry-tls" not found
Nov 01 05:03:08.152214 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:08.151613    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls podName:62652d56-1be6-47cc-a774-77d4fa9d206a nodeName:}" failed. No retries permitted until 2026-11-01 05:03:12.151600419 +0000 UTC m=+9.041976403 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls") pod "image-registry-79659c8779-dnfbf" (UID: "62652d56-1be6-47cc-a774-77d4fa9d206a") : secret "image-registry-tls" not found
Nov 01 05:03:08.152214 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:08.151632    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls podName:48c05e4f-a673-4f5f-8c21-8dc1ff716683 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:12.151623384 +0000 UTC m=+9.041999377 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls") pod "image-registry-6969bc8dd6-85rsc" (UID: "48c05e4f-a673-4f5f-8c21-8dc1ff716683") : secret "image-registry-tls" not found
Nov 01 05:03:08.152214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:08.151962    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82335b9e-5584-4afc-b1fc-cc003dbcad4b-service-ca-bundle\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:08.153855 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:08.153838    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82335b9e-5584-4afc-b1fc-cc003dbcad4b-metrics-certs\") pod \"router-default-74dd76c985-n79rt\" (UID: \"82335b9e-5584-4afc-b1fc-cc003dbcad4b\") " pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:08.153916 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:08.153894    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f6185c12-8b19-47d8-b0b3-01aa0f1192a7-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-ht4j4\" (UID: \"f6185c12-8b19-47d8-b0b3-01aa0f1192a7\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"
Nov 01 05:03:08.245704 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:08.245673    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:08.246773 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:08.246749    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"
Nov 01 05:03:08.699204 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:08.699178    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-85452_0197ee7e-0d4a-4e9c-8057-26356d6f34b3/dns-node-resolver/0.log"
Nov 01 05:03:08.715685 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:08.715582    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-ht4j4"]
Nov 01 05:03:08.719242 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:08.719217    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6185c12_8b19_47d8_b0b3_01aa0f1192a7.slice/crio-d5a55df90883384c553d34ccc0c1eaf4129dcdd87e5b6c463447fd140c6cc930 WatchSource:0}: Error finding container d5a55df90883384c553d34ccc0c1eaf4129dcdd87e5b6c463447fd140c6cc930: Status 404 returned error can't find the container with id d5a55df90883384c553d34ccc0c1eaf4129dcdd87e5b6c463447fd140c6cc930
Nov 01 05:03:08.719712 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:08.719693    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74dd76c985-n79rt"]
Nov 01 05:03:08.721270 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:08.721256    4864 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 05:03:08.768737 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:08.768709    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82335b9e_5584_4afc_b1fc_cc003dbcad4b.slice/crio-123cb659b798914056ab97cec3e7099be480d13d08b7f63b1e43110cb6d59fd4 WatchSource:0}: Error finding container 123cb659b798914056ab97cec3e7099be480d13d08b7f63b1e43110cb6d59fd4: Status 404 returned error can't find the container with id 123cb659b798914056ab97cec3e7099be480d13d08b7f63b1e43110cb6d59fd4
Nov 01 05:03:09.317493 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:09.317445    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4" event={"ID":"f6185c12-8b19-47d8-b0b3-01aa0f1192a7","Type":"ContainerStarted","Data":"d5a55df90883384c553d34ccc0c1eaf4129dcdd87e5b6c463447fd140c6cc930"}
Nov 01 05:03:09.319341 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:09.319315    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74dd76c985-n79rt" event={"ID":"82335b9e-5584-4afc-b1fc-cc003dbcad4b","Type":"ContainerStarted","Data":"4b14b27d9d0c79530908c9bfe9313a7745fa1b9063f829635a5b92364ee3cb51"}
Nov 01 05:03:09.319486 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:09.319356    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74dd76c985-n79rt" event={"ID":"82335b9e-5584-4afc-b1fc-cc003dbcad4b","Type":"ContainerStarted","Data":"123cb659b798914056ab97cec3e7099be480d13d08b7f63b1e43110cb6d59fd4"}
Nov 01 05:03:09.372879 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:09.372843    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:03:09.899910 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:09.899879    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m9ckd_d1f4c040-7436-4df1-8462-6a38acee2c5a/node-ca/0.log"
Nov 01 05:03:10.246541 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:10.246462    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:10.249220 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:10.249197    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:10.300087 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:10.300065    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74dd76c985-n79rt_82335b9e-5584-4afc-b1fc-cc003dbcad4b/router/0.log"
Nov 01 05:03:10.324864 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:10.324831    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-ht4j4" event={"ID":"f6185c12-8b19-47d8-b0b3-01aa0f1192a7","Type":"ContainerStarted","Data":"6a32f2f696172bbba2a6f54685ebb2f4abd52e031633311d5047b4954976ebe5"}
Nov 01 05:03:10.899465 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:10.899439    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n8nnj_b53e86a9-ca94-4753-b57b-47eb94bbd879/serve-healthcheck-canary/0.log"
Nov 01 05:03:11.257915 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:11.257831    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:11.258050 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:11.258021    4864 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 01 05:03:11.328041 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:11.328012    4864 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 01 05:03:11.500897 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:11.500864    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz_636cccf5-2397-4dca-899a-3598a7b8096b/kube-storage-version-migrator-operator/0.log"
Nov 01 05:03:12.190163 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:12.190122    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:12.190366 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:12.190184    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:12.190366 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:12.190218    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs\") pod \"network-metrics-daemon-rvg8t\" (UID: \"ef46e568-5c98-4520-99e5-c49c4c3aca2a\") " pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:03:12.190366 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:12.190278    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:03:12.190366 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:12.190284    4864 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 05:03:12.190366 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:12.190303    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:03:12.190366 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:12.190336    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:12.190366 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:12.190310    4864 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79659c8779-dnfbf: secret "image-registry-tls" not found
Nov 01 05:03:12.190707 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:12.190366    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:03:12.190707 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:12.190325    4864 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 05:03:12.190707 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:12.190358    4864 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 01 05:03:12.190874 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:12.190842    4864 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 01 05:03:12.190949 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:12.190934    4864 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 01 05:03:12.191006 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:12.190432    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls podName:62652d56-1be6-47cc-a774-77d4fa9d206a nodeName:}" failed. No retries permitted until 2026-11-01 05:03:20.19041336 +0000 UTC m=+17.080789332 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls") pod "image-registry-79659c8779-dnfbf" (UID: "62652d56-1be6-47cc-a774-77d4fa9d206a") : secret "image-registry-tls" not found
Nov 01 05:03:12.191068 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:12.190612    4864 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6969bc8dd6-85rsc: secret "image-registry-tls" not found
Nov 01 05:03:12.191123 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:12.191069    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls podName:7fb489a8-8a4f-45f2-977b-594b6126281b nodeName:}" failed. No retries permitted until 2026-11-01 05:03:20.191051179 +0000 UTC m=+17.081427147 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-d8pkk" (UID: "7fb489a8-8a4f-45f2-977b-594b6126281b") : secret "samples-operator-tls" not found
Nov 01 05:03:12.191123 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:12.191095    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls podName:7c9eec03-4159-4a5e-a284-174d634ae0c9 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:20.191077382 +0000 UTC m=+17.081453359 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b6sl7" (UID: "7c9eec03-4159-4a5e-a284-174d634ae0c9") : secret "insights-runtime-extractor-tls" not found
Nov 01 05:03:12.191123 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:12.191111    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs podName:ef46e568-5c98-4520-99e5-c49c4c3aca2a nodeName:}" failed. No retries permitted until 2026-11-01 05:03:20.191102608 +0000 UTC m=+17.081478566 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs") pod "network-metrics-daemon-rvg8t" (UID: "ef46e568-5c98-4520-99e5-c49c4c3aca2a") : secret "metrics-daemon-secret" not found
Nov 01 05:03:12.191287 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:12.191126    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls podName:48c05e4f-a673-4f5f-8c21-8dc1ff716683 nodeName:}" failed. No retries permitted until 2026-11-01 05:03:20.191117905 +0000 UTC m=+17.081493885 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls") pod "image-registry-6969bc8dd6-85rsc" (UID: "48c05e4f-a673-4f5f-8c21-8dc1ff716683") : secret "image-registry-tls" not found
Nov 01 05:03:12.193709 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:12.193679    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5dc708b0-6c35-4f79-a415-e7533381b95b-metrics-tls\") pod \"dns-default-wcrwt\" (UID: \"5dc708b0-6c35-4f79-a415-e7533381b95b\") " pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:03:12.195841 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:12.195810    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-mtms9\" (UID: \"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:03:12.349188 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:12.349155    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:12.349592 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:12.349282    4864 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 01 05:03:12.350493 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:12.350472    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-74dd76c985-n79rt"
Nov 01 05:03:12.444549 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:12.444472    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"
Nov 01 05:03:12.457886 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:12.457861    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:03:12.867145 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:12.867112    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9"]
Nov 01 05:03:12.870422 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:12.870388    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0b1d57c_ca0d_4dfe_8ba4_98668bd12b47.slice/crio-273636c7f6a10b3a872cbd00e1cf40944bb27230e9b17d7d21646b1dabea6cbb WatchSource:0}: Error finding container 273636c7f6a10b3a872cbd00e1cf40944bb27230e9b17d7d21646b1dabea6cbb: Status 404 returned error can't find the container with id 273636c7f6a10b3a872cbd00e1cf40944bb27230e9b17d7d21646b1dabea6cbb
Nov 01 05:03:12.871477 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:12.871453    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wcrwt"]
Nov 01 05:03:12.912966 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:12.912939    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dc708b0_6c35_4f79_a415_e7533381b95b.slice/crio-5e87c40a1e0f82365cb00e04bcd3a4febe12b28f87a36f293c27e2b1f5c941c5 WatchSource:0}: Error finding container 5e87c40a1e0f82365cb00e04bcd3a4febe12b28f87a36f293c27e2b1f5c941c5: Status 404 returned error can't find the container with id 5e87c40a1e0f82365cb00e04bcd3a4febe12b28f87a36f293c27e2b1f5c941c5
Nov 01 05:03:13.334590 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:13.334557    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wcrwt" event={"ID":"5dc708b0-6c35-4f79-a415-e7533381b95b","Type":"ContainerStarted","Data":"5e87c40a1e0f82365cb00e04bcd3a4febe12b28f87a36f293c27e2b1f5c941c5"}
Nov 01 05:03:13.335666 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:13.335644    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9" event={"ID":"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47","Type":"ContainerStarted","Data":"273636c7f6a10b3a872cbd00e1cf40944bb27230e9b17d7d21646b1dabea6cbb"}
Nov 01 05:03:14.443989 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:14.443952    4864 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:03:14.444580 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:14.444415    4864 scope.go:122] "RemoveContainer" containerID="c08dd451c0d57973f398a0c2fe171502d897ada6b44aa99cab4ecf5eb7cbdc17"
Nov 01 05:03:14.444673 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:14.444647    4864 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-6twtw_openshift-console-operator(878896d2-fa82-46ef-af5e-0bf491216b03)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw" podUID="878896d2-fa82-46ef-af5e-0bf491216b03"
Nov 01 05:03:15.348303 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:15.348272    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-mtms9" event={"ID":"c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47","Type":"ContainerStarted","Data":"b2d25423cbb360466ffbe3ceffdcb9481c6e2f1842455e6fe2666c3c566c840c"}
Nov 01 05:03:15.350020 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:15.349990    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wcrwt" event={"ID":"5dc708b0-6c35-4f79-a415-e7533381b95b","Type":"ContainerStarted","Data":"4f590af5eb118aa7249bdfa11d09cb2472384fc27c3662ecfe821a14373ee731"}
Nov 01 05:03:15.350154 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:15.350044    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wcrwt" event={"ID":"5dc708b0-6c35-4f79-a415-e7533381b95b","Type":"ContainerStarted","Data":"2ede0655f88422a7a268dcc2af578689995edf7074dba806609db591d2d7e850"}
Nov 01 05:03:15.350235 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:15.350221    4864 scope.go:122] "RemoveContainer" containerID="c08dd451c0d57973f398a0c2fe171502d897ada6b44aa99cab4ecf5eb7cbdc17"
Nov 01 05:03:16.355756 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:16.355725    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:03:16.356160 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:16.355843    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw" event={"ID":"878896d2-fa82-46ef-af5e-0bf491216b03","Type":"ContainerStarted","Data":"f2597f99aa36ca0060e302f7b0588951c0a69d944a205c71d83b52c462bfd1a5"}
Nov 01 05:03:16.356229 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:16.356161    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:03:16.356513 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:16.356496    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:03:16.361103 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:16.361081    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-6twtw"
Nov 01 05:03:20.269757 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:20.269717    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:20.270276 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:20.269760    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:03:20.270276 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:20.269841    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:20.270276 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:20.269876    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:20.270276 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:20.269907    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs\") pod \"network-metrics-daemon-rvg8t\" (UID: \"ef46e568-5c98-4520-99e5-c49c4c3aca2a\") " pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:03:20.272408 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:20.272377    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef46e568-5c98-4520-99e5-c49c4c3aca2a-metrics-certs\") pod \"network-metrics-daemon-rvg8t\" (UID: \"ef46e568-5c98-4520-99e5-c49c4c3aca2a\") " pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:03:20.272537 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:20.272510    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls\") pod \"image-registry-79659c8779-dnfbf\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") " pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:20.272633 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:20.272610    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls\") pod \"image-registry-6969bc8dd6-85rsc\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") " pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:20.272740 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:20.272725    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c9eec03-4159-4a5e-a284-174d634ae0c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6sl7\" (UID: \"7c9eec03-4159-4a5e-a284-174d634ae0c9\") " pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:20.272819 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:20.272780    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fb489a8-8a4f-45f2-977b-594b6126281b-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-d8pkk\" (UID: \"7fb489a8-8a4f-45f2-977b-594b6126281b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:03:20.543525 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:20.543432    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"
Nov 01 05:03:20.546666 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:20.546648    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:20.547722 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:20.547704    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvg8t"
Nov 01 05:03:20.556283 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:20.556264    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:20.558363 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:20.558201    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-b6sl7"
Nov 01 05:03:20.936104 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:20.936068    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk"]
Nov 01 05:03:20.985450 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:20.985430    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rvg8t"]
Nov 01 05:03:20.988458 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:20.988436    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79659c8779-dnfbf"]
Nov 01 05:03:20.992001 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:20.991974    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef46e568_5c98_4520_99e5_c49c4c3aca2a.slice/crio-64b9710d29909890ce68f06347f36005c5fb6f47e7d94f879a7db194e446c492 WatchSource:0}: Error finding container 64b9710d29909890ce68f06347f36005c5fb6f47e7d94f879a7db194e446c492: Status 404 returned error can't find the container with id 64b9710d29909890ce68f06347f36005c5fb6f47e7d94f879a7db194e446c492
Nov 01 05:03:21.025986 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:21.025962    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62652d56_1be6_47cc_a774_77d4fa9d206a.slice/crio-a510523a1afbc8d54ebf4c7c937335c511b50b1a2f091e97125b8083f514755a WatchSource:0}: Error finding container a510523a1afbc8d54ebf4c7c937335c511b50b1a2f091e97125b8083f514755a: Status 404 returned error can't find the container with id a510523a1afbc8d54ebf4c7c937335c511b50b1a2f091e97125b8083f514755a
Nov 01 05:03:21.036505 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:21.036425    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-b6sl7"]
Nov 01 05:03:21.039822 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:21.039776    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6969bc8dd6-85rsc"]
Nov 01 05:03:21.040119 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:21.040094    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c9eec03_4159_4a5e_a284_174d634ae0c9.slice/crio-9cf7fa52ad6e2fa5b6071e994b83d4312b31a7048f55ba428df4771d746269b1 WatchSource:0}: Error finding container 9cf7fa52ad6e2fa5b6071e994b83d4312b31a7048f55ba428df4771d746269b1: Status 404 returned error can't find the container with id 9cf7fa52ad6e2fa5b6071e994b83d4312b31a7048f55ba428df4771d746269b1
Nov 01 05:03:21.043307 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:21.043287    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48c05e4f_a673_4f5f_8c21_8dc1ff716683.slice/crio-d5fea4e8ea2de0409cf2f7beea639dec8e1836ecd921f816450ec5a596d9a39d WatchSource:0}: Error finding container d5fea4e8ea2de0409cf2f7beea639dec8e1836ecd921f816450ec5a596d9a39d: Status 404 returned error can't find the container with id d5fea4e8ea2de0409cf2f7beea639dec8e1836ecd921f816450ec5a596d9a39d
Nov 01 05:03:21.375365 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:21.375330    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79659c8779-dnfbf" event={"ID":"62652d56-1be6-47cc-a774-77d4fa9d206a","Type":"ContainerStarted","Data":"5d0ffedcbc5d00eefa11cea4f6f669e716369730ca622de216328c5da5e11f85"}
Nov 01 05:03:21.375867 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:21.375390    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79659c8779-dnfbf" event={"ID":"62652d56-1be6-47cc-a774-77d4fa9d206a","Type":"ContainerStarted","Data":"a510523a1afbc8d54ebf4c7c937335c511b50b1a2f091e97125b8083f514755a"}
Nov 01 05:03:21.375867 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:21.375515    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:21.377174 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:21.377146    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b6sl7" event={"ID":"7c9eec03-4159-4a5e-a284-174d634ae0c9","Type":"ContainerStarted","Data":"74b20c066cb4fd5d0652e7d87b7da05a3655d3c7a5c1e1482df91a4b6ca5e41e"}
Nov 01 05:03:21.377305 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:21.377204    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b6sl7" event={"ID":"7c9eec03-4159-4a5e-a284-174d634ae0c9","Type":"ContainerStarted","Data":"9cf7fa52ad6e2fa5b6071e994b83d4312b31a7048f55ba428df4771d746269b1"}
Nov 01 05:03:21.378520 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:21.378494    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk" event={"ID":"7fb489a8-8a4f-45f2-977b-594b6126281b","Type":"ContainerStarted","Data":"a1fa581d8c91f3818bf3996bc1d7f9f7a4be62143610fed5a7adcd3b54fbde79"}
Nov 01 05:03:21.380155 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:21.380133    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc" event={"ID":"48c05e4f-a673-4f5f-8c21-8dc1ff716683","Type":"ContainerStarted","Data":"da8859d512bfebf4ab55f7244ca0c10119cd71623c84c32858eae93c3f16ac4f"}
Nov 01 05:03:21.380263 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:21.380172    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc" event={"ID":"48c05e4f-a673-4f5f-8c21-8dc1ff716683","Type":"ContainerStarted","Data":"d5fea4e8ea2de0409cf2f7beea639dec8e1836ecd921f816450ec5a596d9a39d"}
Nov 01 05:03:21.380263 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:21.380198    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:21.381598 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:21.381578    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rvg8t" event={"ID":"ef46e568-5c98-4520-99e5-c49c4c3aca2a","Type":"ContainerStarted","Data":"64b9710d29909890ce68f06347f36005c5fb6f47e7d94f879a7db194e446c492"}
Nov 01 05:03:21.873718 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:21.873687    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-79659c8779-dnfbf"]
Nov 01 05:03:22.389457 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:22.389406    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b6sl7" event={"ID":"7c9eec03-4159-4a5e-a284-174d634ae0c9","Type":"ContainerStarted","Data":"cad758397a00da188249fd8dd691a4427f55e022db13ebf82b1b9045ec11aa1f"}
Nov 01 05:03:22.554889 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:22.554858    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jswq7"
Nov 01 05:03:23.396285 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:23.396233    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk" event={"ID":"7fb489a8-8a4f-45f2-977b-594b6126281b","Type":"ContainerStarted","Data":"407a678596e9ddf4f83ff8ef0c0cfcb582ea2c55c61fae7c38d9786e8d4e7d8d"}
Nov 01 05:03:23.396749 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:23.396298    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-d8pkk" event={"ID":"7fb489a8-8a4f-45f2-977b-594b6126281b","Type":"ContainerStarted","Data":"7a07a8f760aa3e97f4594c6db91387cea76da1a8aa02101cb375ee9d8672f027"}
Nov 01 05:03:23.398844 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:23.398817    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rvg8t" event={"ID":"ef46e568-5c98-4520-99e5-c49c4c3aca2a","Type":"ContainerStarted","Data":"422b2669bf9ac508aca243cc803404e82f1ea5c95d4493a9d82b7449b5934adf"}
Nov 01 05:03:23.399033 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:23.399014    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rvg8t" event={"ID":"ef46e568-5c98-4520-99e5-c49c4c3aca2a","Type":"ContainerStarted","Data":"337d38e99458ff120d4ee711dffb4eb2837f215089b3099351bd5a90cb9b1831"}
Nov 01 05:03:23.504930 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:23.504850    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-89wdz"
Nov 01 05:03:24.406190 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:24.406098    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b6sl7" event={"ID":"7c9eec03-4159-4a5e-a284-174d634ae0c9","Type":"ContainerStarted","Data":"52838f78e64ac438a9982bbbf4a5dc26ed0de77f66fa5e1cf73803288dcdcee2"}
Nov 01 05:03:26.362185 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:26.362156    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wcrwt"
Nov 01 05:03:31.901851 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:31.901818    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tpp66"]
Nov 01 05:03:31.941130 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:31.941101    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.007906 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.007860    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 01 05:03:32.042269 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.042238    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 01 05:03:32.051650 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.051616    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 01 05:03:32.051773 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.051688    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-t567f\""
Nov 01 05:03:32.051845 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.051812    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 01 05:03:32.073480 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.073453    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2db64172-f3dd-498b-a505-47477423806c-node-exporter-tls\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.073630 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.073487    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2db64172-f3dd-498b-a505-47477423806c-node-exporter-textfile\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.073630 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.073525    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2db64172-f3dd-498b-a505-47477423806c-node-exporter-accelerators-collector-config\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.073755 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.073658    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2db64172-f3dd-498b-a505-47477423806c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.073755 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.073697    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kdl8\" (UniqueName: \"kubernetes.io/projected/2db64172-f3dd-498b-a505-47477423806c-kube-api-access-6kdl8\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.073755 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.073726    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2db64172-f3dd-498b-a505-47477423806c-root\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.073899 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.073774    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2db64172-f3dd-498b-a505-47477423806c-sys\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.073899 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.073872    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2db64172-f3dd-498b-a505-47477423806c-node-exporter-wtmp\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.074013 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.073900    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2db64172-f3dd-498b-a505-47477423806c-metrics-client-ca\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.175121 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.175027    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2db64172-f3dd-498b-a505-47477423806c-node-exporter-wtmp\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.175121 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.175067    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2db64172-f3dd-498b-a505-47477423806c-metrics-client-ca\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.175121 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.175106    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2db64172-f3dd-498b-a505-47477423806c-node-exporter-tls\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.175409 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.175126    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2db64172-f3dd-498b-a505-47477423806c-node-exporter-textfile\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.175409 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.175302    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2db64172-f3dd-498b-a505-47477423806c-node-exporter-wtmp\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.175409 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.175357    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2db64172-f3dd-498b-a505-47477423806c-node-exporter-accelerators-collector-config\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.175409 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.175392    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2db64172-f3dd-498b-a505-47477423806c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.175622 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.175417    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kdl8\" (UniqueName: \"kubernetes.io/projected/2db64172-f3dd-498b-a505-47477423806c-kube-api-access-6kdl8\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.175622 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.175443    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2db64172-f3dd-498b-a505-47477423806c-root\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.175622 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.175474    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2db64172-f3dd-498b-a505-47477423806c-sys\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.175622 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.175533    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2db64172-f3dd-498b-a505-47477423806c-node-exporter-textfile\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.175622 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.175600    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2db64172-f3dd-498b-a505-47477423806c-root\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.175915 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.175636    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2db64172-f3dd-498b-a505-47477423806c-sys\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.176234 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.176209    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2db64172-f3dd-498b-a505-47477423806c-metrics-client-ca\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.176418 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.176392    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2db64172-f3dd-498b-a505-47477423806c-node-exporter-accelerators-collector-config\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.178444 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.178419    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2db64172-f3dd-498b-a505-47477423806c-node-exporter-tls\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.178444 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.178437    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2db64172-f3dd-498b-a505-47477423806c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.236091 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.236059    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kdl8\" (UniqueName: \"kubernetes.io/projected/2db64172-f3dd-498b-a505-47477423806c-kube-api-access-6kdl8\") pod \"node-exporter-tpp66\" (UID: \"2db64172-f3dd-498b-a505-47477423806c\") " pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.289995 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.289957    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tpp66"
Nov 01 05:03:32.329812 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:03:32.329743    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2db64172_f3dd_498b_a505_47477423806c.slice/crio-0258f04a4e8e5cd7f0528d48021e2cc6274e0f7e06d6b6862a27b9fef91c7be8 WatchSource:0}: Error finding container 0258f04a4e8e5cd7f0528d48021e2cc6274e0f7e06d6b6862a27b9fef91c7be8: Status 404 returned error can't find the container with id 0258f04a4e8e5cd7f0528d48021e2cc6274e0f7e06d6b6862a27b9fef91c7be8
Nov 01 05:03:32.436371 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:32.436288    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tpp66" event={"ID":"2db64172-f3dd-498b-a505-47477423806c","Type":"ContainerStarted","Data":"0258f04a4e8e5cd7f0528d48021e2cc6274e0f7e06d6b6862a27b9fef91c7be8"}
Nov 01 05:03:34.451155 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:34.451077    4864 generic.go:356] "Generic (PLEG): container finished" podID="2db64172-f3dd-498b-a505-47477423806c" containerID="4d223ddf4c80dce211c259cd0acc5b582ac0f13fa5c4365c5b122df431f7264a" exitCode=0
Nov 01 05:03:34.451155 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:34.451143    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tpp66" event={"ID":"2db64172-f3dd-498b-a505-47477423806c","Type":"ContainerDied","Data":"4d223ddf4c80dce211c259cd0acc5b582ac0f13fa5c4365c5b122df431f7264a"}
Nov 01 05:03:35.457858 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:35.457824    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tpp66" event={"ID":"2db64172-f3dd-498b-a505-47477423806c","Type":"ContainerStarted","Data":"dc1f15a49989ec926033c4d9e4b13915730ccc35140f0259032874e0d7143c1e"}
Nov 01 05:03:35.458263 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:35.457877    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tpp66" event={"ID":"2db64172-f3dd-498b-a505-47477423806c","Type":"ContainerStarted","Data":"c572eafe0b837bf5a084faaad6aed3640eca0f6300ae2c86f4af4403b6e2e066"}
Nov 01 05:03:35.575437 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:35.575381    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tpp66" podStartSLOduration=2.692785741 podStartE2EDuration="4.575366813s" totalImagesPullingTime="1.793152865s" totalInitContainerRuntime="89.428207ms" isStatefulPod=true podCreationTimestamp="2026-11-01 05:03:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:03:35.574477476 +0000 UTC m=+32.464853452" watchObservedRunningTime="2026-11-01 05:03:35.575366813 +0000 UTC m=+32.465742788"
Nov 01 05:03:39.375918 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:39.375867    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tl74d"
Nov 01 05:03:42.393744 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:42.393663    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:03:42.505646 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:42.505599    4864 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jswq7_45a40b3e-ab72-4b18-933f-f52272a6742c/konnectivity-agent/0.log" limit=1024
Nov 01 05:03:42.750635 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:42.750547    4864 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jswq7_45a40b3e-ab72-4b18-933f-f52272a6742c/konnectivity-agent/0.log" limit=1024
Nov 01 05:03:43.105726 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:43.105683    4864 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jswq7_45a40b3e-ab72-4b18-933f-f52272a6742c/konnectivity-agent/0.log" limit=1024
Nov 01 05:03:43.405056 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:43.404981    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:44.220359 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:44.220324    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6969bc8dd6-85rsc"]
Nov 01 05:03:48.422562 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.422497    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-79659c8779-dnfbf" podUID="62652d56-1be6-47cc-a774-77d4fa9d206a" containerName="registry" containerID="cri-o://5d0ffedcbc5d00eefa11cea4f6f669e716369730ca622de216328c5da5e11f85" gracePeriod=30
Nov 01 05:03:48.654744 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.654720    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:48.701375 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.701303    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/62652d56-1be6-47cc-a774-77d4fa9d206a-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/62652d56-1be6-47cc-a774-77d4fa9d206a-trusted-ca\") pod \"62652d56-1be6-47cc-a774-77d4fa9d206a\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") "
Nov 01 05:03:48.701375 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.701348    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/62652d56-1be6-47cc-a774-77d4fa9d206a-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/62652d56-1be6-47cc-a774-77d4fa9d206a-installation-pull-secrets\") pod \"62652d56-1be6-47cc-a774-77d4fa9d206a\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") "
Nov 01 05:03:48.701375 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.701374    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-certificates\") pod \"62652d56-1be6-47cc-a774-77d4fa9d206a\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") "
Nov 01 05:03:48.701631 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.701397    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-kube-api-access-9n4xr\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-kube-api-access-9n4xr\") pod \"62652d56-1be6-47cc-a774-77d4fa9d206a\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") "
Nov 01 05:03:48.701631 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.701436    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls\") pod \"62652d56-1be6-47cc-a774-77d4fa9d206a\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") "
Nov 01 05:03:48.701631 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.701468    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/62652d56-1be6-47cc-a774-77d4fa9d206a-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/62652d56-1be6-47cc-a774-77d4fa9d206a-image-registry-private-configuration\") pod \"62652d56-1be6-47cc-a774-77d4fa9d206a\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") "
Nov 01 05:03:48.701631 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.701500    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62652d56-1be6-47cc-a774-77d4fa9d206a-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/62652d56-1be6-47cc-a774-77d4fa9d206a-ca-trust-extracted\") pod \"62652d56-1be6-47cc-a774-77d4fa9d206a\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") "
Nov 01 05:03:48.701631 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.701527    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-bound-sa-token\") pod \"62652d56-1be6-47cc-a774-77d4fa9d206a\" (UID: \"62652d56-1be6-47cc-a774-77d4fa9d206a\") "
Nov 01 05:03:48.702471 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.701769    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62652d56-1be6-47cc-a774-77d4fa9d206a-trusted-ca" pod "62652d56-1be6-47cc-a774-77d4fa9d206a" (UID: "62652d56-1be6-47cc-a774-77d4fa9d206a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:03:48.702471 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.701957    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-certificates" pod "62652d56-1be6-47cc-a774-77d4fa9d206a" (UID: "62652d56-1be6-47cc-a774-77d4fa9d206a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:03:48.704091 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.704061    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62652d56-1be6-47cc-a774-77d4fa9d206a-installation-pull-secrets" pod "62652d56-1be6-47cc-a774-77d4fa9d206a" (UID: "62652d56-1be6-47cc-a774-77d4fa9d206a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:03:48.704488 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.704456    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62652d56-1be6-47cc-a774-77d4fa9d206a-image-registry-private-configuration" pod "62652d56-1be6-47cc-a774-77d4fa9d206a" (UID: "62652d56-1be6-47cc-a774-77d4fa9d206a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:03:48.704598 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.704552    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-kube-api-access-9n4xr" pod "62652d56-1be6-47cc-a774-77d4fa9d206a" (UID: "62652d56-1be6-47cc-a774-77d4fa9d206a"). InnerVolumeSpecName "kube-api-access-9n4xr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:03:48.704668 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.704628    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls" pod "62652d56-1be6-47cc-a774-77d4fa9d206a" (UID: "62652d56-1be6-47cc-a774-77d4fa9d206a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:03:48.704668 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.704637    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-bound-sa-token" pod "62652d56-1be6-47cc-a774-77d4fa9d206a" (UID: "62652d56-1be6-47cc-a774-77d4fa9d206a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:03:48.708203 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.708177    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62652d56-1be6-47cc-a774-77d4fa9d206a-ca-trust-extracted" pod "62652d56-1be6-47cc-a774-77d4fa9d206a" (UID: "62652d56-1be6-47cc-a774-77d4fa9d206a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:03:48.802086 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.802049    4864 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/62652d56-1be6-47cc-a774-77d4fa9d206a-installation-pull-secrets\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:03:48.802086 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.802079    4864 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-certificates\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:03:48.802086 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.802092    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9n4xr\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-kube-api-access-9n4xr\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:03:48.802324 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.802105    4864 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-registry-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:03:48.802324 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.802118    4864 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/62652d56-1be6-47cc-a774-77d4fa9d206a-image-registry-private-configuration\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:03:48.802324 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.802130    4864 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/62652d56-1be6-47cc-a774-77d4fa9d206a-ca-trust-extracted\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:03:48.802324 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.802143    4864 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/62652d56-1be6-47cc-a774-77d4fa9d206a-bound-sa-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:03:48.802324 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:48.802154    4864 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/62652d56-1be6-47cc-a774-77d4fa9d206a-trusted-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:03:49.517821 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:49.517727    4864 generic.go:356] "Generic (PLEG): container finished" podID="62652d56-1be6-47cc-a774-77d4fa9d206a" containerID="5d0ffedcbc5d00eefa11cea4f6f669e716369730ca622de216328c5da5e11f85" exitCode=0
Nov 01 05:03:49.517821 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:49.517769    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79659c8779-dnfbf" event={"ID":"62652d56-1be6-47cc-a774-77d4fa9d206a","Type":"ContainerDied","Data":"5d0ffedcbc5d00eefa11cea4f6f669e716369730ca622de216328c5da5e11f85"}
Nov 01 05:03:49.517821 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:49.517774    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79659c8779-dnfbf"
Nov 01 05:03:49.518300 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:49.517839    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79659c8779-dnfbf" event={"ID":"62652d56-1be6-47cc-a774-77d4fa9d206a","Type":"ContainerDied","Data":"a510523a1afbc8d54ebf4c7c937335c511b50b1a2f091e97125b8083f514755a"}
Nov 01 05:03:49.518300 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:49.517861    4864 scope.go:122] "RemoveContainer" containerID="5d0ffedcbc5d00eefa11cea4f6f669e716369730ca622de216328c5da5e11f85"
Nov 01 05:03:49.526036 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:49.526013    4864 scope.go:122] "RemoveContainer" containerID="5d0ffedcbc5d00eefa11cea4f6f669e716369730ca622de216328c5da5e11f85"
Nov 01 05:03:49.526301 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:49.526281    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d0ffedcbc5d00eefa11cea4f6f669e716369730ca622de216328c5da5e11f85\": container with ID starting with 5d0ffedcbc5d00eefa11cea4f6f669e716369730ca622de216328c5da5e11f85 not found: ID does not exist" containerID="5d0ffedcbc5d00eefa11cea4f6f669e716369730ca622de216328c5da5e11f85"
Nov 01 05:03:49.526370 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:49.526315    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d0ffedcbc5d00eefa11cea4f6f669e716369730ca622de216328c5da5e11f85"} err="failed to get container status \"5d0ffedcbc5d00eefa11cea4f6f669e716369730ca622de216328c5da5e11f85\": rpc error: code = NotFound desc = could not find container \"5d0ffedcbc5d00eefa11cea4f6f669e716369730ca622de216328c5da5e11f85\": container with ID starting with 5d0ffedcbc5d00eefa11cea4f6f669e716369730ca622de216328c5da5e11f85 not found: ID does not exist"
Nov 01 05:03:49.729946 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:49.729911    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-79659c8779-dnfbf"]
Nov 01 05:03:49.733223 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:49.733195    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-79659c8779-dnfbf"]
Nov 01 05:03:51.244732 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:03:51.244702    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="62652d56-1be6-47cc-a774-77d4fa9d206a" path="/var/lib/kubelet/pods/62652d56-1be6-47cc-a774-77d4fa9d206a/volumes"
Nov 01 05:03:52.524013 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:03:52.523979    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:04:09.245996 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.245936    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc" podUID="48c05e4f-a673-4f5f-8c21-8dc1ff716683" containerName="registry" containerID="cri-o://da8859d512bfebf4ab55f7244ca0c10119cd71623c84c32858eae93c3f16ac4f" gracePeriod=30
Nov 01 05:04:09.500113 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.500088    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:04:09.573069 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.573035    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/48c05e4f-a673-4f5f-8c21-8dc1ff716683-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48c05e4f-a673-4f5f-8c21-8dc1ff716683-trusted-ca\") pod \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") "
Nov 01 05:04:09.573242 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.573077    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-bound-sa-token\") pod \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") "
Nov 01 05:04:09.573242 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.573111    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-certificates\") pod \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") "
Nov 01 05:04:09.573242 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.573144    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/48c05e4f-a673-4f5f-8c21-8dc1ff716683-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48c05e4f-a673-4f5f-8c21-8dc1ff716683-installation-pull-secrets\") pod \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") "
Nov 01 05:04:09.573242 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.573181    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls\") pod \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") "
Nov 01 05:04:09.573242 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.573212    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/48c05e4f-a673-4f5f-8c21-8dc1ff716683-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48c05e4f-a673-4f5f-8c21-8dc1ff716683-image-registry-private-configuration\") pod \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") "
Nov 01 05:04:09.573488 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.573278    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48c05e4f-a673-4f5f-8c21-8dc1ff716683-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48c05e4f-a673-4f5f-8c21-8dc1ff716683-ca-trust-extracted\") pod \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") "
Nov 01 05:04:09.573488 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.573355    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-kube-api-access-rdspf\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-kube-api-access-rdspf\") pod \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\" (UID: \"48c05e4f-a673-4f5f-8c21-8dc1ff716683\") "
Nov 01 05:04:09.573594 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.573509    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48c05e4f-a673-4f5f-8c21-8dc1ff716683-trusted-ca" pod "48c05e4f-a673-4f5f-8c21-8dc1ff716683" (UID: "48c05e4f-a673-4f5f-8c21-8dc1ff716683"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:04:09.573594 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.573566    4864 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48c05e4f-a673-4f5f-8c21-8dc1ff716683-trusted-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:04:09.573695 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.573585    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-certificates" pod "48c05e4f-a673-4f5f-8c21-8dc1ff716683" (UID: "48c05e4f-a673-4f5f-8c21-8dc1ff716683"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:04:09.575889 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.575838    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls" pod "48c05e4f-a673-4f5f-8c21-8dc1ff716683" (UID: "48c05e4f-a673-4f5f-8c21-8dc1ff716683"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:04:09.576017 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.575922    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48c05e4f-a673-4f5f-8c21-8dc1ff716683-installation-pull-secrets" pod "48c05e4f-a673-4f5f-8c21-8dc1ff716683" (UID: "48c05e4f-a673-4f5f-8c21-8dc1ff716683"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:04:09.576203 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.576170    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-kube-api-access-rdspf" pod "48c05e4f-a673-4f5f-8c21-8dc1ff716683" (UID: "48c05e4f-a673-4f5f-8c21-8dc1ff716683"). InnerVolumeSpecName "kube-api-access-rdspf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:04:09.576252 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.576222    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48c05e4f-a673-4f5f-8c21-8dc1ff716683-image-registry-private-configuration" pod "48c05e4f-a673-4f5f-8c21-8dc1ff716683" (UID: "48c05e4f-a673-4f5f-8c21-8dc1ff716683"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:04:09.576311 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.576288    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-bound-sa-token" pod "48c05e4f-a673-4f5f-8c21-8dc1ff716683" (UID: "48c05e4f-a673-4f5f-8c21-8dc1ff716683"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:04:09.580543 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.580518    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48c05e4f-a673-4f5f-8c21-8dc1ff716683-ca-trust-extracted" pod "48c05e4f-a673-4f5f-8c21-8dc1ff716683" (UID: "48c05e4f-a673-4f5f-8c21-8dc1ff716683"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:04:09.590478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.590456    4864 generic.go:356] "Generic (PLEG): container finished" podID="636cccf5-2397-4dca-899a-3598a7b8096b" containerID="a1dfbb52ff2fe274c4d3b3679a0eb12b7fb4994e68479f0aacc58ccff75a48cd" exitCode=0
Nov 01 05:04:09.590568 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.590484    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz" event={"ID":"636cccf5-2397-4dca-899a-3598a7b8096b","Type":"ContainerDied","Data":"a1dfbb52ff2fe274c4d3b3679a0eb12b7fb4994e68479f0aacc58ccff75a48cd"}
Nov 01 05:04:09.590849 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.590834    4864 scope.go:122] "RemoveContainer" containerID="a1dfbb52ff2fe274c4d3b3679a0eb12b7fb4994e68479f0aacc58ccff75a48cd"
Nov 01 05:04:09.591804 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.591758    4864 generic.go:356] "Generic (PLEG): container finished" podID="48c05e4f-a673-4f5f-8c21-8dc1ff716683" containerID="da8859d512bfebf4ab55f7244ca0c10119cd71623c84c32858eae93c3f16ac4f" exitCode=0
Nov 01 05:04:09.591905 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.591814    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc" event={"ID":"48c05e4f-a673-4f5f-8c21-8dc1ff716683","Type":"ContainerDied","Data":"da8859d512bfebf4ab55f7244ca0c10119cd71623c84c32858eae93c3f16ac4f"}
Nov 01 05:04:09.591905 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.591849    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc" event={"ID":"48c05e4f-a673-4f5f-8c21-8dc1ff716683","Type":"ContainerDied","Data":"d5fea4e8ea2de0409cf2f7beea639dec8e1836ecd921f816450ec5a596d9a39d"}
Nov 01 05:04:09.591905 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.591867    4864 scope.go:122] "RemoveContainer" containerID="da8859d512bfebf4ab55f7244ca0c10119cd71623c84c32858eae93c3f16ac4f"
Nov 01 05:04:09.591905 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.591819    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6969bc8dd6-85rsc"
Nov 01 05:04:09.600361 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.600345    4864 scope.go:122] "RemoveContainer" containerID="da8859d512bfebf4ab55f7244ca0c10119cd71623c84c32858eae93c3f16ac4f"
Nov 01 05:04:09.600728 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:04:09.600709    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da8859d512bfebf4ab55f7244ca0c10119cd71623c84c32858eae93c3f16ac4f\": container with ID starting with da8859d512bfebf4ab55f7244ca0c10119cd71623c84c32858eae93c3f16ac4f not found: ID does not exist" containerID="da8859d512bfebf4ab55f7244ca0c10119cd71623c84c32858eae93c3f16ac4f"
Nov 01 05:04:09.600829 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.600735    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da8859d512bfebf4ab55f7244ca0c10119cd71623c84c32858eae93c3f16ac4f"} err="failed to get container status \"da8859d512bfebf4ab55f7244ca0c10119cd71623c84c32858eae93c3f16ac4f\": rpc error: code = NotFound desc = could not find container \"da8859d512bfebf4ab55f7244ca0c10119cd71623c84c32858eae93c3f16ac4f\": container with ID starting with da8859d512bfebf4ab55f7244ca0c10119cd71623c84c32858eae93c3f16ac4f not found: ID does not exist"
Nov 01 05:04:09.675449 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.674867    4864 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48c05e4f-a673-4f5f-8c21-8dc1ff716683-ca-trust-extracted\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:04:09.675449 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.674907    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdspf\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-kube-api-access-rdspf\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:04:09.675449 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.674926    4864 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-bound-sa-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:04:09.675449 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.674943    4864 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-certificates\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:04:09.675449 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.674962    4864 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48c05e4f-a673-4f5f-8c21-8dc1ff716683-installation-pull-secrets\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:04:09.675449 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.674982    4864 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48c05e4f-a673-4f5f-8c21-8dc1ff716683-registry-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:04:09.675449 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.674999    4864 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48c05e4f-a673-4f5f-8c21-8dc1ff716683-image-registry-private-configuration\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:04:09.936434 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.936356    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6969bc8dd6-85rsc"]
Nov 01 05:04:09.940258 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:09.940234    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6969bc8dd6-85rsc"]
Nov 01 05:04:10.597986 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:10.597945    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-vh8xz" event={"ID":"636cccf5-2397-4dca-899a-3598a7b8096b","Type":"ContainerStarted","Data":"be626128bfb953981613f7af946ddaedd680b277da24745ceade359c0af737e4"}
Nov 01 05:04:11.244657 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:11.244624    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="48c05e4f-a673-4f5f-8c21-8dc1ff716683" path="/var/lib/kubelet/pods/48c05e4f-a673-4f5f-8c21-8dc1ff716683/volumes"
Nov 01 05:04:22.526176 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:04:22.526135    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:04:24.649699 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:24.649660    4864 generic.go:356] "Generic (PLEG): container finished" podID="1967227b-a4ea-4ee6-945d-efb0b5040a13" containerID="89b65be52ff10c382485b53bbed1d83d875379e0e2807574783c565d4fea6754" exitCode=0
Nov 01 05:04:24.650094 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:24.649739    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-cvn58" event={"ID":"1967227b-a4ea-4ee6-945d-efb0b5040a13","Type":"ContainerDied","Data":"89b65be52ff10c382485b53bbed1d83d875379e0e2807574783c565d4fea6754"}
Nov 01 05:04:24.650152 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:24.650131    4864 scope.go:122] "RemoveContainer" containerID="89b65be52ff10c382485b53bbed1d83d875379e0e2807574783c565d4fea6754"
Nov 01 05:04:25.655058 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:25.655021    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-cvn58" event={"ID":"1967227b-a4ea-4ee6-945d-efb0b5040a13","Type":"ContainerStarted","Data":"7eae668963789e5914995e72085823c78845738c0ef57b15066333a1eb97ca94"}
Nov 01 05:04:34.688072 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:34.688042    4864 generic.go:356] "Generic (PLEG): container finished" podID="bce36747-2159-4b72-8c0e-d98226588d18" containerID="78316fc3e2e1580ee4441b2fe703f1ae50bcd7fc40646f1152ca1c8644fc7e07" exitCode=0
Nov 01 05:04:34.688391 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:34.688096    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27" event={"ID":"bce36747-2159-4b72-8c0e-d98226588d18","Type":"ContainerDied","Data":"78316fc3e2e1580ee4441b2fe703f1ae50bcd7fc40646f1152ca1c8644fc7e07"}
Nov 01 05:04:34.688432 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:34.688388    4864 scope.go:122] "RemoveContainer" containerID="78316fc3e2e1580ee4441b2fe703f1ae50bcd7fc40646f1152ca1c8644fc7e07"
Nov 01 05:04:35.694067 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:04:35.694034    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-97k27" event={"ID":"bce36747-2159-4b72-8c0e-d98226588d18","Type":"ContainerStarted","Data":"a4fb18bbada01e6e7a950025a6edb153e559cbabc635724c6421d6e52b6ac856"}
Nov 01 05:04:52.527247 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:04:52.527199    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:05:22.522693 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:05:22.522590    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:05:52.523489 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:05:52.523450    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:06:22.523275 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:06:22.523232    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:06:52.522662 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:06:52.522578    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:07:22.523248 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:07:22.523206    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:07:44.845287 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:44.845238    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8"]
Nov 01 05:07:44.845758 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:44.845598    4864 cpu_manager.go:414] "containerMap: removing container" podUID="48c05e4f-a673-4f5f-8c21-8dc1ff716683" containerName="registry"
Nov 01 05:07:44.845758 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:44.845611    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48c05e4f-a673-4f5f-8c21-8dc1ff716683" containerName="registry"
Nov 01 05:07:44.845758 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:44.845625    4864 cpu_manager.go:414] "containerMap: removing container" podUID="62652d56-1be6-47cc-a774-77d4fa9d206a" containerName="registry"
Nov 01 05:07:44.845758 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:44.845630    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="62652d56-1be6-47cc-a774-77d4fa9d206a" containerName="registry"
Nov 01 05:07:44.845758 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:44.845686    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="48c05e4f-a673-4f5f-8c21-8dc1ff716683" containerName="registry"
Nov 01 05:07:44.845758 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:44.845695    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="62652d56-1be6-47cc-a774-77d4fa9d206a" containerName="registry"
Nov 01 05:07:44.848823 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:44.848803    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8"
Nov 01 05:07:44.950433 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:44.950399    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 01 05:07:44.950433 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:44.950412    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 01 05:07:44.950671 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:44.950489    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-tsnmb\""
Nov 01 05:07:44.951576 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:44.951560    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8"]
Nov 01 05:07:44.966987 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:44.966961    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b"]
Nov 01 05:07:44.967210 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:44.967190    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a52e528e-c723-42b6-8d1a-03c3675518b9-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8\" (UID: \"a52e528e-c723-42b6-8d1a-03c3675518b9\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8"
Nov 01 05:07:44.967287 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:44.967256    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a52e528e-c723-42b6-8d1a-03c3675518b9-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8\" (UID: \"a52e528e-c723-42b6-8d1a-03c3675518b9\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8"
Nov 01 05:07:44.967348 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:44.967317    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zww6r\" (UniqueName: \"kubernetes.io/projected/a52e528e-c723-42b6-8d1a-03c3675518b9-kube-api-access-zww6r\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8\" (UID: \"a52e528e-c723-42b6-8d1a-03c3675518b9\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8"
Nov 01 05:07:44.970417 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:44.970396    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b"
Nov 01 05:07:45.039633 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:45.039596    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b"]
Nov 01 05:07:45.068380 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:45.068338    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/50f35a8f-80ec-41e3-9255-e1358ca8ca52-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b\" (UID: \"50f35a8f-80ec-41e3-9255-e1358ca8ca52\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b"
Nov 01 05:07:45.068595 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:45.068390    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/50f35a8f-80ec-41e3-9255-e1358ca8ca52-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b\" (UID: \"50f35a8f-80ec-41e3-9255-e1358ca8ca52\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b"
Nov 01 05:07:45.068595 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:45.068463    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kn57\" (UniqueName: \"kubernetes.io/projected/50f35a8f-80ec-41e3-9255-e1358ca8ca52-kube-api-access-9kn57\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b\" (UID: \"50f35a8f-80ec-41e3-9255-e1358ca8ca52\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b"
Nov 01 05:07:45.068595 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:45.068508    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a52e528e-c723-42b6-8d1a-03c3675518b9-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8\" (UID: \"a52e528e-c723-42b6-8d1a-03c3675518b9\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8"
Nov 01 05:07:45.068595 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:45.068526    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zww6r\" (UniqueName: \"kubernetes.io/projected/a52e528e-c723-42b6-8d1a-03c3675518b9-kube-api-access-zww6r\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8\" (UID: \"a52e528e-c723-42b6-8d1a-03c3675518b9\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8"
Nov 01 05:07:45.068595 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:45.068566    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a52e528e-c723-42b6-8d1a-03c3675518b9-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8\" (UID: \"a52e528e-c723-42b6-8d1a-03c3675518b9\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8"
Nov 01 05:07:45.069009 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:45.068989    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a52e528e-c723-42b6-8d1a-03c3675518b9-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8\" (UID: \"a52e528e-c723-42b6-8d1a-03c3675518b9\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8"
Nov 01 05:07:45.069057 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:45.069010    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a52e528e-c723-42b6-8d1a-03c3675518b9-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8\" (UID: \"a52e528e-c723-42b6-8d1a-03c3675518b9\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8"
Nov 01 05:07:45.102320 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:45.102244    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-kc24v\""
Nov 01 05:07:45.129615 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:45.129581    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zww6r\" (UniqueName: \"kubernetes.io/projected/a52e528e-c723-42b6-8d1a-03c3675518b9-kube-api-access-zww6r\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8\" (UID: \"a52e528e-c723-42b6-8d1a-03c3675518b9\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8"
Nov 01 05:07:45.164265 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:45.164226    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8"
Nov 01 05:07:45.169536 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:45.169506    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/50f35a8f-80ec-41e3-9255-e1358ca8ca52-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b\" (UID: \"50f35a8f-80ec-41e3-9255-e1358ca8ca52\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b"
Nov 01 05:07:45.169652 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:45.169552    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/50f35a8f-80ec-41e3-9255-e1358ca8ca52-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b\" (UID: \"50f35a8f-80ec-41e3-9255-e1358ca8ca52\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b"
Nov 01 05:07:45.169652 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:45.169575    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kn57\" (UniqueName: \"kubernetes.io/projected/50f35a8f-80ec-41e3-9255-e1358ca8ca52-kube-api-access-9kn57\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b\" (UID: \"50f35a8f-80ec-41e3-9255-e1358ca8ca52\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b"
Nov 01 05:07:45.169909 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:45.169892    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/50f35a8f-80ec-41e3-9255-e1358ca8ca52-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b\" (UID: \"50f35a8f-80ec-41e3-9255-e1358ca8ca52\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b"
Nov 01 05:07:45.169952 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:45.169932    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/50f35a8f-80ec-41e3-9255-e1358ca8ca52-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b\" (UID: \"50f35a8f-80ec-41e3-9255-e1358ca8ca52\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b"
Nov 01 05:07:45.230614 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:45.230583    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kn57\" (UniqueName: \"kubernetes.io/projected/50f35a8f-80ec-41e3-9255-e1358ca8ca52-kube-api-access-9kn57\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b\" (UID: \"50f35a8f-80ec-41e3-9255-e1358ca8ca52\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b"
Nov 01 05:07:45.285237 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:45.285197    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b"
Nov 01 05:07:45.536460 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:45.536381    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8"]
Nov 01 05:07:45.540495 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:07:45.540470    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda52e528e_c723_42b6_8d1a_03c3675518b9.slice/crio-b320d3c79fbd8723ca22866c7318750b0d099e961798972fd544f262ceb65f9c WatchSource:0}: Error finding container b320d3c79fbd8723ca22866c7318750b0d099e961798972fd544f262ceb65f9c: Status 404 returned error can't find the container with id b320d3c79fbd8723ca22866c7318750b0d099e961798972fd544f262ceb65f9c
Nov 01 05:07:45.659944 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:45.659909    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b"]
Nov 01 05:07:45.662839 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:07:45.662813    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50f35a8f_80ec_41e3_9255_e1358ca8ca52.slice/crio-b41e3a7a85ff221750ac1ce9fd85882ba9ffe754f267d4e5b77e33339a72b973 WatchSource:0}: Error finding container b41e3a7a85ff221750ac1ce9fd85882ba9ffe754f267d4e5b77e33339a72b973: Status 404 returned error can't find the container with id b41e3a7a85ff221750ac1ce9fd85882ba9ffe754f267d4e5b77e33339a72b973
Nov 01 05:07:46.347603 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:46.347564    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b" event={"ID":"50f35a8f-80ec-41e3-9255-e1358ca8ca52","Type":"ContainerStarted","Data":"b41e3a7a85ff221750ac1ce9fd85882ba9ffe754f267d4e5b77e33339a72b973"}
Nov 01 05:07:46.348748 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:46.348725    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8" event={"ID":"a52e528e-c723-42b6-8d1a-03c3675518b9","Type":"ContainerStarted","Data":"b320d3c79fbd8723ca22866c7318750b0d099e961798972fd544f262ceb65f9c"}
Nov 01 05:07:46.911955 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:46.911917    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr"]
Nov 01 05:07:46.915775 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:46.915748    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr"
Nov 01 05:07:47.018846 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:47.018807    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr"]
Nov 01 05:07:47.019056 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:47.018956    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-bdfh7\""
Nov 01 05:07:47.086989 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:47.086946    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr\" (UID: \"1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr"
Nov 01 05:07:47.087195 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:47.087031    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr\" (UID: \"1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr"
Nov 01 05:07:47.087195 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:47.087129    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkpbn\" (UniqueName: \"kubernetes.io/projected/1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf-kube-api-access-qkpbn\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr\" (UID: \"1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr"
Nov 01 05:07:47.188385 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:47.188292    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr\" (UID: \"1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr"
Nov 01 05:07:47.188552 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:47.188488    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr\" (UID: \"1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr"
Nov 01 05:07:47.188552 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:47.188533    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkpbn\" (UniqueName: \"kubernetes.io/projected/1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf-kube-api-access-qkpbn\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr\" (UID: \"1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr"
Nov 01 05:07:47.188755 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:47.188724    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr\" (UID: \"1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr"
Nov 01 05:07:47.189030 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:47.189002    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr\" (UID: \"1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr"
Nov 01 05:07:47.250130 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:47.250102    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkpbn\" (UniqueName: \"kubernetes.io/projected/1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf-kube-api-access-qkpbn\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr\" (UID: \"1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr"
Nov 01 05:07:47.534658 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:47.534575    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr"
Nov 01 05:07:47.905470 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:47.905441    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr"]
Nov 01 05:07:47.909151 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:07:47.909120    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e13f47f_2e8e_47e0_8eba_1c2a4f17ffbf.slice/crio-7b7f47a2a9e68811e543ab41cb543c0ac59959f756a0faa5f28ec0f54791824b WatchSource:0}: Error finding container 7b7f47a2a9e68811e543ab41cb543c0ac59959f756a0faa5f28ec0f54791824b: Status 404 returned error can't find the container with id 7b7f47a2a9e68811e543ab41cb543c0ac59959f756a0faa5f28ec0f54791824b
Nov 01 05:07:48.360069 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:48.359968    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr" event={"ID":"1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf","Type":"ContainerStarted","Data":"7b7f47a2a9e68811e543ab41cb543c0ac59959f756a0faa5f28ec0f54791824b"}
Nov 01 05:07:49.005654 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.005620    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5"]
Nov 01 05:07:49.009734 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.009699    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5"
Nov 01 05:07:49.062454 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.061825    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-4482w\""
Nov 01 05:07:49.068460 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.068428    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5"]
Nov 01 05:07:49.105139 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.105103    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf"]
Nov 01 05:07:49.107216 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.107182    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp62m\" (UniqueName: \"kubernetes.io/projected/b47e4d74-35df-4229-88f2-7d775d66ba9c-kube-api-access-zp62m\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5\" (UID: \"b47e4d74-35df-4229-88f2-7d775d66ba9c\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5"
Nov 01 05:07:49.107358 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.107227    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b47e4d74-35df-4229-88f2-7d775d66ba9c-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5\" (UID: \"b47e4d74-35df-4229-88f2-7d775d66ba9c\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5"
Nov 01 05:07:49.107358 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.107326    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b47e4d74-35df-4229-88f2-7d775d66ba9c-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5\" (UID: \"b47e4d74-35df-4229-88f2-7d775d66ba9c\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5"
Nov 01 05:07:49.109108 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.109085    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf"
Nov 01 05:07:49.162091 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.162043    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-9rd4x\""
Nov 01 05:07:49.167203 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.167173    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf"]
Nov 01 05:07:49.202152 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.202115    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj"]
Nov 01 05:07:49.206371 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.205967    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj"
Nov 01 05:07:49.207927 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.207901    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a5ceca4-7882-4c74-9758-45e4bb783b66-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf\" (UID: \"4a5ceca4-7882-4c74-9758-45e4bb783b66\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf"
Nov 01 05:07:49.208061 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.207968    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a5ceca4-7882-4c74-9758-45e4bb783b66-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf\" (UID: \"4a5ceca4-7882-4c74-9758-45e4bb783b66\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf"
Nov 01 05:07:49.208061 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.208040    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp62m\" (UniqueName: \"kubernetes.io/projected/b47e4d74-35df-4229-88f2-7d775d66ba9c-kube-api-access-zp62m\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5\" (UID: \"b47e4d74-35df-4229-88f2-7d775d66ba9c\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5"
Nov 01 05:07:49.210835 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.210575    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b47e4d74-35df-4229-88f2-7d775d66ba9c-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5\" (UID: \"b47e4d74-35df-4229-88f2-7d775d66ba9c\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5"
Nov 01 05:07:49.210835 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.210616    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml2pl\" (UniqueName: \"kubernetes.io/projected/4a5ceca4-7882-4c74-9758-45e4bb783b66-kube-api-access-ml2pl\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf\" (UID: \"4a5ceca4-7882-4c74-9758-45e4bb783b66\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf"
Nov 01 05:07:49.210835 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.210766    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b47e4d74-35df-4229-88f2-7d775d66ba9c-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5\" (UID: \"b47e4d74-35df-4229-88f2-7d775d66ba9c\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5"
Nov 01 05:07:49.211174 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.211120    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b47e4d74-35df-4229-88f2-7d775d66ba9c-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5\" (UID: \"b47e4d74-35df-4229-88f2-7d775d66ba9c\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5"
Nov 01 05:07:49.211557 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.211498    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b47e4d74-35df-4229-88f2-7d775d66ba9c-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5\" (UID: \"b47e4d74-35df-4229-88f2-7d775d66ba9c\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5"
Nov 01 05:07:49.262983 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.262887    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-w2hl6\""
Nov 01 05:07:49.266512 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.266477    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj"]
Nov 01 05:07:49.274382 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.274292    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp62m\" (UniqueName: \"kubernetes.io/projected/b47e4d74-35df-4229-88f2-7d775d66ba9c-kube-api-access-zp62m\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5\" (UID: \"b47e4d74-35df-4229-88f2-7d775d66ba9c\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5"
Nov 01 05:07:49.304201 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.304167    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz"]
Nov 01 05:07:49.307886 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.307860    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz"
Nov 01 05:07:49.312410 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.312368    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a5ceca4-7882-4c74-9758-45e4bb783b66-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf\" (UID: \"4a5ceca4-7882-4c74-9758-45e4bb783b66\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf"
Nov 01 05:07:49.312410 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.312407    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a5ceca4-7882-4c74-9758-45e4bb783b66-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf\" (UID: \"4a5ceca4-7882-4c74-9758-45e4bb783b66\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf"
Nov 01 05:07:49.312609 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.312442    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/967c127b-a8a7-41cc-aef7-96c2cc3c36cb-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj\" (UID: \"967c127b-a8a7-41cc-aef7-96c2cc3c36cb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj"
Nov 01 05:07:49.312609 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.312476    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4b4r\" (UniqueName: \"kubernetes.io/projected/967c127b-a8a7-41cc-aef7-96c2cc3c36cb-kube-api-access-t4b4r\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj\" (UID: \"967c127b-a8a7-41cc-aef7-96c2cc3c36cb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj"
Nov 01 05:07:49.312609 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.312521    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml2pl\" (UniqueName: \"kubernetes.io/projected/4a5ceca4-7882-4c74-9758-45e4bb783b66-kube-api-access-ml2pl\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf\" (UID: \"4a5ceca4-7882-4c74-9758-45e4bb783b66\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf"
Nov 01 05:07:49.312609 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.312567    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/967c127b-a8a7-41cc-aef7-96c2cc3c36cb-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj\" (UID: \"967c127b-a8a7-41cc-aef7-96c2cc3c36cb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj"
Nov 01 05:07:49.313089 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.313065    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a5ceca4-7882-4c74-9758-45e4bb783b66-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf\" (UID: \"4a5ceca4-7882-4c74-9758-45e4bb783b66\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf"
Nov 01 05:07:49.313177 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.313075    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a5ceca4-7882-4c74-9758-45e4bb783b66-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf\" (UID: \"4a5ceca4-7882-4c74-9758-45e4bb783b66\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf"
Nov 01 05:07:49.330801 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.330759    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5"
Nov 01 05:07:49.363949 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.363574    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-sbtvg\""
Nov 01 05:07:49.366389 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.366318    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz"]
Nov 01 05:07:49.380383 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.376602    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml2pl\" (UniqueName: \"kubernetes.io/projected/4a5ceca4-7882-4c74-9758-45e4bb783b66-kube-api-access-ml2pl\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf\" (UID: \"4a5ceca4-7882-4c74-9758-45e4bb783b66\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf"
Nov 01 05:07:49.413982 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.413704    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/967c127b-a8a7-41cc-aef7-96c2cc3c36cb-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj\" (UID: \"967c127b-a8a7-41cc-aef7-96c2cc3c36cb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj"
Nov 01 05:07:49.413982 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.413737    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4b4r\" (UniqueName: \"kubernetes.io/projected/967c127b-a8a7-41cc-aef7-96c2cc3c36cb-kube-api-access-t4b4r\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj\" (UID: \"967c127b-a8a7-41cc-aef7-96c2cc3c36cb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj"
Nov 01 05:07:49.413982 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.413768    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/14396006-01c5-4daa-a0f2-8c863285b5bd-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz\" (UID: \"14396006-01c5-4daa-a0f2-8c863285b5bd\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz"
Nov 01 05:07:49.413982 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.413814    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz4t4\" (UniqueName: \"kubernetes.io/projected/14396006-01c5-4daa-a0f2-8c863285b5bd-kube-api-access-wz4t4\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz\" (UID: \"14396006-01c5-4daa-a0f2-8c863285b5bd\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz"
Nov 01 05:07:49.413982 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.413835    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/967c127b-a8a7-41cc-aef7-96c2cc3c36cb-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj\" (UID: \"967c127b-a8a7-41cc-aef7-96c2cc3c36cb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj"
Nov 01 05:07:49.413982 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.413865    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/14396006-01c5-4daa-a0f2-8c863285b5bd-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz\" (UID: \"14396006-01c5-4daa-a0f2-8c863285b5bd\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz"
Nov 01 05:07:49.414423 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.414223    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/967c127b-a8a7-41cc-aef7-96c2cc3c36cb-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj\" (UID: \"967c127b-a8a7-41cc-aef7-96c2cc3c36cb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj"
Nov 01 05:07:49.414423 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.414403    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/967c127b-a8a7-41cc-aef7-96c2cc3c36cb-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj\" (UID: \"967c127b-a8a7-41cc-aef7-96c2cc3c36cb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj"
Nov 01 05:07:49.428645 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.428611    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf"
Nov 01 05:07:49.477857 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.477826    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4b4r\" (UniqueName: \"kubernetes.io/projected/967c127b-a8a7-41cc-aef7-96c2cc3c36cb-kube-api-access-t4b4r\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj\" (UID: \"967c127b-a8a7-41cc-aef7-96c2cc3c36cb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj"
Nov 01 05:07:49.517611 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.515605    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/14396006-01c5-4daa-a0f2-8c863285b5bd-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz\" (UID: \"14396006-01c5-4daa-a0f2-8c863285b5bd\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz"
Nov 01 05:07:49.517611 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.515860    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz4t4\" (UniqueName: \"kubernetes.io/projected/14396006-01c5-4daa-a0f2-8c863285b5bd-kube-api-access-wz4t4\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz\" (UID: \"14396006-01c5-4daa-a0f2-8c863285b5bd\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz"
Nov 01 05:07:49.517611 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.515933    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/14396006-01c5-4daa-a0f2-8c863285b5bd-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz\" (UID: \"14396006-01c5-4daa-a0f2-8c863285b5bd\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz"
Nov 01 05:07:49.517611 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.516826    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/14396006-01c5-4daa-a0f2-8c863285b5bd-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz\" (UID: \"14396006-01c5-4daa-a0f2-8c863285b5bd\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz"
Nov 01 05:07:49.517611 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.517141    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/14396006-01c5-4daa-a0f2-8c863285b5bd-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz\" (UID: \"14396006-01c5-4daa-a0f2-8c863285b5bd\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz"
Nov 01 05:07:49.530030 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.530002    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj"
Nov 01 05:07:49.583522 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.583420    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz4t4\" (UniqueName: \"kubernetes.io/projected/14396006-01c5-4daa-a0f2-8c863285b5bd-kube-api-access-wz4t4\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz\" (UID: \"14396006-01c5-4daa-a0f2-8c863285b5bd\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz"
Nov 01 05:07:49.629460 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.629170    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz"
Nov 01 05:07:49.722294 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.722169    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5"]
Nov 01 05:07:49.733125 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:07:49.733056    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb47e4d74_35df_4229_88f2_7d775d66ba9c.slice/crio-9b380734af36efd6f3a11956221f12a3e97f4e4a458fddf46fb95d3bfd7b9ef8 WatchSource:0}: Error finding container 9b380734af36efd6f3a11956221f12a3e97f4e4a458fddf46fb95d3bfd7b9ef8: Status 404 returned error can't find the container with id 9b380734af36efd6f3a11956221f12a3e97f4e4a458fddf46fb95d3bfd7b9ef8
Nov 01 05:07:49.823888 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.823478    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf"]
Nov 01 05:07:49.826550 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:07:49.826517    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a5ceca4_7882_4c74_9758_45e4bb783b66.slice/crio-d833cc91981061adad8b394f2c39e608d54bcb1d9614205e2c9c9f68c4e76e64 WatchSource:0}: Error finding container d833cc91981061adad8b394f2c39e608d54bcb1d9614205e2c9c9f68c4e76e64: Status 404 returned error can't find the container with id d833cc91981061adad8b394f2c39e608d54bcb1d9614205e2c9c9f68c4e76e64
Nov 01 05:07:49.918797 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:49.918752    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj"]
Nov 01 05:07:49.923929 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:07:49.923899    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod967c127b_a8a7_41cc_aef7_96c2cc3c36cb.slice/crio-eaf4b3316d6435f1317482073e6391f191c3848af203783a641b1e2a33cd6bd3 WatchSource:0}: Error finding container eaf4b3316d6435f1317482073e6391f191c3848af203783a641b1e2a33cd6bd3: Status 404 returned error can't find the container with id eaf4b3316d6435f1317482073e6391f191c3848af203783a641b1e2a33cd6bd3
Nov 01 05:07:50.014657 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:50.014406    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz"]
Nov 01 05:07:50.017961 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:07:50.017919    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14396006_01c5_4daa_a0f2_8c863285b5bd.slice/crio-bd257231936d7c33f14283700de5b2c0ef36370dfbb90dea170c755e72215417 WatchSource:0}: Error finding container bd257231936d7c33f14283700de5b2c0ef36370dfbb90dea170c755e72215417: Status 404 returned error can't find the container with id bd257231936d7c33f14283700de5b2c0ef36370dfbb90dea170c755e72215417
Nov 01 05:07:50.373594 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:50.373552    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz" event={"ID":"14396006-01c5-4daa-a0f2-8c863285b5bd","Type":"ContainerStarted","Data":"bd257231936d7c33f14283700de5b2c0ef36370dfbb90dea170c755e72215417"}
Nov 01 05:07:50.375577 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:50.375544    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf" event={"ID":"4a5ceca4-7882-4c74-9758-45e4bb783b66","Type":"ContainerStarted","Data":"d833cc91981061adad8b394f2c39e608d54bcb1d9614205e2c9c9f68c4e76e64"}
Nov 01 05:07:50.377152 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:50.377123    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj" event={"ID":"967c127b-a8a7-41cc-aef7-96c2cc3c36cb","Type":"ContainerStarted","Data":"eaf4b3316d6435f1317482073e6391f191c3848af203783a641b1e2a33cd6bd3"}
Nov 01 05:07:50.379105 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:50.379075    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5" event={"ID":"b47e4d74-35df-4229-88f2-7d775d66ba9c","Type":"ContainerStarted","Data":"9b380734af36efd6f3a11956221f12a3e97f4e4a458fddf46fb95d3bfd7b9ef8"}
Nov 01 05:07:50.730997 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:50.730916    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv"]
Nov 01 05:07:50.735314 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:50.735286    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv"
Nov 01 05:07:50.793002 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:50.792927    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv"]
Nov 01 05:07:50.827125 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:50.827064    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/070c9b8b-2af9-40a5-b85f-2ef0446e012e-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv\" (UID: \"070c9b8b-2af9-40a5-b85f-2ef0446e012e\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv"
Nov 01 05:07:50.827278 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:50.827130    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t9t2\" (UniqueName: \"kubernetes.io/projected/070c9b8b-2af9-40a5-b85f-2ef0446e012e-kube-api-access-7t9t2\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv\" (UID: \"070c9b8b-2af9-40a5-b85f-2ef0446e012e\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv"
Nov 01 05:07:50.827278 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:50.827184    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/070c9b8b-2af9-40a5-b85f-2ef0446e012e-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv\" (UID: \"070c9b8b-2af9-40a5-b85f-2ef0446e012e\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv"
Nov 01 05:07:50.927874 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:50.927823    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/070c9b8b-2af9-40a5-b85f-2ef0446e012e-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv\" (UID: \"070c9b8b-2af9-40a5-b85f-2ef0446e012e\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv"
Nov 01 05:07:50.928073 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:50.927913    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t9t2\" (UniqueName: \"kubernetes.io/projected/070c9b8b-2af9-40a5-b85f-2ef0446e012e-kube-api-access-7t9t2\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv\" (UID: \"070c9b8b-2af9-40a5-b85f-2ef0446e012e\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv"
Nov 01 05:07:50.928073 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:50.927977    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/070c9b8b-2af9-40a5-b85f-2ef0446e012e-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv\" (UID: \"070c9b8b-2af9-40a5-b85f-2ef0446e012e\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv"
Nov 01 05:07:50.928516 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:50.928487    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/070c9b8b-2af9-40a5-b85f-2ef0446e012e-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv\" (UID: \"070c9b8b-2af9-40a5-b85f-2ef0446e012e\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv"
Nov 01 05:07:50.928820 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:50.928777    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/070c9b8b-2af9-40a5-b85f-2ef0446e012e-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv\" (UID: \"070c9b8b-2af9-40a5-b85f-2ef0446e012e\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv"
Nov 01 05:07:51.006937 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:51.006744    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t9t2\" (UniqueName: \"kubernetes.io/projected/070c9b8b-2af9-40a5-b85f-2ef0446e012e-kube-api-access-7t9t2\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv\" (UID: \"070c9b8b-2af9-40a5-b85f-2ef0446e012e\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv"
Nov 01 05:07:51.060004 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:51.059956    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv"
Nov 01 05:07:51.086139 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:51.082306    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr"]
Nov 01 05:07:51.087991 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:51.087920    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr"
Nov 01 05:07:51.129830 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:51.129770    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st9dx\" (UniqueName: \"kubernetes.io/projected/ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68-kube-api-access-st9dx\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr\" (UID: \"ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr"
Nov 01 05:07:51.129991 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:51.129881    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr\" (UID: \"ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr"
Nov 01 05:07:51.129991 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:51.129924    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr\" (UID: \"ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr"
Nov 01 05:07:51.146599 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:51.146495    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr"]
Nov 01 05:07:51.230462 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:51.230424    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr\" (UID: \"ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr"
Nov 01 05:07:51.230639 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:51.230489    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr\" (UID: \"ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr"
Nov 01 05:07:51.230639 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:51.230535    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st9dx\" (UniqueName: \"kubernetes.io/projected/ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68-kube-api-access-st9dx\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr\" (UID: \"ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr"
Nov 01 05:07:51.231239 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:51.231212    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr\" (UID: \"ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr"
Nov 01 05:07:51.231660 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:51.231638    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr\" (UID: \"ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr"
Nov 01 05:07:51.322300 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:51.322267    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-st9dx\" (UniqueName: \"kubernetes.io/projected/ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68-kube-api-access-st9dx\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr\" (UID: \"ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr"
Nov 01 05:07:51.428030 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:51.427920    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr"
Nov 01 05:07:51.491823 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:51.489993    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv"]
Nov 01 05:07:51.497761 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:07:51.497724    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod070c9b8b_2af9_40a5_b85f_2ef0446e012e.slice/crio-6ac2d4b529f523c95ec73d1df0a3b9621ea2156669be6bd117f08d3e8317d913 WatchSource:0}: Error finding container 6ac2d4b529f523c95ec73d1df0a3b9621ea2156669be6bd117f08d3e8317d913: Status 404 returned error can't find the container with id 6ac2d4b529f523c95ec73d1df0a3b9621ea2156669be6bd117f08d3e8317d913
Nov 01 05:07:51.844831 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:51.840215    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr"]
Nov 01 05:07:52.396444 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:52.396399    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr" event={"ID":"ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68","Type":"ContainerStarted","Data":"0e5ce0667300ad3a8e50505c5ee3f1d6e7c83673baa7c0bfd59edfa22ee2e4ed"}
Nov 01 05:07:52.405455 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:52.405421    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv" event={"ID":"070c9b8b-2af9-40a5-b85f-2ef0446e012e","Type":"ContainerStarted","Data":"6ac2d4b529f523c95ec73d1df0a3b9621ea2156669be6bd117f08d3e8317d913"}
Nov 01 05:07:52.564391 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:07:52.564200    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:07:52.867230 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:52.867196    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"]
Nov 01 05:07:52.871187 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:52.871157    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:52.930027 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:52.929856    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 01 05:07:52.945633 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:52.945541    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-8hgdn\""
Nov 01 05:07:52.951480 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:52.951021    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92982041-1d56-4b3d-8067-3d393da5cad5-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:52.951480 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:52.951072    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92982041-1d56-4b3d-8067-3d393da5cad5-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:52.951480 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:52.951100    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5gld\" (UniqueName: \"kubernetes.io/projected/92982041-1d56-4b3d-8067-3d393da5cad5-kube-api-access-k5gld\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:52.951480 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:52.951129    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92982041-1d56-4b3d-8067-3d393da5cad5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:52.951480 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:52.951158    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92982041-1d56-4b3d-8067-3d393da5cad5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:52.951480 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:52.951185    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92982041-1d56-4b3d-8067-3d393da5cad5-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:52.951480 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:52.951211    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92982041-1d56-4b3d-8067-3d393da5cad5-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:52.951480 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:52.951252    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92982041-1d56-4b3d-8067-3d393da5cad5-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:52.951480 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:52.951290    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92982041-1d56-4b3d-8067-3d393da5cad5-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:52.982981 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:52.982947    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"]
Nov 01 05:07:53.052406 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.052334    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92982041-1d56-4b3d-8067-3d393da5cad5-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:53.052406 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.052388    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5gld\" (UniqueName: \"kubernetes.io/projected/92982041-1d56-4b3d-8067-3d393da5cad5-kube-api-access-k5gld\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:53.052653 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.052417    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92982041-1d56-4b3d-8067-3d393da5cad5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:53.052653 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.052447    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92982041-1d56-4b3d-8067-3d393da5cad5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:53.052653 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.052473    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92982041-1d56-4b3d-8067-3d393da5cad5-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:53.052653 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.052500    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92982041-1d56-4b3d-8067-3d393da5cad5-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:53.052653 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.052542    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92982041-1d56-4b3d-8067-3d393da5cad5-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:53.052653 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.052566    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92982041-1d56-4b3d-8067-3d393da5cad5-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:53.052979 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.052662    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92982041-1d56-4b3d-8067-3d393da5cad5-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:53.053564 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.053055    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92982041-1d56-4b3d-8067-3d393da5cad5-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:53.053564 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.053095    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92982041-1d56-4b3d-8067-3d393da5cad5-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:53.053564 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.053173    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92982041-1d56-4b3d-8067-3d393da5cad5-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:53.053564 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.053393    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92982041-1d56-4b3d-8067-3d393da5cad5-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:53.054209 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.054185    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92982041-1d56-4b3d-8067-3d393da5cad5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:53.075308 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.075273    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92982041-1d56-4b3d-8067-3d393da5cad5-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:53.076368 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.076263    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92982041-1d56-4b3d-8067-3d393da5cad5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:53.121178 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.121009    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92982041-1d56-4b3d-8067-3d393da5cad5-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:53.132615 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.132584    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5gld\" (UniqueName: \"kubernetes.io/projected/92982041-1d56-4b3d-8067-3d393da5cad5-kube-api-access-k5gld\") pod \"maas-default-gateway-openshift-default-58d898bd55-ck46t\" (UID: \"92982041-1d56-4b3d-8067-3d393da5cad5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:53.179901 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.178622    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"]
Nov 01 05:07:53.187096 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.186457    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.189738 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.189509    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:07:53.251101 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.251072    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-ai-inference-dockercfg-28g9k\""
Nov 01 05:07:53.256145 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.256032    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8860f9dd-4607-488f-9f91-045a50877f36-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.256597 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.256117    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8860f9dd-4607-488f-9f91-045a50877f36-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.256852 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.256710    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nww5w\" (UniqueName: \"kubernetes.io/projected/8860f9dd-4607-488f-9f91-045a50877f36-kube-api-access-nww5w\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.257159 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.256929    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8860f9dd-4607-488f-9f91-045a50877f36-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.257159 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.257079    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8860f9dd-4607-488f-9f91-045a50877f36-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.257613 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.257227    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8860f9dd-4607-488f-9f91-045a50877f36-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.257613 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.257271    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8860f9dd-4607-488f-9f91-045a50877f36-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.257613 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.257444    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8860f9dd-4607-488f-9f91-045a50877f36-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.258567 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.258493    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8860f9dd-4607-488f-9f91-045a50877f36-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.268953 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.268889    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"]
Nov 01 05:07:53.359325 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.359268    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8860f9dd-4607-488f-9f91-045a50877f36-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.359325 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.359328    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8860f9dd-4607-488f-9f91-045a50877f36-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.359711 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.359362    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nww5w\" (UniqueName: \"kubernetes.io/projected/8860f9dd-4607-488f-9f91-045a50877f36-kube-api-access-nww5w\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.359711 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.359396    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8860f9dd-4607-488f-9f91-045a50877f36-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.359711 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.359428    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8860f9dd-4607-488f-9f91-045a50877f36-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.359711 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.359459    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8860f9dd-4607-488f-9f91-045a50877f36-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.359711 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.359493    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8860f9dd-4607-488f-9f91-045a50877f36-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.359711 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.359518    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8860f9dd-4607-488f-9f91-045a50877f36-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.359711 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.359566    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8860f9dd-4607-488f-9f91-045a50877f36-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.360845 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.360810    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8860f9dd-4607-488f-9f91-045a50877f36-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.361203 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.361168    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8860f9dd-4607-488f-9f91-045a50877f36-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.361859 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.361774    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8860f9dd-4607-488f-9f91-045a50877f36-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.362441 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.362417    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8860f9dd-4607-488f-9f91-045a50877f36-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.363012 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.362981    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8860f9dd-4607-488f-9f91-045a50877f36-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.377655 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.377594    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8860f9dd-4607-488f-9f91-045a50877f36-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.391420 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.391395    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8860f9dd-4607-488f-9f91-045a50877f36-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.431047 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.430944    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nww5w\" (UniqueName: \"kubernetes.io/projected/8860f9dd-4607-488f-9f91-045a50877f36-kube-api-access-nww5w\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.436020 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.435969    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8860f9dd-4607-488f-9f91-045a50877f36-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v\" (UID: \"8860f9dd-4607-488f-9f91-045a50877f36\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.520318 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.520280    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:07:53.667908 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.667727    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"]
Nov 01 05:07:53.954033 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:53.953950    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"]
Nov 01 05:07:53.961156 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:07:53.961127    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8860f9dd_4607_488f_9f91_045a50877f36.slice/crio-b96f016bd59a52981c4e56ea07b2431967264f45d94f1d98fd571e1eb5dffacf WatchSource:0}: Error finding container b96f016bd59a52981c4e56ea07b2431967264f45d94f1d98fd571e1eb5dffacf: Status 404 returned error can't find the container with id b96f016bd59a52981c4e56ea07b2431967264f45d94f1d98fd571e1eb5dffacf
Nov 01 05:07:54.433210 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:54.433163    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v" event={"ID":"8860f9dd-4607-488f-9f91-045a50877f36","Type":"ContainerStarted","Data":"b96f016bd59a52981c4e56ea07b2431967264f45d94f1d98fd571e1eb5dffacf"}
Nov 01 05:07:54.446952 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:07:54.446913    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t" event={"ID":"92982041-1d56-4b3d-8067-3d393da5cad5","Type":"ContainerStarted","Data":"672a96ed73e40fa340a4a75d3332e471ca3bd7a7d03cb16551dd93071fc6a951"}
Nov 01 05:08:03.480585 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:03.480557    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:08:03.481652 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:03.481623    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:08:04.568193 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:04.568155    4864 generic.go:356] "Generic (PLEG): container finished" podID="50f35a8f-80ec-41e3-9255-e1358ca8ca52" containerID="e44ccfd65214c43a6070d33740153ff3b25b721ea038ae1deb0fddd358c67dc3" exitCode=0
Nov 01 05:08:04.568633 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:04.568251    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b" event={"ID":"50f35a8f-80ec-41e3-9255-e1358ca8ca52","Type":"ContainerDied","Data":"e44ccfd65214c43a6070d33740153ff3b25b721ea038ae1deb0fddd358c67dc3"}
Nov 01 05:08:04.573342 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:04.573313    4864 generic.go:356] "Generic (PLEG): container finished" podID="a52e528e-c723-42b6-8d1a-03c3675518b9" containerID="0f3c39be36e7111351f52e65dcb4767c9a59d65f77e79a58c692655769d2da6f" exitCode=0
Nov 01 05:08:04.573458 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:04.573403    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8" event={"ID":"a52e528e-c723-42b6-8d1a-03c3675518b9","Type":"ContainerDied","Data":"0f3c39be36e7111351f52e65dcb4767c9a59d65f77e79a58c692655769d2da6f"}
Nov 01 05:08:04.575746 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:04.575715    4864 generic.go:356] "Generic (PLEG): container finished" podID="967c127b-a8a7-41cc-aef7-96c2cc3c36cb" containerID="ab963aaa7dfb96009c9c70d90d98bb57149a0a398f199203b9783025b1dfa655" exitCode=0
Nov 01 05:08:04.575868 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:04.575811    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj" event={"ID":"967c127b-a8a7-41cc-aef7-96c2cc3c36cb","Type":"ContainerDied","Data":"ab963aaa7dfb96009c9c70d90d98bb57149a0a398f199203b9783025b1dfa655"}
Nov 01 05:08:04.579015 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:04.578970    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv" event={"ID":"070c9b8b-2af9-40a5-b85f-2ef0446e012e","Type":"ContainerStarted","Data":"a274ad7f995d9d42618cc2dbc7b5b0a4d2883f828662a1e6b048588a976038ae"}
Nov 01 05:08:04.762349 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:04.762312    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:08:04.762490 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:04.762404    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:08:04.762490 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:04.762452    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:08:04.983941 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:04.983904    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:08:04.984067 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:04.983995    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:08:04.984067 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:04.984049    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:08:05.586505 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:05.586466    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t" event={"ID":"92982041-1d56-4b3d-8067-3d393da5cad5","Type":"ContainerStarted","Data":"b5f2cb8d051a37a2e6e90dcc06631283b736fc909bd5cb529b03394619a9336f"}
Nov 01 05:08:05.588717 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:05.588668    4864 generic.go:356] "Generic (PLEG): container finished" podID="1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf" containerID="7bea6b20fdd17c1089cbaccadca0dae06506fd5cb5bd5ec4ddd628b51fb572c2" exitCode=0
Nov 01 05:08:05.589021 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:05.588995    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr" event={"ID":"1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf","Type":"ContainerDied","Data":"7bea6b20fdd17c1089cbaccadca0dae06506fd5cb5bd5ec4ddd628b51fb572c2"}
Nov 01 05:08:05.591959 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:05.591935    4864 generic.go:356] "Generic (PLEG): container finished" podID="ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68" containerID="3ffb2d14ec2941208c71ff8a4ed9e88a7c827bf9891512f4d10f774742e5e84f" exitCode=0
Nov 01 05:08:05.592050 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:05.591970    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr" event={"ID":"ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68","Type":"ContainerDied","Data":"3ffb2d14ec2941208c71ff8a4ed9e88a7c827bf9891512f4d10f774742e5e84f"}
Nov 01 05:08:05.595859 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:05.595379    4864 generic.go:356] "Generic (PLEG): container finished" podID="070c9b8b-2af9-40a5-b85f-2ef0446e012e" containerID="a274ad7f995d9d42618cc2dbc7b5b0a4d2883f828662a1e6b048588a976038ae" exitCode=0
Nov 01 05:08:05.595859 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:05.595495    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv" event={"ID":"070c9b8b-2af9-40a5-b85f-2ef0446e012e","Type":"ContainerDied","Data":"a274ad7f995d9d42618cc2dbc7b5b0a4d2883f828662a1e6b048588a976038ae"}
Nov 01 05:08:05.599448 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:05.599424    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v" event={"ID":"8860f9dd-4607-488f-9f91-045a50877f36","Type":"ContainerStarted","Data":"429c9d8bca8f16cd015041f29fa5f5c9d0a72bf594694cdbc8d9ab4a87723233"}
Nov 01 05:08:05.710729 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:05.710628    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t" podStartSLOduration=2.420476224 podStartE2EDuration="13.710612239s" totalImagesPullingTime="11.290136015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:07:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:08:05.70970617 +0000 UTC m=+302.600082163" watchObservedRunningTime="2026-11-01 05:08:05.710612239 +0000 UTC m=+302.600988209"
Nov 01 05:08:06.069107 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:06.069027    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v" podStartSLOduration=2.274471508 podStartE2EDuration="13.069005607s" totalImagesPullingTime="10.794534099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:07:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:08:06.066692445 +0000 UTC m=+302.957068403" watchObservedRunningTime="2026-11-01 05:08:06.069005607 +0000 UTC m=+302.959381580"
Nov 01 05:08:06.190470 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:06.190440    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:08:06.196148 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:06.196115    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:08:06.521460 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:06.521376    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:08:06.528095 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:06.528064    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:08:06.609286 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:06.609252    4864 generic.go:356] "Generic (PLEG): container finished" podID="50f35a8f-80ec-41e3-9255-e1358ca8ca52" containerID="25a38430b71900a00eb4679ea55a38074b973cef2b78a9d20b425f3fcf9d2aa0" exitCode=0
Nov 01 05:08:06.612006 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:06.610988    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b" event={"ID":"50f35a8f-80ec-41e3-9255-e1358ca8ca52","Type":"ContainerDied","Data":"25a38430b71900a00eb4679ea55a38074b973cef2b78a9d20b425f3fcf9d2aa0"}
Nov 01 05:08:06.612006 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:06.611046    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:08:06.612006 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:06.611754    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:08:06.613031 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:06.612850    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v"
Nov 01 05:08:06.613031 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:06.612945    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:08:07.620602 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:07.618047    4864 generic.go:356] "Generic (PLEG): container finished" podID="14396006-01c5-4daa-a0f2-8c863285b5bd" containerID="effc566504ffdf61cc6fa171abef03464d89d14b494c71d3ae4665a58e2d8005" exitCode=0
Nov 01 05:08:07.620602 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:07.618126    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz" event={"ID":"14396006-01c5-4daa-a0f2-8c863285b5bd","Type":"ContainerDied","Data":"effc566504ffdf61cc6fa171abef03464d89d14b494c71d3ae4665a58e2d8005"}
Nov 01 05:08:07.623105 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:07.623070    4864 generic.go:356] "Generic (PLEG): container finished" podID="967c127b-a8a7-41cc-aef7-96c2cc3c36cb" containerID="e60b566c5b7f71e1a9c77b719a4cc4ca3bece6c0fe49a02350cdedb0c3f1d077" exitCode=0
Nov 01 05:08:07.623235 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:07.623142    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj" event={"ID":"967c127b-a8a7-41cc-aef7-96c2cc3c36cb","Type":"ContainerDied","Data":"e60b566c5b7f71e1a9c77b719a4cc4ca3bece6c0fe49a02350cdedb0c3f1d077"}
Nov 01 05:08:07.625518 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:07.625492    4864 generic.go:356] "Generic (PLEG): container finished" podID="ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68" containerID="1e1f69bc4d9fd2b6685fd0dda20b6756cf1c292a2a92f07a384d6d020fa1c9d9" exitCode=0
Nov 01 05:08:07.625886 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:07.625857    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr" event={"ID":"ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68","Type":"ContainerDied","Data":"1e1f69bc4d9fd2b6685fd0dda20b6756cf1c292a2a92f07a384d6d020fa1c9d9"}
Nov 01 05:08:08.632225 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:08.632192    4864 generic.go:356] "Generic (PLEG): container finished" podID="b47e4d74-35df-4229-88f2-7d775d66ba9c" containerID="9633a2a7d3f1f74581b51d92356b272f09af6399d929584d8970c808898a8b51" exitCode=0
Nov 01 05:08:08.632638 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:08.632281    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5" event={"ID":"b47e4d74-35df-4229-88f2-7d775d66ba9c","Type":"ContainerDied","Data":"9633a2a7d3f1f74581b51d92356b272f09af6399d929584d8970c808898a8b51"}
Nov 01 05:08:09.645951 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:09.645911    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8" event={"ID":"a52e528e-c723-42b6-8d1a-03c3675518b9","Type":"ContainerStarted","Data":"436e2c7570d25a9631802ba0c4151fa467b09c80d8e5faafdd2363409f1a93eb"}
Nov 01 05:08:09.647924 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:09.647904    4864 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 05:08:09.651442 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:09.651173    4864 generic.go:356] "Generic (PLEG): container finished" podID="4a5ceca4-7882-4c74-9758-45e4bb783b66" containerID="2be39baa5a98ffc3a75d576824aa75888c782678ee6f94b4cc107a2291a48ba8" exitCode=0
Nov 01 05:08:09.651442 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:09.651270    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf" event={"ID":"4a5ceca4-7882-4c74-9758-45e4bb783b66","Type":"ContainerDied","Data":"2be39baa5a98ffc3a75d576824aa75888c782678ee6f94b4cc107a2291a48ba8"}
Nov 01 05:08:09.655810 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:09.655184    4864 generic.go:356] "Generic (PLEG): container finished" podID="070c9b8b-2af9-40a5-b85f-2ef0446e012e" containerID="a5c0cb97248661699fed8541c5743619648c76f3f11f3c8400da9d76607556b8" exitCode=0
Nov 01 05:08:09.655810 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:09.655324    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv" event={"ID":"070c9b8b-2af9-40a5-b85f-2ef0446e012e","Type":"ContainerDied","Data":"a5c0cb97248661699fed8541c5743619648c76f3f11f3c8400da9d76607556b8"}
Nov 01 05:08:10.674976 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:10.674743    4864 generic.go:356] "Generic (PLEG): container finished" podID="a52e528e-c723-42b6-8d1a-03c3675518b9" containerID="436e2c7570d25a9631802ba0c4151fa467b09c80d8e5faafdd2363409f1a93eb" exitCode=0
Nov 01 05:08:10.674976 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:10.674903    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8" event={"ID":"a52e528e-c723-42b6-8d1a-03c3675518b9","Type":"ContainerDied","Data":"436e2c7570d25a9631802ba0c4151fa467b09c80d8e5faafdd2363409f1a93eb"}
Nov 01 05:08:10.680236 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:10.679142    4864 generic.go:356] "Generic (PLEG): container finished" podID="b47e4d74-35df-4229-88f2-7d775d66ba9c" containerID="2d53a122f65841b4014d7e69c2779f66e9b3ce8c64fdcbfe26c7f95f95c2ed70" exitCode=0
Nov 01 05:08:10.680236 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:10.679192    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5" event={"ID":"b47e4d74-35df-4229-88f2-7d775d66ba9c","Type":"ContainerDied","Data":"2d53a122f65841b4014d7e69c2779f66e9b3ce8c64fdcbfe26c7f95f95c2ed70"}
Nov 01 05:08:11.691264 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:11.691009    4864 generic.go:356] "Generic (PLEG): container finished" podID="1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf" containerID="0c5c94fd57571a88599e0a77ed148ad5c12c4ed1c569831ba5db371b002a0ffe" exitCode=0
Nov 01 05:08:11.691264 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:11.691116    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr" event={"ID":"1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf","Type":"ContainerDied","Data":"0c5c94fd57571a88599e0a77ed148ad5c12c4ed1c569831ba5db371b002a0ffe"}
Nov 01 05:08:11.701191 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:11.701148    4864 generic.go:356] "Generic (PLEG): container finished" podID="4a5ceca4-7882-4c74-9758-45e4bb783b66" containerID="909d89d8d811acb279ca86a06b08d518564a21dc63b39447787e57e0e15e5518" exitCode=0
Nov 01 05:08:11.701311 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:11.701266    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf" event={"ID":"4a5ceca4-7882-4c74-9758-45e4bb783b66","Type":"ContainerDied","Data":"909d89d8d811acb279ca86a06b08d518564a21dc63b39447787e57e0e15e5518"}
Nov 01 05:08:14.741228 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:14.740100    4864 generic.go:356] "Generic (PLEG): container finished" podID="14396006-01c5-4daa-a0f2-8c863285b5bd" containerID="601a308d8d0c28bc8ee027ad749e1a845f521fb016be41e041d6c7c8bf5ca808" exitCode=0
Nov 01 05:08:14.741228 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:14.740207    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz" event={"ID":"14396006-01c5-4daa-a0f2-8c863285b5bd","Type":"ContainerDied","Data":"601a308d8d0c28bc8ee027ad749e1a845f521fb016be41e041d6c7c8bf5ca808"}
Nov 01 05:08:22.548101 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:08:22.548055    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:08:25.871322 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:25.870627    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr" event={"ID":"ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68","Type":"ContainerStarted","Data":"26799ead9b6d606dd44728acfb40aa78910d00a37eacd873c389ab71b5a2eef3"}
Nov 01 05:08:25.990740 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:25.990000    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr" podStartSLOduration=3.361722817 podStartE2EDuration="34.989979671s" totalImagesPullingTime="31.628256854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:07:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:08:25.987833299 +0000 UTC m=+322.878209277" watchObservedRunningTime="2026-11-01 05:08:25.989979671 +0000 UTC m=+322.880355647"
Nov 01 05:08:26.883030 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:26.882997    4864 generic.go:356] "Generic (PLEG): container finished" podID="ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68" containerID="26799ead9b6d606dd44728acfb40aa78910d00a37eacd873c389ab71b5a2eef3" exitCode=0
Nov 01 05:08:26.883453 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:26.883168    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr" event={"ID":"ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68","Type":"ContainerDied","Data":"26799ead9b6d606dd44728acfb40aa78910d00a37eacd873c389ab71b5a2eef3"}
Nov 01 05:08:26.890090 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:26.890061    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b" event={"ID":"50f35a8f-80ec-41e3-9255-e1358ca8ca52","Type":"ContainerStarted","Data":"b6a0ccd373ff3c1963b3d5aec2645b3ad64d8308ae2f40d089780d1629ab6bd8"}
Nov 01 05:08:27.158045 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:27.156706    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b" podStartSLOduration=3.935769593 podStartE2EDuration="43.156685931s" totalImagesPullingTime="39.220916338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:07:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:08:27.155422892 +0000 UTC m=+324.045798867" watchObservedRunningTime="2026-11-01 05:08:27.156685931 +0000 UTC m=+324.047061907"
Nov 01 05:08:27.904997 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:27.904962    4864 generic.go:356] "Generic (PLEG): container finished" podID="14396006-01c5-4daa-a0f2-8c863285b5bd" containerID="a30895ad66402764febacc1f6bcee01059ed6212b1d83c154b3d65b5eb98c6a8" exitCode=0
Nov 01 05:08:27.905557 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:27.905457    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz" event={"ID":"14396006-01c5-4daa-a0f2-8c863285b5bd","Type":"ContainerDied","Data":"a30895ad66402764febacc1f6bcee01059ed6212b1d83c154b3d65b5eb98c6a8"}
Nov 01 05:08:27.909853 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:27.909693    4864 generic.go:356] "Generic (PLEG): container finished" podID="967c127b-a8a7-41cc-aef7-96c2cc3c36cb" containerID="38200b3fa91774782d4a43d03270f27e016e3cd4dc7d2145f16c875ed6ee70ee" exitCode=0
Nov 01 05:08:27.909853 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:27.909808    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj" event={"ID":"967c127b-a8a7-41cc-aef7-96c2cc3c36cb","Type":"ContainerDied","Data":"38200b3fa91774782d4a43d03270f27e016e3cd4dc7d2145f16c875ed6ee70ee"}
Nov 01 05:08:27.916440 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:27.916029    4864 generic.go:356] "Generic (PLEG): container finished" podID="50f35a8f-80ec-41e3-9255-e1358ca8ca52" containerID="b6a0ccd373ff3c1963b3d5aec2645b3ad64d8308ae2f40d089780d1629ab6bd8" exitCode=0
Nov 01 05:08:27.916440 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:27.916393    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b" event={"ID":"50f35a8f-80ec-41e3-9255-e1358ca8ca52","Type":"ContainerDied","Data":"b6a0ccd373ff3c1963b3d5aec2645b3ad64d8308ae2f40d089780d1629ab6bd8"}
Nov 01 05:08:28.106848 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:28.106773    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr"
Nov 01 05:08:28.163664 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:28.163628    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68-bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68-bundle\") pod \"ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68\" (UID: \"ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68\") "
Nov 01 05:08:28.163846 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:28.163742    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68-kube-api-access-st9dx\" (UniqueName: \"kubernetes.io/projected/ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68-kube-api-access-st9dx\") pod \"ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68\" (UID: \"ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68\") "
Nov 01 05:08:28.163846 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:28.163771    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68-util\" (UniqueName: \"kubernetes.io/empty-dir/ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68-util\") pod \"ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68\" (UID: \"ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68\") "
Nov 01 05:08:28.164623 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:28.164594    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68-bundle" pod "ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68" (UID: "ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:08:28.166698 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:28.166639    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68-kube-api-access-st9dx" pod "ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68" (UID: "ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68"). InnerVolumeSpecName "kube-api-access-st9dx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:08:28.170391 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:28.170353    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68-util" pod "ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68" (UID: "ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:08:28.264899 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:28.264860    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-st9dx\" (UniqueName: \"kubernetes.io/projected/ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68-kube-api-access-st9dx\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:28.264899 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:28.264898    4864 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68-util\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:28.265124 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:28.264910    4864 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68-bundle\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:28.924412 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:28.924375    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5" event={"ID":"b47e4d74-35df-4229-88f2-7d775d66ba9c","Type":"ContainerStarted","Data":"baa0b0dbfcc33ef74ebb2b540a1dd46a8345b354989ed4a84c1c363ff5b0619d"}
Nov 01 05:08:28.926226 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:28.926199    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:08:28.927116 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:28.926946    4864 generic.go:356] "Generic (PLEG): container finished" podID="a52e528e-c723-42b6-8d1a-03c3675518b9" containerID="d692449a7c605b9d117dbd6b649298e4de50b099ebd218a80092bc5773cc25a5" exitCode=1
Nov 01 05:08:28.927116 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:28.927055    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8" event={"ID":"a52e528e-c723-42b6-8d1a-03c3675518b9","Type":"ContainerDied","Data":"d692449a7c605b9d117dbd6b649298e4de50b099ebd218a80092bc5773cc25a5"}
Nov 01 05:08:28.929157 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:28.929138    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr"
Nov 01 05:08:28.929157 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:28.929141    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7158tvr" event={"ID":"ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68","Type":"ContainerDied","Data":"0e5ce0667300ad3a8e50505c5ee3f1d6e7c83673baa7c0bfd59edfa22ee2e4ed"}
Nov 01 05:08:28.929339 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:28.929184    4864 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e5ce0667300ad3a8e50505c5ee3f1d6e7c83673baa7c0bfd59edfa22ee2e4ed"
Nov 01 05:08:28.931870 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:28.931684    4864 generic.go:356] "Generic (PLEG): container finished" podID="070c9b8b-2af9-40a5-b85f-2ef0446e012e" containerID="201c05e8581cc0ddaf6d56365a7ac0c78350408d4aaddb5ef20fb63fe4f653f3" exitCode=0
Nov 01 05:08:28.931870 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:28.931714    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv" event={"ID":"070c9b8b-2af9-40a5-b85f-2ef0446e012e","Type":"ContainerDied","Data":"201c05e8581cc0ddaf6d56365a7ac0c78350408d4aaddb5ef20fb63fe4f653f3"}
Nov 01 05:08:29.110612 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.110581    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz"
Nov 01 05:08:29.146952 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.146921    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b"
Nov 01 05:08:29.163138 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.163105    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj"
Nov 01 05:08:29.175028 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.175001    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50f35a8f-80ec-41e3-9255-e1358ca8ca52-util\" (UniqueName: \"kubernetes.io/empty-dir/50f35a8f-80ec-41e3-9255-e1358ca8ca52-util\") pod \"50f35a8f-80ec-41e3-9255-e1358ca8ca52\" (UID: \"50f35a8f-80ec-41e3-9255-e1358ca8ca52\") "
Nov 01 05:08:29.175177 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.175087    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14396006-01c5-4daa-a0f2-8c863285b5bd-util\" (UniqueName: \"kubernetes.io/empty-dir/14396006-01c5-4daa-a0f2-8c863285b5bd-util\") pod \"14396006-01c5-4daa-a0f2-8c863285b5bd\" (UID: \"14396006-01c5-4daa-a0f2-8c863285b5bd\") "
Nov 01 05:08:29.175177 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.175109    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14396006-01c5-4daa-a0f2-8c863285b5bd-kube-api-access-wz4t4\" (UniqueName: \"kubernetes.io/projected/14396006-01c5-4daa-a0f2-8c863285b5bd-kube-api-access-wz4t4\") pod \"14396006-01c5-4daa-a0f2-8c863285b5bd\" (UID: \"14396006-01c5-4daa-a0f2-8c863285b5bd\") "
Nov 01 05:08:29.175177 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.175137    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50f35a8f-80ec-41e3-9255-e1358ca8ca52-kube-api-access-9kn57\" (UniqueName: \"kubernetes.io/projected/50f35a8f-80ec-41e3-9255-e1358ca8ca52-kube-api-access-9kn57\") pod \"50f35a8f-80ec-41e3-9255-e1358ca8ca52\" (UID: \"50f35a8f-80ec-41e3-9255-e1358ca8ca52\") "
Nov 01 05:08:29.175177 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.175168    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14396006-01c5-4daa-a0f2-8c863285b5bd-bundle\" (UniqueName: \"kubernetes.io/empty-dir/14396006-01c5-4daa-a0f2-8c863285b5bd-bundle\") pod \"14396006-01c5-4daa-a0f2-8c863285b5bd\" (UID: \"14396006-01c5-4daa-a0f2-8c863285b5bd\") "
Nov 01 05:08:29.175389 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.175220    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50f35a8f-80ec-41e3-9255-e1358ca8ca52-bundle\" (UniqueName: \"kubernetes.io/empty-dir/50f35a8f-80ec-41e3-9255-e1358ca8ca52-bundle\") pod \"50f35a8f-80ec-41e3-9255-e1358ca8ca52\" (UID: \"50f35a8f-80ec-41e3-9255-e1358ca8ca52\") "
Nov 01 05:08:29.175761 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.175724    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14396006-01c5-4daa-a0f2-8c863285b5bd-bundle" pod "14396006-01c5-4daa-a0f2-8c863285b5bd" (UID: "14396006-01c5-4daa-a0f2-8c863285b5bd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:08:29.176061 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.175804    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50f35a8f-80ec-41e3-9255-e1358ca8ca52-bundle" pod "50f35a8f-80ec-41e3-9255-e1358ca8ca52" (UID: "50f35a8f-80ec-41e3-9255-e1358ca8ca52"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:08:29.177613 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.177588    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50f35a8f-80ec-41e3-9255-e1358ca8ca52-kube-api-access-9kn57" pod "50f35a8f-80ec-41e3-9255-e1358ca8ca52" (UID: "50f35a8f-80ec-41e3-9255-e1358ca8ca52"). InnerVolumeSpecName "kube-api-access-9kn57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:08:29.177613 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.177594    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14396006-01c5-4daa-a0f2-8c863285b5bd-kube-api-access-wz4t4" pod "14396006-01c5-4daa-a0f2-8c863285b5bd" (UID: "14396006-01c5-4daa-a0f2-8c863285b5bd"). InnerVolumeSpecName "kube-api-access-wz4t4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:08:29.183413 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.183387    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50f35a8f-80ec-41e3-9255-e1358ca8ca52-util" pod "50f35a8f-80ec-41e3-9255-e1358ca8ca52" (UID: "50f35a8f-80ec-41e3-9255-e1358ca8ca52"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:08:29.184892 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.184874    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14396006-01c5-4daa-a0f2-8c863285b5bd-util" pod "14396006-01c5-4daa-a0f2-8c863285b5bd" (UID: "14396006-01c5-4daa-a0f2-8c863285b5bd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:08:29.276519 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.276476    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/967c127b-a8a7-41cc-aef7-96c2cc3c36cb-util\" (UniqueName: \"kubernetes.io/empty-dir/967c127b-a8a7-41cc-aef7-96c2cc3c36cb-util\") pod \"967c127b-a8a7-41cc-aef7-96c2cc3c36cb\" (UID: \"967c127b-a8a7-41cc-aef7-96c2cc3c36cb\") "
Nov 01 05:08:29.276682 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.276529    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/967c127b-a8a7-41cc-aef7-96c2cc3c36cb-kube-api-access-t4b4r\" (UniqueName: \"kubernetes.io/projected/967c127b-a8a7-41cc-aef7-96c2cc3c36cb-kube-api-access-t4b4r\") pod \"967c127b-a8a7-41cc-aef7-96c2cc3c36cb\" (UID: \"967c127b-a8a7-41cc-aef7-96c2cc3c36cb\") "
Nov 01 05:08:29.276682 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.276574    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/967c127b-a8a7-41cc-aef7-96c2cc3c36cb-bundle\" (UniqueName: \"kubernetes.io/empty-dir/967c127b-a8a7-41cc-aef7-96c2cc3c36cb-bundle\") pod \"967c127b-a8a7-41cc-aef7-96c2cc3c36cb\" (UID: \"967c127b-a8a7-41cc-aef7-96c2cc3c36cb\") "
Nov 01 05:08:29.276811 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.276728    4864 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/50f35a8f-80ec-41e3-9255-e1358ca8ca52-util\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:29.276811 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.276747    4864 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/14396006-01c5-4daa-a0f2-8c863285b5bd-util\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:29.276811 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.276764    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wz4t4\" (UniqueName: \"kubernetes.io/projected/14396006-01c5-4daa-a0f2-8c863285b5bd-kube-api-access-wz4t4\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:29.276811 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.276779    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kn57\" (UniqueName: \"kubernetes.io/projected/50f35a8f-80ec-41e3-9255-e1358ca8ca52-kube-api-access-9kn57\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:29.277000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.276814    4864 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/14396006-01c5-4daa-a0f2-8c863285b5bd-bundle\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:29.277000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.276829    4864 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/50f35a8f-80ec-41e3-9255-e1358ca8ca52-bundle\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:29.277085 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.277055    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/967c127b-a8a7-41cc-aef7-96c2cc3c36cb-bundle" pod "967c127b-a8a7-41cc-aef7-96c2cc3c36cb" (UID: "967c127b-a8a7-41cc-aef7-96c2cc3c36cb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:08:29.278839 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.278807    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/967c127b-a8a7-41cc-aef7-96c2cc3c36cb-kube-api-access-t4b4r" pod "967c127b-a8a7-41cc-aef7-96c2cc3c36cb" (UID: "967c127b-a8a7-41cc-aef7-96c2cc3c36cb"). InnerVolumeSpecName "kube-api-access-t4b4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:08:29.283100 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.283071    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/967c127b-a8a7-41cc-aef7-96c2cc3c36cb-util" pod "967c127b-a8a7-41cc-aef7-96c2cc3c36cb" (UID: "967c127b-a8a7-41cc-aef7-96c2cc3c36cb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:08:29.378144 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.378114    4864 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/967c127b-a8a7-41cc-aef7-96c2cc3c36cb-bundle\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:29.378144 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.378141    4864 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/967c127b-a8a7-41cc-aef7-96c2cc3c36cb-util\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:29.378144 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.378151    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4b4r\" (UniqueName: \"kubernetes.io/projected/967c127b-a8a7-41cc-aef7-96c2cc3c36cb-kube-api-access-t4b4r\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:29.946572 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.946537    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz" event={"ID":"14396006-01c5-4daa-a0f2-8c863285b5bd","Type":"ContainerDied","Data":"bd257231936d7c33f14283700de5b2c0ef36370dfbb90dea170c755e72215417"}
Nov 01 05:08:29.946572 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.946559    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75rczz"
Nov 01 05:08:29.947035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.946619    4864 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd257231936d7c33f14283700de5b2c0ef36370dfbb90dea170c755e72215417"
Nov 01 05:08:29.948758 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.948731    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf" event={"ID":"4a5ceca4-7882-4c74-9758-45e4bb783b66","Type":"ContainerStarted","Data":"e62a51d84997b7a2ae2c1bfe5c5e9fbd527c2893d3b4d9b3ad03bf65461dcb45"}
Nov 01 05:08:29.950426 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.950412    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj"
Nov 01 05:08:29.950518 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.950435    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nbsjj" event={"ID":"967c127b-a8a7-41cc-aef7-96c2cc3c36cb","Type":"ContainerDied","Data":"eaf4b3316d6435f1317482073e6391f191c3848af203783a641b1e2a33cd6bd3"}
Nov 01 05:08:29.950518 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.950463    4864 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eaf4b3316d6435f1317482073e6391f191c3848af203783a641b1e2a33cd6bd3"
Nov 01 05:08:29.952475 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.952452    4864 generic.go:356] "Generic (PLEG): container finished" podID="b47e4d74-35df-4229-88f2-7d775d66ba9c" containerID="baa0b0dbfcc33ef74ebb2b540a1dd46a8345b354989ed4a84c1c363ff5b0619d" exitCode=0
Nov 01 05:08:29.952541 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.952522    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5" event={"ID":"b47e4d74-35df-4229-88f2-7d775d66ba9c","Type":"ContainerDied","Data":"baa0b0dbfcc33ef74ebb2b540a1dd46a8345b354989ed4a84c1c363ff5b0619d"}
Nov 01 05:08:29.954392 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.954377    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b"
Nov 01 05:08:29.954470 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.954389    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mgf2b" event={"ID":"50f35a8f-80ec-41e3-9255-e1358ca8ca52","Type":"ContainerDied","Data":"b41e3a7a85ff221750ac1ce9fd85882ba9ffe754f267d4e5b77e33339a72b973"}
Nov 01 05:08:29.954470 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.954425    4864 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b41e3a7a85ff221750ac1ce9fd85882ba9ffe754f267d4e5b77e33339a72b973"
Nov 01 05:08:29.956466 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:29.956441    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr" event={"ID":"1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf","Type":"ContainerStarted","Data":"4d322fa5514555d6f5f717350b1afd375e20e9a2c46ecf885a04783a76b4f068"}
Nov 01 05:08:30.101979 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.101956    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv"
Nov 01 05:08:30.103730 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.103710    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:08:30.105171 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.105153    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8"
Nov 01 05:08:30.185408 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.185323    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a52e528e-c723-42b6-8d1a-03c3675518b9-util\" (UniqueName: \"kubernetes.io/empty-dir/a52e528e-c723-42b6-8d1a-03c3675518b9-util\") pod \"a52e528e-c723-42b6-8d1a-03c3675518b9\" (UID: \"a52e528e-c723-42b6-8d1a-03c3675518b9\") "
Nov 01 05:08:30.185408 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.185387    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/070c9b8b-2af9-40a5-b85f-2ef0446e012e-util\" (UniqueName: \"kubernetes.io/empty-dir/070c9b8b-2af9-40a5-b85f-2ef0446e012e-util\") pod \"070c9b8b-2af9-40a5-b85f-2ef0446e012e\" (UID: \"070c9b8b-2af9-40a5-b85f-2ef0446e012e\") "
Nov 01 05:08:30.185607 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.185417    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a52e528e-c723-42b6-8d1a-03c3675518b9-kube-api-access-zww6r\" (UniqueName: \"kubernetes.io/projected/a52e528e-c723-42b6-8d1a-03c3675518b9-kube-api-access-zww6r\") pod \"a52e528e-c723-42b6-8d1a-03c3675518b9\" (UID: \"a52e528e-c723-42b6-8d1a-03c3675518b9\") "
Nov 01 05:08:30.185607 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.185451    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a52e528e-c723-42b6-8d1a-03c3675518b9-bundle\" (UniqueName: \"kubernetes.io/empty-dir/a52e528e-c723-42b6-8d1a-03c3675518b9-bundle\") pod \"a52e528e-c723-42b6-8d1a-03c3675518b9\" (UID: \"a52e528e-c723-42b6-8d1a-03c3675518b9\") "
Nov 01 05:08:30.185607 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.185497    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/070c9b8b-2af9-40a5-b85f-2ef0446e012e-bundle\" (UniqueName: \"kubernetes.io/empty-dir/070c9b8b-2af9-40a5-b85f-2ef0446e012e-bundle\") pod \"070c9b8b-2af9-40a5-b85f-2ef0446e012e\" (UID: \"070c9b8b-2af9-40a5-b85f-2ef0446e012e\") "
Nov 01 05:08:30.185607 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.185527    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/070c9b8b-2af9-40a5-b85f-2ef0446e012e-kube-api-access-7t9t2\" (UniqueName: \"kubernetes.io/projected/070c9b8b-2af9-40a5-b85f-2ef0446e012e-kube-api-access-7t9t2\") pod \"070c9b8b-2af9-40a5-b85f-2ef0446e012e\" (UID: \"070c9b8b-2af9-40a5-b85f-2ef0446e012e\") "
Nov 01 05:08:30.186171 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.186141    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a52e528e-c723-42b6-8d1a-03c3675518b9-bundle" pod "a52e528e-c723-42b6-8d1a-03c3675518b9" (UID: "a52e528e-c723-42b6-8d1a-03c3675518b9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:08:30.186516 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.186483    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/070c9b8b-2af9-40a5-b85f-2ef0446e012e-bundle" pod "070c9b8b-2af9-40a5-b85f-2ef0446e012e" (UID: "070c9b8b-2af9-40a5-b85f-2ef0446e012e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:08:30.187828 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.187808    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/070c9b8b-2af9-40a5-b85f-2ef0446e012e-kube-api-access-7t9t2" pod "070c9b8b-2af9-40a5-b85f-2ef0446e012e" (UID: "070c9b8b-2af9-40a5-b85f-2ef0446e012e"). InnerVolumeSpecName "kube-api-access-7t9t2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:08:30.187920 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.187852    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a52e528e-c723-42b6-8d1a-03c3675518b9-kube-api-access-zww6r" pod "a52e528e-c723-42b6-8d1a-03c3675518b9" (UID: "a52e528e-c723-42b6-8d1a-03c3675518b9"). InnerVolumeSpecName "kube-api-access-zww6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:08:30.191611 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.191588    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/070c9b8b-2af9-40a5-b85f-2ef0446e012e-util" pod "070c9b8b-2af9-40a5-b85f-2ef0446e012e" (UID: "070c9b8b-2af9-40a5-b85f-2ef0446e012e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:08:30.193193 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.193173    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a52e528e-c723-42b6-8d1a-03c3675518b9-util" pod "a52e528e-c723-42b6-8d1a-03c3675518b9" (UID: "a52e528e-c723-42b6-8d1a-03c3675518b9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:08:30.286610 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.286578    4864 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/070c9b8b-2af9-40a5-b85f-2ef0446e012e-bundle\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:30.286610 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.286610    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7t9t2\" (UniqueName: \"kubernetes.io/projected/070c9b8b-2af9-40a5-b85f-2ef0446e012e-kube-api-access-7t9t2\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:30.286610 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.286621    4864 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a52e528e-c723-42b6-8d1a-03c3675518b9-util\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:30.286939 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.286631    4864 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/070c9b8b-2af9-40a5-b85f-2ef0446e012e-util\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:30.286939 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.286640    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zww6r\" (UniqueName: \"kubernetes.io/projected/a52e528e-c723-42b6-8d1a-03c3675518b9-kube-api-access-zww6r\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:30.286939 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.286648    4864 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a52e528e-c723-42b6-8d1a-03c3675518b9-bundle\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:30.402513 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.402459    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr" podStartSLOduration=3.77175652 podStartE2EDuration="44.402443948s" totalImagesPullingTime="40.630687428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:07:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:08:30.401942196 +0000 UTC m=+327.292318173" watchObservedRunningTime="2026-11-01 05:08:30.402443948 +0000 UTC m=+327.292819918"
Nov 01 05:08:30.963682 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.963647    4864 generic.go:356] "Generic (PLEG): container finished" podID="4a5ceca4-7882-4c74-9758-45e4bb783b66" containerID="e62a51d84997b7a2ae2c1bfe5c5e9fbd527c2893d3b4d9b3ad03bf65461dcb45" exitCode=0
Nov 01 05:08:30.964200 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.963724    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf" event={"ID":"4a5ceca4-7882-4c74-9758-45e4bb783b66","Type":"ContainerDied","Data":"e62a51d84997b7a2ae2c1bfe5c5e9fbd527c2893d3b4d9b3ad03bf65461dcb45"}
Nov 01 05:08:30.966051 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.966029    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv"
Nov 01 05:08:30.966158 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.966069    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgkrvv" event={"ID":"070c9b8b-2af9-40a5-b85f-2ef0446e012e","Type":"ContainerDied","Data":"6ac2d4b529f523c95ec73d1df0a3b9621ea2156669be6bd117f08d3e8317d913"}
Nov 01 05:08:30.966158 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.966122    4864 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ac2d4b529f523c95ec73d1df0a3b9621ea2156669be6bd117f08d3e8317d913"
Nov 01 05:08:30.967723 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.967704    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:08:30.968515 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.968494    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8"
Nov 01 05:08:30.968605 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.968515    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8" event={"ID":"a52e528e-c723-42b6-8d1a-03c3675518b9","Type":"ContainerDied","Data":"b320d3c79fbd8723ca22866c7318750b0d099e961798972fd544f262ceb65f9c"}
Nov 01 05:08:30.968605 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.968552    4864 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b320d3c79fbd8723ca22866c7318750b0d099e961798972fd544f262ceb65f9c"
Nov 01 05:08:30.971110 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.971082    4864 generic.go:356] "Generic (PLEG): container finished" podID="1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf" containerID="4d322fa5514555d6f5f717350b1afd375e20e9a2c46ecf885a04783a76b4f068" exitCode=0
Nov 01 05:08:30.971229 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:30.971196    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr" event={"ID":"1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf","Type":"ContainerDied","Data":"4d322fa5514555d6f5f717350b1afd375e20e9a2c46ecf885a04783a76b4f068"}
Nov 01 05:08:31.099773 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.099749    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf"
Nov 01 05:08:31.133385 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.133359    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5"
Nov 01 05:08:31.193515 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.193480    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b47e4d74-35df-4229-88f2-7d775d66ba9c-util\" (UniqueName: \"kubernetes.io/empty-dir/b47e4d74-35df-4229-88f2-7d775d66ba9c-util\") pod \"b47e4d74-35df-4229-88f2-7d775d66ba9c\" (UID: \"b47e4d74-35df-4229-88f2-7d775d66ba9c\") "
Nov 01 05:08:31.193678 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.193553    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b47e4d74-35df-4229-88f2-7d775d66ba9c-kube-api-access-zp62m\" (UniqueName: \"kubernetes.io/projected/b47e4d74-35df-4229-88f2-7d775d66ba9c-kube-api-access-zp62m\") pod \"b47e4d74-35df-4229-88f2-7d775d66ba9c\" (UID: \"b47e4d74-35df-4229-88f2-7d775d66ba9c\") "
Nov 01 05:08:31.193678 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.193606    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a5ceca4-7882-4c74-9758-45e4bb783b66-util\" (UniqueName: \"kubernetes.io/empty-dir/4a5ceca4-7882-4c74-9758-45e4bb783b66-util\") pod \"4a5ceca4-7882-4c74-9758-45e4bb783b66\" (UID: \"4a5ceca4-7882-4c74-9758-45e4bb783b66\") "
Nov 01 05:08:31.193836 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.193718    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b47e4d74-35df-4229-88f2-7d775d66ba9c-bundle\" (UniqueName: \"kubernetes.io/empty-dir/b47e4d74-35df-4229-88f2-7d775d66ba9c-bundle\") pod \"b47e4d74-35df-4229-88f2-7d775d66ba9c\" (UID: \"b47e4d74-35df-4229-88f2-7d775d66ba9c\") "
Nov 01 05:08:31.193836 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.193758    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a5ceca4-7882-4c74-9758-45e4bb783b66-kube-api-access-ml2pl\" (UniqueName: \"kubernetes.io/projected/4a5ceca4-7882-4c74-9758-45e4bb783b66-kube-api-access-ml2pl\") pod \"4a5ceca4-7882-4c74-9758-45e4bb783b66\" (UID: \"4a5ceca4-7882-4c74-9758-45e4bb783b66\") "
Nov 01 05:08:31.193836 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.193828    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a5ceca4-7882-4c74-9758-45e4bb783b66-bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a5ceca4-7882-4c74-9758-45e4bb783b66-bundle\") pod \"4a5ceca4-7882-4c74-9758-45e4bb783b66\" (UID: \"4a5ceca4-7882-4c74-9758-45e4bb783b66\") "
Nov 01 05:08:31.194895 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.194867    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b47e4d74-35df-4229-88f2-7d775d66ba9c-bundle" pod "b47e4d74-35df-4229-88f2-7d775d66ba9c" (UID: "b47e4d74-35df-4229-88f2-7d775d66ba9c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:08:31.195046 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.195008    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a5ceca4-7882-4c74-9758-45e4bb783b66-bundle" pod "4a5ceca4-7882-4c74-9758-45e4bb783b66" (UID: "4a5ceca4-7882-4c74-9758-45e4bb783b66"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:08:31.196238 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.196203    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b47e4d74-35df-4229-88f2-7d775d66ba9c-kube-api-access-zp62m" pod "b47e4d74-35df-4229-88f2-7d775d66ba9c" (UID: "b47e4d74-35df-4229-88f2-7d775d66ba9c"). InnerVolumeSpecName "kube-api-access-zp62m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:08:31.196316 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.196253    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a5ceca4-7882-4c74-9758-45e4bb783b66-kube-api-access-ml2pl" pod "4a5ceca4-7882-4c74-9758-45e4bb783b66" (UID: "4a5ceca4-7882-4c74-9758-45e4bb783b66"). InnerVolumeSpecName "kube-api-access-ml2pl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:08:31.200336 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.200314    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a5ceca4-7882-4c74-9758-45e4bb783b66-util" pod "4a5ceca4-7882-4c74-9758-45e4bb783b66" (UID: "4a5ceca4-7882-4c74-9758-45e4bb783b66"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:08:31.200646 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.200624    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b47e4d74-35df-4229-88f2-7d775d66ba9c-util" pod "b47e4d74-35df-4229-88f2-7d775d66ba9c" (UID: "b47e4d74-35df-4229-88f2-7d775d66ba9c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:08:31.295360 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.295328    4864 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b47e4d74-35df-4229-88f2-7d775d66ba9c-util\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:31.295360 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.295355    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zp62m\" (UniqueName: \"kubernetes.io/projected/b47e4d74-35df-4229-88f2-7d775d66ba9c-kube-api-access-zp62m\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:31.295543 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.295369    4864 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a5ceca4-7882-4c74-9758-45e4bb783b66-util\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:31.295543 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.295382    4864 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b47e4d74-35df-4229-88f2-7d775d66ba9c-bundle\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:31.295543 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.295390    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ml2pl\" (UniqueName: \"kubernetes.io/projected/4a5ceca4-7882-4c74-9758-45e4bb783b66-kube-api-access-ml2pl\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:31.295543 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.295398    4864 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a5ceca4-7882-4c74-9758-45e4bb783b66-bundle\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:31.978314 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.978287    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5"
Nov 01 05:08:31.978883 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.978315    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwzlq5" event={"ID":"b47e4d74-35df-4229-88f2-7d775d66ba9c","Type":"ContainerDied","Data":"9b380734af36efd6f3a11956221f12a3e97f4e4a458fddf46fb95d3bfd7b9ef8"}
Nov 01 05:08:31.978883 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.978364    4864 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b380734af36efd6f3a11956221f12a3e97f4e4a458fddf46fb95d3bfd7b9ef8"
Nov 01 05:08:31.980343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.980314    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf" event={"ID":"4a5ceca4-7882-4c74-9758-45e4bb783b66","Type":"ContainerDied","Data":"d833cc91981061adad8b394f2c39e608d54bcb1d9614205e2c9c9f68c4e76e64"}
Nov 01 05:08:31.980489 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.980388    4864 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d833cc91981061adad8b394f2c39e608d54bcb1d9614205e2c9c9f68c4e76e64"
Nov 01 05:08:31.980591 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:31.980574    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahzfgf"
Nov 01 05:08:32.101201 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:32.101178    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr"
Nov 01 05:08:32.201945 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:32.201916    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf-bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf-bundle\") pod \"1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf\" (UID: \"1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf\") "
Nov 01 05:08:32.202116 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:32.201964    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf-kube-api-access-qkpbn\" (UniqueName: \"kubernetes.io/projected/1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf-kube-api-access-qkpbn\") pod \"1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf\" (UID: \"1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf\") "
Nov 01 05:08:32.202116 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:32.202015    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf-util\" (UniqueName: \"kubernetes.io/empty-dir/1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf-util\") pod \"1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf\" (UID: \"1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf\") "
Nov 01 05:08:32.202409 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:32.202372    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf-bundle" pod "1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf" (UID: "1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:08:32.204242 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:32.204218    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf-kube-api-access-qkpbn" pod "1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf" (UID: "1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf"). InnerVolumeSpecName "kube-api-access-qkpbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:08:32.207641 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:32.207621    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf-util" pod "1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf" (UID: "1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:08:32.307948 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:32.303386    4864 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf-util\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:32.307948 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:32.303427    4864 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf-bundle\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:32.307948 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:32.303441    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkpbn\" (UniqueName: \"kubernetes.io/projected/1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf-kube-api-access-qkpbn\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:08:32.986878 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:32.986848    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr"
Nov 01 05:08:32.987351 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:32.986842    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6444d6nr" event={"ID":"1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf","Type":"ContainerDied","Data":"7b7f47a2a9e68811e543ab41cb543c0ac59959f756a0faa5f28ec0f54791824b"}
Nov 01 05:08:32.987351 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:32.986993    4864 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b7f47a2a9e68811e543ab41cb543c0ac59959f756a0faa5f28ec0f54791824b"
Nov 01 05:08:35.051928 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.051888    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"]
Nov 01 05:08:35.052478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052319    4864 cpu_manager.go:414] "containerMap: removing container" podUID="1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf" containerName="pull"
Nov 01 05:08:35.052478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052336    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf" containerName="pull"
Nov 01 05:08:35.052478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052348    4864 cpu_manager.go:414] "containerMap: removing container" podUID="14396006-01c5-4daa-a0f2-8c863285b5bd" containerName="pull"
Nov 01 05:08:35.052478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052355    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14396006-01c5-4daa-a0f2-8c863285b5bd" containerName="pull"
Nov 01 05:08:35.052478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052370    4864 cpu_manager.go:414] "containerMap: removing container" podUID="50f35a8f-80ec-41e3-9255-e1358ca8ca52" containerName="util"
Nov 01 05:08:35.052478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052377    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50f35a8f-80ec-41e3-9255-e1358ca8ca52" containerName="util"
Nov 01 05:08:35.052478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052386    4864 cpu_manager.go:414] "containerMap: removing container" podUID="1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf" containerName="util"
Nov 01 05:08:35.052478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052394    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf" containerName="util"
Nov 01 05:08:35.052478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052410    4864 cpu_manager.go:414] "containerMap: removing container" podUID="a52e528e-c723-42b6-8d1a-03c3675518b9" containerName="pull"
Nov 01 05:08:35.052478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052417    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a52e528e-c723-42b6-8d1a-03c3675518b9" containerName="pull"
Nov 01 05:08:35.052478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052427    4864 cpu_manager.go:414] "containerMap: removing container" podUID="ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68" containerName="extract"
Nov 01 05:08:35.052478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052434    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68" containerName="extract"
Nov 01 05:08:35.052478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052443    4864 cpu_manager.go:414] "containerMap: removing container" podUID="14396006-01c5-4daa-a0f2-8c863285b5bd" containerName="util"
Nov 01 05:08:35.052478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052449    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14396006-01c5-4daa-a0f2-8c863285b5bd" containerName="util"
Nov 01 05:08:35.052478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052464    4864 cpu_manager.go:414] "containerMap: removing container" podUID="967c127b-a8a7-41cc-aef7-96c2cc3c36cb" containerName="util"
Nov 01 05:08:35.052478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052471    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="967c127b-a8a7-41cc-aef7-96c2cc3c36cb" containerName="util"
Nov 01 05:08:35.052478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052482    4864 cpu_manager.go:414] "containerMap: removing container" podUID="a52e528e-c723-42b6-8d1a-03c3675518b9" containerName="util"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052488    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a52e528e-c723-42b6-8d1a-03c3675518b9" containerName="util"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052502    4864 cpu_manager.go:414] "containerMap: removing container" podUID="b47e4d74-35df-4229-88f2-7d775d66ba9c" containerName="util"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052508    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b47e4d74-35df-4229-88f2-7d775d66ba9c" containerName="util"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052516    4864 cpu_manager.go:414] "containerMap: removing container" podUID="50f35a8f-80ec-41e3-9255-e1358ca8ca52" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052523    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50f35a8f-80ec-41e3-9255-e1358ca8ca52" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052535    4864 cpu_manager.go:414] "containerMap: removing container" podUID="070c9b8b-2af9-40a5-b85f-2ef0446e012e" containerName="pull"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052542    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="070c9b8b-2af9-40a5-b85f-2ef0446e012e" containerName="pull"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052552    4864 cpu_manager.go:414] "containerMap: removing container" podUID="b47e4d74-35df-4229-88f2-7d775d66ba9c" containerName="pull"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052558    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b47e4d74-35df-4229-88f2-7d775d66ba9c" containerName="pull"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052567    4864 cpu_manager.go:414] "containerMap: removing container" podUID="b47e4d74-35df-4229-88f2-7d775d66ba9c" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052573    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b47e4d74-35df-4229-88f2-7d775d66ba9c" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052588    4864 cpu_manager.go:414] "containerMap: removing container" podUID="14396006-01c5-4daa-a0f2-8c863285b5bd" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052594    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14396006-01c5-4daa-a0f2-8c863285b5bd" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052605    4864 cpu_manager.go:414] "containerMap: removing container" podUID="a52e528e-c723-42b6-8d1a-03c3675518b9" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052612    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a52e528e-c723-42b6-8d1a-03c3675518b9" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052622    4864 cpu_manager.go:414] "containerMap: removing container" podUID="4a5ceca4-7882-4c74-9758-45e4bb783b66" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052628    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a5ceca4-7882-4c74-9758-45e4bb783b66" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052639    4864 cpu_manager.go:414] "containerMap: removing container" podUID="967c127b-a8a7-41cc-aef7-96c2cc3c36cb" containerName="pull"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052646    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="967c127b-a8a7-41cc-aef7-96c2cc3c36cb" containerName="pull"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052658    4864 cpu_manager.go:414] "containerMap: removing container" podUID="ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68" containerName="pull"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052664    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68" containerName="pull"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052673    4864 cpu_manager.go:414] "containerMap: removing container" podUID="1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052680    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052694    4864 cpu_manager.go:414] "containerMap: removing container" podUID="4a5ceca4-7882-4c74-9758-45e4bb783b66" containerName="pull"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052701    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a5ceca4-7882-4c74-9758-45e4bb783b66" containerName="pull"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052709    4864 cpu_manager.go:414] "containerMap: removing container" podUID="967c127b-a8a7-41cc-aef7-96c2cc3c36cb" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052716    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="967c127b-a8a7-41cc-aef7-96c2cc3c36cb" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052727    4864 cpu_manager.go:414] "containerMap: removing container" podUID="070c9b8b-2af9-40a5-b85f-2ef0446e012e" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052734    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="070c9b8b-2af9-40a5-b85f-2ef0446e012e" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052746    4864 cpu_manager.go:414] "containerMap: removing container" podUID="ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68" containerName="util"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052753    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68" containerName="util"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052761    4864 cpu_manager.go:414] "containerMap: removing container" podUID="50f35a8f-80ec-41e3-9255-e1358ca8ca52" containerName="pull"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052769    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50f35a8f-80ec-41e3-9255-e1358ca8ca52" containerName="pull"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052779    4864 cpu_manager.go:414] "containerMap: removing container" podUID="4a5ceca4-7882-4c74-9758-45e4bb783b66" containerName="util"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052808    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a5ceca4-7882-4c74-9758-45e4bb783b66" containerName="util"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052817    4864 cpu_manager.go:414] "containerMap: removing container" podUID="070c9b8b-2af9-40a5-b85f-2ef0446e012e" containerName="util"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052824    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="070c9b8b-2af9-40a5-b85f-2ef0446e012e" containerName="util"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052917    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="ebd5b0cc-90d0-4f17-9ffe-f023dd3f2e68" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052932    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="50f35a8f-80ec-41e3-9255-e1358ca8ca52" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052942    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="b47e4d74-35df-4229-88f2-7d775d66ba9c" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052952    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="14396006-01c5-4daa-a0f2-8c863285b5bd" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052961    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="a52e528e-c723-42b6-8d1a-03c3675518b9" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052971    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a5ceca4-7882-4c74-9758-45e4bb783b66" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052979    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e13f47f-2e8e-47e0-8eba-1c2a4f17ffbf" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052988    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="967c127b-a8a7-41cc-aef7-96c2cc3c36cb" containerName="extract"
Nov 01 05:08:35.053343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.052997    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="070c9b8b-2af9-40a5-b85f-2ef0446e012e" containerName="extract"
Nov 01 05:08:35.055369 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.055351    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:35.112143 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.112116    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 01 05:08:35.112345 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.112324    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 01 05:08:35.112404 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.112390    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 01 05:08:35.112454 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.112413    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-75h7v\""
Nov 01 05:08:35.127743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.127704    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/579cc219-e419-42b6-92cb-8832293be15e-webhook-cert\") pod \"opendatahub-operator-controller-manager-85d75859fc-vkmcw\" (UID: \"579cc219-e419-42b6-92cb-8832293be15e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:35.127854 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.127756    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/579cc219-e419-42b6-92cb-8832293be15e-manifests\") pod \"opendatahub-operator-controller-manager-85d75859fc-vkmcw\" (UID: \"579cc219-e419-42b6-92cb-8832293be15e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:35.127854 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.127822    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/579cc219-e419-42b6-92cb-8832293be15e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85d75859fc-vkmcw\" (UID: \"579cc219-e419-42b6-92cb-8832293be15e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:35.127934 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.127912    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/579cc219-e419-42b6-92cb-8832293be15e-metrics-cert\") pod \"opendatahub-operator-controller-manager-85d75859fc-vkmcw\" (UID: \"579cc219-e419-42b6-92cb-8832293be15e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:35.127934 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.127930    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/579cc219-e419-42b6-92cb-8832293be15e-tmp\") pod \"opendatahub-operator-controller-manager-85d75859fc-vkmcw\" (UID: \"579cc219-e419-42b6-92cb-8832293be15e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:35.128008 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.127966    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kgsm\" (UniqueName: \"kubernetes.io/projected/579cc219-e419-42b6-92cb-8832293be15e-kube-api-access-2kgsm\") pod \"opendatahub-operator-controller-manager-85d75859fc-vkmcw\" (UID: \"579cc219-e419-42b6-92cb-8832293be15e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:35.161544 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.161519    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 01 05:08:35.161663 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.161567    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 01 05:08:35.173835 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.173809    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"]
Nov 01 05:08:35.228735 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.228702    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/579cc219-e419-42b6-92cb-8832293be15e-metrics-cert\") pod \"opendatahub-operator-controller-manager-85d75859fc-vkmcw\" (UID: \"579cc219-e419-42b6-92cb-8832293be15e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:35.228735 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.228733    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/579cc219-e419-42b6-92cb-8832293be15e-tmp\") pod \"opendatahub-operator-controller-manager-85d75859fc-vkmcw\" (UID: \"579cc219-e419-42b6-92cb-8832293be15e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:35.228983 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.228763    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kgsm\" (UniqueName: \"kubernetes.io/projected/579cc219-e419-42b6-92cb-8832293be15e-kube-api-access-2kgsm\") pod \"opendatahub-operator-controller-manager-85d75859fc-vkmcw\" (UID: \"579cc219-e419-42b6-92cb-8832293be15e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:35.228983 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.228795    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/579cc219-e419-42b6-92cb-8832293be15e-webhook-cert\") pod \"opendatahub-operator-controller-manager-85d75859fc-vkmcw\" (UID: \"579cc219-e419-42b6-92cb-8832293be15e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:35.228983 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.228828    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/579cc219-e419-42b6-92cb-8832293be15e-manifests\") pod \"opendatahub-operator-controller-manager-85d75859fc-vkmcw\" (UID: \"579cc219-e419-42b6-92cb-8832293be15e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:35.228983 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.228858    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/579cc219-e419-42b6-92cb-8832293be15e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85d75859fc-vkmcw\" (UID: \"579cc219-e419-42b6-92cb-8832293be15e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:35.229251 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.229219    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/579cc219-e419-42b6-92cb-8832293be15e-manifests\") pod \"opendatahub-operator-controller-manager-85d75859fc-vkmcw\" (UID: \"579cc219-e419-42b6-92cb-8832293be15e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:35.231441 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.231412    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/579cc219-e419-42b6-92cb-8832293be15e-tmp\") pod \"opendatahub-operator-controller-manager-85d75859fc-vkmcw\" (UID: \"579cc219-e419-42b6-92cb-8832293be15e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:35.231544 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.231490    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/579cc219-e419-42b6-92cb-8832293be15e-webhook-cert\") pod \"opendatahub-operator-controller-manager-85d75859fc-vkmcw\" (UID: \"579cc219-e419-42b6-92cb-8832293be15e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:35.231633 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.231608    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/579cc219-e419-42b6-92cb-8832293be15e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85d75859fc-vkmcw\" (UID: \"579cc219-e419-42b6-92cb-8832293be15e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:35.231680 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.231662    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/579cc219-e419-42b6-92cb-8832293be15e-metrics-cert\") pod \"opendatahub-operator-controller-manager-85d75859fc-vkmcw\" (UID: \"579cc219-e419-42b6-92cb-8832293be15e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:35.289321 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.289290    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kgsm\" (UniqueName: \"kubernetes.io/projected/579cc219-e419-42b6-92cb-8832293be15e-kube-api-access-2kgsm\") pod \"opendatahub-operator-controller-manager-85d75859fc-vkmcw\" (UID: \"579cc219-e419-42b6-92cb-8832293be15e\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:35.367413 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.367379    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:35.866160 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:35.866135    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"]
Nov 01 05:08:35.867014 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:08:35.866990    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod579cc219_e419_42b6_92cb_8832293be15e.slice/crio-3958b1fd1b76661c2c9cd918b779608250f1624480b86b1eca8c3d4ce45cb8b3 WatchSource:0}: Error finding container 3958b1fd1b76661c2c9cd918b779608250f1624480b86b1eca8c3d4ce45cb8b3: Status 404 returned error can't find the container with id 3958b1fd1b76661c2c9cd918b779608250f1624480b86b1eca8c3d4ce45cb8b3
Nov 01 05:08:36.003066 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:36.003024    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw" event={"ID":"579cc219-e419-42b6-92cb-8832293be15e","Type":"ContainerStarted","Data":"3958b1fd1b76661c2c9cd918b779608250f1624480b86b1eca8c3d4ce45cb8b3"}
Nov 01 05:08:39.115509 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:39.115473    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-qbk8d"]
Nov 01 05:08:39.121968 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:39.121943    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-qbk8d"
Nov 01 05:08:39.166090 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:39.166030    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz9sg\" (UniqueName: \"kubernetes.io/projected/c35764b7-016b-46b2-b0fa-4d03963ef38d-kube-api-access-tz9sg\") pod \"jobset-operator-674cd59dc5-qbk8d\" (UID: \"c35764b7-016b-46b2-b0fa-4d03963ef38d\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-qbk8d"
Nov 01 05:08:39.166240 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:39.166155    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c35764b7-016b-46b2-b0fa-4d03963ef38d-tmp\") pod \"jobset-operator-674cd59dc5-qbk8d\" (UID: \"c35764b7-016b-46b2-b0fa-4d03963ef38d\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-qbk8d"
Nov 01 05:08:39.227337 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:39.227295    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Nov 01 05:08:39.227337 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:39.227316    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Nov 01 05:08:39.227564 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:39.227396    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-rqprj\""
Nov 01 05:08:39.229296 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:39.229274    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-qbk8d"]
Nov 01 05:08:39.267412 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:39.267376    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c35764b7-016b-46b2-b0fa-4d03963ef38d-tmp\") pod \"jobset-operator-674cd59dc5-qbk8d\" (UID: \"c35764b7-016b-46b2-b0fa-4d03963ef38d\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-qbk8d"
Nov 01 05:08:39.267585 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:39.267475    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz9sg\" (UniqueName: \"kubernetes.io/projected/c35764b7-016b-46b2-b0fa-4d03963ef38d-kube-api-access-tz9sg\") pod \"jobset-operator-674cd59dc5-qbk8d\" (UID: \"c35764b7-016b-46b2-b0fa-4d03963ef38d\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-qbk8d"
Nov 01 05:08:39.267733 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:39.267711    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c35764b7-016b-46b2-b0fa-4d03963ef38d-tmp\") pod \"jobset-operator-674cd59dc5-qbk8d\" (UID: \"c35764b7-016b-46b2-b0fa-4d03963ef38d\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-qbk8d"
Nov 01 05:08:39.329979 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:39.329951    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz9sg\" (UniqueName: \"kubernetes.io/projected/c35764b7-016b-46b2-b0fa-4d03963ef38d-kube-api-access-tz9sg\") pod \"jobset-operator-674cd59dc5-qbk8d\" (UID: \"c35764b7-016b-46b2-b0fa-4d03963ef38d\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-qbk8d"
Nov 01 05:08:39.453160 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:39.453072    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-qbk8d"
Nov 01 05:08:39.841486 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:39.841453    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-qbk8d"]
Nov 01 05:08:39.844243 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:08:39.844207    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc35764b7_016b_46b2_b0fa_4d03963ef38d.slice/crio-1910186ef68eab43e42c913db2a0fb7f1acc3d1bd1afb2ed63d64ac2e1129f0c WatchSource:0}: Error finding container 1910186ef68eab43e42c913db2a0fb7f1acc3d1bd1afb2ed63d64ac2e1129f0c: Status 404 returned error can't find the container with id 1910186ef68eab43e42c913db2a0fb7f1acc3d1bd1afb2ed63d64ac2e1129f0c
Nov 01 05:08:40.025968 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:40.025924    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-qbk8d" event={"ID":"c35764b7-016b-46b2-b0fa-4d03963ef38d","Type":"ContainerStarted","Data":"1910186ef68eab43e42c913db2a0fb7f1acc3d1bd1afb2ed63d64ac2e1129f0c"}
Nov 01 05:08:41.031748 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:41.031663    4864 generic.go:356] "Generic (PLEG): container finished" podID="579cc219-e419-42b6-92cb-8832293be15e" containerID="af291c6a5d548a59646d6277b431d302e9e8b56ad385e132a808a2d77a3fc313" exitCode=0
Nov 01 05:08:41.031748 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:41.031719    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw" event={"ID":"579cc219-e419-42b6-92cb-8832293be15e","Type":"ContainerDied","Data":"af291c6a5d548a59646d6277b431d302e9e8b56ad385e132a808a2d77a3fc313"}
Nov 01 05:08:42.040371 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:42.040332    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw" event={"ID":"579cc219-e419-42b6-92cb-8832293be15e","Type":"ContainerStarted","Data":"590f1bec3f2e378063dc43c480bef31e2a0e5f15b548971ca67ad3415a3be503"}
Nov 01 05:08:42.040816 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:42.040474    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:42.178607 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:42.178506    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw" podStartSLOduration=2.560260868 podStartE2EDuration="8.178487171s" totalImagesPullingTime="5.08455111s" totalInitContainerRuntime="533.675193ms" isStatefulPod=false podCreationTimestamp="2026-11-01 05:08:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:08:42.177089777 +0000 UTC m=+339.067465756" watchObservedRunningTime="2026-11-01 05:08:42.178487171 +0000 UTC m=+339.068863148"
Nov 01 05:08:43.046684 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:43.046600    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-qbk8d" event={"ID":"c35764b7-016b-46b2-b0fa-4d03963ef38d","Type":"ContainerStarted","Data":"f35b54914311f8b4ad68828fe7094cfc1366fb0c5b25507ef444bd401e8b6981"}
Nov 01 05:08:43.164208 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:43.164155    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-qbk8d" podStartSLOduration=1.246290223 podStartE2EDuration="4.164139111s" totalImagesPullingTime="2.917848888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:08:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:08:43.162565999 +0000 UTC m=+340.052941974" watchObservedRunningTime="2026-11-01 05:08:43.164139111 +0000 UTC m=+340.054515088"
Nov 01 05:08:48.743748 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:48.743709    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-xq2f6"]
Nov 01 05:08:48.747017 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:48.747001    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-xq2f6"
Nov 01 05:08:48.802795 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:48.802754    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-xq2f6"]
Nov 01 05:08:48.850133 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:48.850107    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Nov 01 05:08:48.850133 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:48.850109    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Nov 01 05:08:48.850341 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:48.850144    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-5tm96\""
Nov 01 05:08:48.860478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:48.860453    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a890624-f5da-4eea-9ad9-e1ae4cf4bd69-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-xq2f6\" (UID: \"5a890624-f5da-4eea-9ad9-e1ae4cf4bd69\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-xq2f6"
Nov 01 05:08:48.860595 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:48.860578    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wtt9\" (UniqueName: \"kubernetes.io/projected/5a890624-f5da-4eea-9ad9-e1ae4cf4bd69-kube-api-access-9wtt9\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-xq2f6\" (UID: \"5a890624-f5da-4eea-9ad9-e1ae4cf4bd69\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-xq2f6"
Nov 01 05:08:48.961628 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:48.961597    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wtt9\" (UniqueName: \"kubernetes.io/projected/5a890624-f5da-4eea-9ad9-e1ae4cf4bd69-kube-api-access-9wtt9\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-xq2f6\" (UID: \"5a890624-f5da-4eea-9ad9-e1ae4cf4bd69\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-xq2f6"
Nov 01 05:08:48.961628 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:48.961633    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a890624-f5da-4eea-9ad9-e1ae4cf4bd69-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-xq2f6\" (UID: \"5a890624-f5da-4eea-9ad9-e1ae4cf4bd69\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-xq2f6"
Nov 01 05:08:48.962038 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:48.962019    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a890624-f5da-4eea-9ad9-e1ae4cf4bd69-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-xq2f6\" (UID: \"5a890624-f5da-4eea-9ad9-e1ae4cf4bd69\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-xq2f6"
Nov 01 05:08:49.024241 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:49.024162    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wtt9\" (UniqueName: \"kubernetes.io/projected/5a890624-f5da-4eea-9ad9-e1ae4cf4bd69-kube-api-access-9wtt9\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-xq2f6\" (UID: \"5a890624-f5da-4eea-9ad9-e1ae4cf4bd69\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-xq2f6"
Nov 01 05:08:49.057022 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:49.056987    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-xq2f6"
Nov 01 05:08:49.530759 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:49.530730    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-xq2f6"]
Nov 01 05:08:49.532940 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:08:49.532915    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a890624_f5da_4eea_9ad9_e1ae4cf4bd69.slice/crio-eb36d2af3bcf79c825f8e9e55a24c9ed0c16845f8ea50db0b0f1f9cd54fd4038 WatchSource:0}: Error finding container eb36d2af3bcf79c825f8e9e55a24c9ed0c16845f8ea50db0b0f1f9cd54fd4038: Status 404 returned error can't find the container with id eb36d2af3bcf79c825f8e9e55a24c9ed0c16845f8ea50db0b0f1f9cd54fd4038
Nov 01 05:08:50.079403 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:50.079364    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-xq2f6" event={"ID":"5a890624-f5da-4eea-9ad9-e1ae4cf4bd69","Type":"ContainerStarted","Data":"eb36d2af3bcf79c825f8e9e55a24c9ed0c16845f8ea50db0b0f1f9cd54fd4038"}
Nov 01 05:08:52.482401 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:52.482370    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-k2nkj"]
Nov 01 05:08:52.484556 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:52.484529    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-k2nkj"
Nov 01 05:08:52.525010 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:08:52.524973    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:08:52.536907 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:52.536881    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 01 05:08:52.536907 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:52.536879    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 01 05:08:52.586289 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:52.586259    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-tdpzh\""
Nov 01 05:08:52.592183 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:52.592150    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-k2nkj"]
Nov 01 05:08:52.593724 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:52.593696    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g85zl\" (UniqueName: \"kubernetes.io/projected/cb788d11-cc64-4f1c-b1e0-0bed74754f03-kube-api-access-g85zl\") pod \"authorino-operator-7b7bb4bc77-k2nkj\" (UID: \"cb788d11-cc64-4f1c-b1e0-0bed74754f03\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-k2nkj"
Nov 01 05:08:52.694753 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:52.694725    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g85zl\" (UniqueName: \"kubernetes.io/projected/cb788d11-cc64-4f1c-b1e0-0bed74754f03-kube-api-access-g85zl\") pod \"authorino-operator-7b7bb4bc77-k2nkj\" (UID: \"cb788d11-cc64-4f1c-b1e0-0bed74754f03\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-k2nkj"
Nov 01 05:08:52.755825 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:52.755732    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g85zl\" (UniqueName: \"kubernetes.io/projected/cb788d11-cc64-4f1c-b1e0-0bed74754f03-kube-api-access-g85zl\") pod \"authorino-operator-7b7bb4bc77-k2nkj\" (UID: \"cb788d11-cc64-4f1c-b1e0-0bed74754f03\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-k2nkj"
Nov 01 05:08:52.797555 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:52.797518    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-k2nkj"
Nov 01 05:08:53.049351 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:53.049270    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw"
Nov 01 05:08:53.095762 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:53.095720    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-xq2f6" event={"ID":"5a890624-f5da-4eea-9ad9-e1ae4cf4bd69","Type":"ContainerStarted","Data":"06a6437a1e3e1dd952d1996c016231a9c3952fc736bb05b8b490a91005a3d690"}
Nov 01 05:08:53.175223 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:53.175194    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-k2nkj"]
Nov 01 05:08:53.176951 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:08:53.176921    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb788d11_cc64_4f1c_b1e0_0bed74754f03.slice/crio-e900d62a9a3f85f3d491e1117062073e3281da3318761b1edac3b2a0fa642d31 WatchSource:0}: Error finding container e900d62a9a3f85f3d491e1117062073e3281da3318761b1edac3b2a0fa642d31: Status 404 returned error can't find the container with id e900d62a9a3f85f3d491e1117062073e3281da3318761b1edac3b2a0fa642d31
Nov 01 05:08:53.322271 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:53.322216    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-xq2f6" podStartSLOduration=2.216911325 podStartE2EDuration="5.322198391s" totalImagesPullingTime="3.105287066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:08:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:08:53.317617816 +0000 UTC m=+350.207993793" watchObservedRunningTime="2026-11-01 05:08:53.322198391 +0000 UTC m=+350.212574369"
Nov 01 05:08:54.104481 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:54.103562    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-k2nkj" event={"ID":"cb788d11-cc64-4f1c-b1e0-0bed74754f03","Type":"ContainerStarted","Data":"e900d62a9a3f85f3d491e1117062073e3281da3318761b1edac3b2a0fa642d31"}
Nov 01 05:08:59.452878 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:59.452777    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6"]
Nov 01 05:08:59.454964 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:59.454948    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6"
Nov 01 05:08:59.507207 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:59.507134    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nrw9v\""
Nov 01 05:08:59.558984 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:59.558946    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6"]
Nov 01 05:08:59.562119 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:59.562093    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6dda8a88-9e43-4686-8b10-723c098b8462-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-9rwv6\" (UID: \"6dda8a88-9e43-4686-8b10-723c098b8462\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6"
Nov 01 05:08:59.562242 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:59.562138    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8dj9\" (UniqueName: \"kubernetes.io/projected/6dda8a88-9e43-4686-8b10-723c098b8462-kube-api-access-v8dj9\") pod \"kuadrant-operator-controller-manager-cf8469d7b-9rwv6\" (UID: \"6dda8a88-9e43-4686-8b10-723c098b8462\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6"
Nov 01 05:08:59.663561 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:59.663522    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6dda8a88-9e43-4686-8b10-723c098b8462-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-9rwv6\" (UID: \"6dda8a88-9e43-4686-8b10-723c098b8462\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6"
Nov 01 05:08:59.663735 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:59.663569    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8dj9\" (UniqueName: \"kubernetes.io/projected/6dda8a88-9e43-4686-8b10-723c098b8462-kube-api-access-v8dj9\") pod \"kuadrant-operator-controller-manager-cf8469d7b-9rwv6\" (UID: \"6dda8a88-9e43-4686-8b10-723c098b8462\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6"
Nov 01 05:08:59.663939 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:59.663918    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6dda8a88-9e43-4686-8b10-723c098b8462-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-9rwv6\" (UID: \"6dda8a88-9e43-4686-8b10-723c098b8462\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6"
Nov 01 05:08:59.724920 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:59.724851    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8dj9\" (UniqueName: \"kubernetes.io/projected/6dda8a88-9e43-4686-8b10-723c098b8462-kube-api-access-v8dj9\") pod \"kuadrant-operator-controller-manager-cf8469d7b-9rwv6\" (UID: \"6dda8a88-9e43-4686-8b10-723c098b8462\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6"
Nov 01 05:08:59.765708 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:08:59.765674    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6"
Nov 01 05:09:00.132571 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:00.132531    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-k2nkj" event={"ID":"cb788d11-cc64-4f1c-b1e0-0bed74754f03","Type":"ContainerStarted","Data":"57b95fa79b7683f82638981a4ee0b96b7cb6de0bfb919d61200448243da77721"}
Nov 01 05:09:00.132756 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:00.132602    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-k2nkj"
Nov 01 05:09:00.249448 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:00.249412    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6"]
Nov 01 05:09:00.276453 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:09:00.276413    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dda8a88_9e43_4686_8b10_723c098b8462.slice/crio-f59ee339a7c398a5313733da7756428eee532213909efd75862a13f20f6a383e WatchSource:0}: Error finding container f59ee339a7c398a5313733da7756428eee532213909efd75862a13f20f6a383e: Status 404 returned error can't find the container with id f59ee339a7c398a5313733da7756428eee532213909efd75862a13f20f6a383e
Nov 01 05:09:01.139564 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:01.139528    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6" event={"ID":"6dda8a88-9e43-4686-8b10-723c098b8462","Type":"ContainerStarted","Data":"f59ee339a7c398a5313733da7756428eee532213909efd75862a13f20f6a383e"}
Nov 01 05:09:03.237439 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:03.237377    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7b7bb4bc77-k2nkj" podStartSLOduration=5.274536886 podStartE2EDuration="11.23735582s" totalImagesPullingTime="5.962818934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:09:00.268426529 +0000 UTC m=+357.158802506" watchObservedRunningTime="2026-11-01 05:09:03.23735582 +0000 UTC m=+360.127731796"
Nov 01 05:09:03.239204 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:03.239179    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-wttv8"]
Nov 01 05:09:03.244014 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:03.243991    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-wttv8"
Nov 01 05:09:03.308231 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:03.307853    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvmxr\" (UniqueName: \"kubernetes.io/projected/af89e1f3-fb0d-44e2-b35f-748885de9474-kube-api-access-fvmxr\") pod \"limitador-operator-controller-manager-5778bb4449-wttv8\" (UID: \"af89e1f3-fb0d-44e2-b35f-748885de9474\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-wttv8"
Nov 01 05:09:03.346115 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:03.346075    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-5w7tr\""
Nov 01 05:09:03.349391 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:03.349363    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-wttv8"]
Nov 01 05:09:03.409082 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:03.409041    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvmxr\" (UniqueName: \"kubernetes.io/projected/af89e1f3-fb0d-44e2-b35f-748885de9474-kube-api-access-fvmxr\") pod \"limitador-operator-controller-manager-5778bb4449-wttv8\" (UID: \"af89e1f3-fb0d-44e2-b35f-748885de9474\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-wttv8"
Nov 01 05:09:03.470565 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:03.470535    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvmxr\" (UniqueName: \"kubernetes.io/projected/af89e1f3-fb0d-44e2-b35f-748885de9474-kube-api-access-fvmxr\") pod \"limitador-operator-controller-manager-5778bb4449-wttv8\" (UID: \"af89e1f3-fb0d-44e2-b35f-748885de9474\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-wttv8"
Nov 01 05:09:03.561302 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:03.561269    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-wttv8"
Nov 01 05:09:03.954108 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:03.954075    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-wttv8"]
Nov 01 05:09:03.955937 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:09:03.955914    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf89e1f3_fb0d_44e2_b35f_748885de9474.slice/crio-a5e48c2d60eb16c2d74ccc69647000db087e5e70069cfbe9272311040e96d95c WatchSource:0}: Error finding container a5e48c2d60eb16c2d74ccc69647000db087e5e70069cfbe9272311040e96d95c: Status 404 returned error can't find the container with id a5e48c2d60eb16c2d74ccc69647000db087e5e70069cfbe9272311040e96d95c
Nov 01 05:09:04.157690 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:04.157657    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-wttv8" event={"ID":"af89e1f3-fb0d-44e2-b35f-748885de9474","Type":"ContainerStarted","Data":"a5e48c2d60eb16c2d74ccc69647000db087e5e70069cfbe9272311040e96d95c"}
Nov 01 05:09:06.171439 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:06.171399    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6" event={"ID":"6dda8a88-9e43-4686-8b10-723c098b8462","Type":"ContainerStarted","Data":"10b957bf4e59e08ace628ad2ef397a36521fe45050c6191b593dab647c45efbb"}
Nov 01 05:09:06.171930 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:06.171523    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6"
Nov 01 05:09:06.399461 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:06.399392    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6" podStartSLOduration=2.051232504 podStartE2EDuration="7.399370694s" totalImagesPullingTime="5.34813819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:09:06.347854819 +0000 UTC m=+363.238230793" watchObservedRunningTime="2026-11-01 05:09:06.399370694 +0000 UTC m=+363.289746670"
Nov 01 05:09:07.177458 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:07.177424    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-wttv8" event={"ID":"af89e1f3-fb0d-44e2-b35f-748885de9474","Type":"ContainerStarted","Data":"bb2fe6b5b7b7c2750a4254159e796e325fbc32968ebcbd8396f53ee3671c2cd6"}
Nov 01 05:09:07.177983 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:07.177595    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-wttv8"
Nov 01 05:09:07.269048 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:07.269015    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-9t2hp"]
Nov 01 05:09:07.272510 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:07.272495    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-9t2hp"
Nov 01 05:09:07.294522 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:07.294476    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-wttv8" podStartSLOduration=1.756215674 podStartE2EDuration="4.294463436s" totalImagesPullingTime="2.538247762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:09:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:09:07.292742499 +0000 UTC m=+364.183118474" watchObservedRunningTime="2026-11-01 05:09:07.294463436 +0000 UTC m=+364.184839413"
Nov 01 05:09:07.324585 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:07.324556    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Nov 01 05:09:07.324726 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:07.324705    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-4n5k8\""
Nov 01 05:09:07.345287 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:07.345251    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7sdd\" (UniqueName: \"kubernetes.io/projected/194e1c98-db83-4d11-b279-b43783313267-kube-api-access-b7sdd\") pod \"dns-operator-controller-manager-8557cbdb84-9t2hp\" (UID: \"194e1c98-db83-4d11-b279-b43783313267\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-9t2hp"
Nov 01 05:09:07.377102 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:07.377069    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-9t2hp"]
Nov 01 05:09:07.445891 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:07.445808    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7sdd\" (UniqueName: \"kubernetes.io/projected/194e1c98-db83-4d11-b279-b43783313267-kube-api-access-b7sdd\") pod \"dns-operator-controller-manager-8557cbdb84-9t2hp\" (UID: \"194e1c98-db83-4d11-b279-b43783313267\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-9t2hp"
Nov 01 05:09:07.509318 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:07.509286    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7sdd\" (UniqueName: \"kubernetes.io/projected/194e1c98-db83-4d11-b279-b43783313267-kube-api-access-b7sdd\") pod \"dns-operator-controller-manager-8557cbdb84-9t2hp\" (UID: \"194e1c98-db83-4d11-b279-b43783313267\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-9t2hp"
Nov 01 05:09:07.585213 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:07.585181    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-9t2hp"
Nov 01 05:09:07.964513 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:07.964484    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-9t2hp"]
Nov 01 05:09:07.967097 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:09:07.967056    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod194e1c98_db83_4d11_b279_b43783313267.slice/crio-4e1d6388fb6439d555c5f6a8c7e7a786d37a0171c71d13b1b72880c3b781c39f WatchSource:0}: Error finding container 4e1d6388fb6439d555c5f6a8c7e7a786d37a0171c71d13b1b72880c3b781c39f: Status 404 returned error can't find the container with id 4e1d6388fb6439d555c5f6a8c7e7a786d37a0171c71d13b1b72880c3b781c39f
Nov 01 05:09:08.184323 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.184289    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-9t2hp" event={"ID":"194e1c98-db83-4d11-b279-b43783313267","Type":"ContainerStarted","Data":"4e1d6388fb6439d555c5f6a8c7e7a786d37a0171c71d13b1b72880c3b781c39f"}
Nov 01 05:09:08.506448 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.506415    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"]
Nov 01 05:09:08.511249 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.511230    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.556399 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.556344    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd8055d-05e5-4204-85ac-3ed22dc9b493-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.556569 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.556412    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd8055d-05e5-4204-85ac-3ed22dc9b493-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.556569 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.556506    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd8055d-05e5-4204-85ac-3ed22dc9b493-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.556569 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.556535    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dd8055d-05e5-4204-85ac-3ed22dc9b493-istio-token\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.556569 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.556561    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dd8055d-05e5-4204-85ac-3ed22dc9b493-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.556736 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.556677    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dd8055d-05e5-4204-85ac-3ed22dc9b493-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.556736 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.556717    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dd8055d-05e5-4204-85ac-3ed22dc9b493-istio-data\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.556844 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.556795    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8tl7\" (UniqueName: \"kubernetes.io/projected/6dd8055d-05e5-4204-85ac-3ed22dc9b493-kube-api-access-q8tl7\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.556844 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.556816    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dd8055d-05e5-4204-85ac-3ed22dc9b493-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.573584 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.573549    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"]
Nov 01 05:09:08.613488 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.613454    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-clbvn\""
Nov 01 05:09:08.658098 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.658068    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd8055d-05e5-4204-85ac-3ed22dc9b493-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.658098 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.658105    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd8055d-05e5-4204-85ac-3ed22dc9b493-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.658351 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.658124    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd8055d-05e5-4204-85ac-3ed22dc9b493-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.658351 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.658140    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dd8055d-05e5-4204-85ac-3ed22dc9b493-istio-token\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.658351 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.658255    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dd8055d-05e5-4204-85ac-3ed22dc9b493-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.658351 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.658308    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dd8055d-05e5-4204-85ac-3ed22dc9b493-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.658351 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.658352    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dd8055d-05e5-4204-85ac-3ed22dc9b493-istio-data\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.658618 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.658500    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8tl7\" (UniqueName: \"kubernetes.io/projected/6dd8055d-05e5-4204-85ac-3ed22dc9b493-kube-api-access-q8tl7\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.658618 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.658535    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd8055d-05e5-4204-85ac-3ed22dc9b493-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.658618 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.658554    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dd8055d-05e5-4204-85ac-3ed22dc9b493-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.658618 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.658607    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd8055d-05e5-4204-85ac-3ed22dc9b493-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.658778 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.658638    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd8055d-05e5-4204-85ac-3ed22dc9b493-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.658778 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.658661    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dd8055d-05e5-4204-85ac-3ed22dc9b493-istio-data\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.659069 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.659043    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dd8055d-05e5-4204-85ac-3ed22dc9b493-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.660868 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.660844    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dd8055d-05e5-4204-85ac-3ed22dc9b493-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.660968 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.660951    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dd8055d-05e5-4204-85ac-3ed22dc9b493-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.724412 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.724374    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dd8055d-05e5-4204-85ac-3ed22dc9b493-istio-token\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.724559 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.724543    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8tl7\" (UniqueName: \"kubernetes.io/projected/6dd8055d-05e5-4204-85ac-3ed22dc9b493-kube-api-access-q8tl7\") pod \"data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9\" (UID: \"6dd8055d-05e5-4204-85ac-3ed22dc9b493\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:08.824975 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:08.824940    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:09.302445 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:09.302419    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"]
Nov 01 05:09:09.303928 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:09:09.303892    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd8055d_05e5_4204_85ac_3ed22dc9b493.slice/crio-b00d56fcfc3dec147546ab1f8e7636057b5949e857f7dcb19e4b5283f7710c7c WatchSource:0}: Error finding container b00d56fcfc3dec147546ab1f8e7636057b5949e857f7dcb19e4b5283f7710c7c: Status 404 returned error can't find the container with id b00d56fcfc3dec147546ab1f8e7636057b5949e857f7dcb19e4b5283f7710c7c
Nov 01 05:09:09.306455 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:09.306421    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:09:09.306546 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:09.306490    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:09:09.306546 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:09.306542    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:09:10.196431 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:10.196393    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9" event={"ID":"6dd8055d-05e5-4204-85ac-3ed22dc9b493","Type":"ContainerStarted","Data":"61808207c5e74703c059077f935272f017968fe85f74dc71ef5d3f29d0aa95c0"}
Nov 01 05:09:10.196625 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:10.196448    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9" event={"ID":"6dd8055d-05e5-4204-85ac-3ed22dc9b493","Type":"ContainerStarted","Data":"b00d56fcfc3dec147546ab1f8e7636057b5949e857f7dcb19e4b5283f7710c7c"}
Nov 01 05:09:10.318936 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:10.318876    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9" podStartSLOduration=2.318861236 podStartE2EDuration="2.318861236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:09:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:09:10.315989923 +0000 UTC m=+367.206365899" watchObservedRunningTime="2026-11-01 05:09:10.318861236 +0000 UTC m=+367.209237212"
Nov 01 05:09:10.825770 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:10.825732    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:10.830901 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:10.830874    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:11.141933 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:11.141900    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-k2nkj"
Nov 01 05:09:11.208221 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:11.208183    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-9t2hp" event={"ID":"194e1c98-db83-4d11-b279-b43783313267","Type":"ContainerStarted","Data":"b379ccef82ab3a5b49b1d715c40eb21d5b8a8c50b1fe88f285a9f5993ae3ab4f"}
Nov 01 05:09:11.208420 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:11.208355    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-9t2hp"
Nov 01 05:09:11.208420 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:11.208388    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:11.209512 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:11.209491    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9"
Nov 01 05:09:11.495026 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:11.494914    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-9t2hp" podStartSLOduration=1.871691827 podStartE2EDuration="4.494899906s" totalImagesPullingTime="2.623208079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:09:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:09:11.492384436 +0000 UTC m=+368.382760412" watchObservedRunningTime="2026-11-01 05:09:11.494899906 +0000 UTC m=+368.385275881"
Nov 01 05:09:17.180022 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:17.179989    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6"
Nov 01 05:09:17.653870 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:17.653833    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9"]
Nov 01 05:09:17.659223 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:17.659203    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9"
Nov 01 05:09:17.722699 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:17.722662    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9"]
Nov 01 05:09:17.747151 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:17.747123    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm55f\" (UniqueName: \"kubernetes.io/projected/a763a5c5-de38-4a17-9cff-69cf629c1270-kube-api-access-nm55f\") pod \"kuadrant-operator-controller-manager-678d87766f-kfkl9\" (UID: \"a763a5c5-de38-4a17-9cff-69cf629c1270\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9"
Nov 01 05:09:17.747327 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:17.747182    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a763a5c5-de38-4a17-9cff-69cf629c1270-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-kfkl9\" (UID: \"a763a5c5-de38-4a17-9cff-69cf629c1270\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9"
Nov 01 05:09:17.848008 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:17.847970    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm55f\" (UniqueName: \"kubernetes.io/projected/a763a5c5-de38-4a17-9cff-69cf629c1270-kube-api-access-nm55f\") pod \"kuadrant-operator-controller-manager-678d87766f-kfkl9\" (UID: \"a763a5c5-de38-4a17-9cff-69cf629c1270\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9"
Nov 01 05:09:17.848187 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:17.848037    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a763a5c5-de38-4a17-9cff-69cf629c1270-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-kfkl9\" (UID: \"a763a5c5-de38-4a17-9cff-69cf629c1270\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9"
Nov 01 05:09:17.848433 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:17.848415    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a763a5c5-de38-4a17-9cff-69cf629c1270-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-kfkl9\" (UID: \"a763a5c5-de38-4a17-9cff-69cf629c1270\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9"
Nov 01 05:09:17.919190 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:17.919106    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm55f\" (UniqueName: \"kubernetes.io/projected/a763a5c5-de38-4a17-9cff-69cf629c1270-kube-api-access-nm55f\") pod \"kuadrant-operator-controller-manager-678d87766f-kfkl9\" (UID: \"a763a5c5-de38-4a17-9cff-69cf629c1270\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9"
Nov 01 05:09:17.970968 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:17.970930    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9"
Nov 01 05:09:18.187396 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:18.187316    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-wttv8"
Nov 01 05:09:18.452354 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:18.452327    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9"]
Nov 01 05:09:18.454214 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:09:18.454192    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda763a5c5_de38_4a17_9cff_69cf629c1270.slice/crio-8b4c11a896cea8547049c97b21eb67d465b88205a9722c1db86b049afc8fd7f0 WatchSource:0}: Error finding container 8b4c11a896cea8547049c97b21eb67d465b88205a9722c1db86b049afc8fd7f0: Status 404 returned error can't find the container with id 8b4c11a896cea8547049c97b21eb67d465b88205a9722c1db86b049afc8fd7f0
Nov 01 05:09:19.250772 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:19.250738    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9" event={"ID":"a763a5c5-de38-4a17-9cff-69cf629c1270","Type":"ContainerStarted","Data":"6295c8daf8f032ea42f58a62d9a342e7bd1297bf9d819a8499691ed189234e4f"}
Nov 01 05:09:19.251225 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:19.250815    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9" event={"ID":"a763a5c5-de38-4a17-9cff-69cf629c1270","Type":"ContainerStarted","Data":"8b4c11a896cea8547049c97b21eb67d465b88205a9722c1db86b049afc8fd7f0"}
Nov 01 05:09:19.251225 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:19.250935    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9"
Nov 01 05:09:19.468420 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:19.468368    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9" podStartSLOduration=2.468353612 podStartE2EDuration="2.468353612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:09:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:09:19.466945765 +0000 UTC m=+376.357321751" watchObservedRunningTime="2026-11-01 05:09:19.468353612 +0000 UTC m=+376.358729588"
Nov 01 05:09:22.216377 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:22.216345    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-9t2hp"
Nov 01 05:09:22.526552 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:09:22.526465    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:09:30.258373 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:30.258344    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9"
Nov 01 05:09:30.481547 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:30.481510    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6"]
Nov 01 05:09:30.481905 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:30.481861    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6" podUID="6dda8a88-9e43-4686-8b10-723c098b8462" containerName="manager" containerID="cri-o://10b957bf4e59e08ace628ad2ef397a36521fe45050c6191b593dab647c45efbb" gracePeriod=10
Nov 01 05:09:31.739018 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:31.738990    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6"
Nov 01 05:09:31.870653 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:31.870573    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6dda8a88-9e43-4686-8b10-723c098b8462-kube-api-access-v8dj9\" (UniqueName: \"kubernetes.io/projected/6dda8a88-9e43-4686-8b10-723c098b8462-kube-api-access-v8dj9\") pod \"6dda8a88-9e43-4686-8b10-723c098b8462\" (UID: \"6dda8a88-9e43-4686-8b10-723c098b8462\") "
Nov 01 05:09:31.870842 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:31.870728    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6dda8a88-9e43-4686-8b10-723c098b8462-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6dda8a88-9e43-4686-8b10-723c098b8462-extensions-socket-volume\") pod \"6dda8a88-9e43-4686-8b10-723c098b8462\" (UID: \"6dda8a88-9e43-4686-8b10-723c098b8462\") "
Nov 01 05:09:31.871236 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:31.871200    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dda8a88-9e43-4686-8b10-723c098b8462-extensions-socket-volume" pod "6dda8a88-9e43-4686-8b10-723c098b8462" (UID: "6dda8a88-9e43-4686-8b10-723c098b8462"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:09:31.873093 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:31.873064    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dda8a88-9e43-4686-8b10-723c098b8462-kube-api-access-v8dj9" pod "6dda8a88-9e43-4686-8b10-723c098b8462" (UID: "6dda8a88-9e43-4686-8b10-723c098b8462"). InnerVolumeSpecName "kube-api-access-v8dj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:09:31.972129 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:31.972097    4864 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6dda8a88-9e43-4686-8b10-723c098b8462-extensions-socket-volume\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:09:31.972129 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:31.972125    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8dj9\" (UniqueName: \"kubernetes.io/projected/6dda8a88-9e43-4686-8b10-723c098b8462-kube-api-access-v8dj9\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:09:32.324121 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:32.324083    4864 generic.go:356] "Generic (PLEG): container finished" podID="6dda8a88-9e43-4686-8b10-723c098b8462" containerID="10b957bf4e59e08ace628ad2ef397a36521fe45050c6191b593dab647c45efbb" exitCode=0
Nov 01 05:09:32.324334 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:32.324133    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6"
Nov 01 05:09:32.324334 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:32.324176    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6" event={"ID":"6dda8a88-9e43-4686-8b10-723c098b8462","Type":"ContainerDied","Data":"10b957bf4e59e08ace628ad2ef397a36521fe45050c6191b593dab647c45efbb"}
Nov 01 05:09:32.324334 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:32.324242    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6" event={"ID":"6dda8a88-9e43-4686-8b10-723c098b8462","Type":"ContainerDied","Data":"f59ee339a7c398a5313733da7756428eee532213909efd75862a13f20f6a383e"}
Nov 01 05:09:32.324334 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:32.324272    4864 scope.go:122] "RemoveContainer" containerID="10b957bf4e59e08ace628ad2ef397a36521fe45050c6191b593dab647c45efbb"
Nov 01 05:09:32.334241 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:32.334217    4864 scope.go:122] "RemoveContainer" containerID="10b957bf4e59e08ace628ad2ef397a36521fe45050c6191b593dab647c45efbb"
Nov 01 05:09:32.334578 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:09:32.334558    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10b957bf4e59e08ace628ad2ef397a36521fe45050c6191b593dab647c45efbb\": container with ID starting with 10b957bf4e59e08ace628ad2ef397a36521fe45050c6191b593dab647c45efbb not found: ID does not exist" containerID="10b957bf4e59e08ace628ad2ef397a36521fe45050c6191b593dab647c45efbb"
Nov 01 05:09:32.334633 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:32.334589    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10b957bf4e59e08ace628ad2ef397a36521fe45050c6191b593dab647c45efbb"} err="failed to get container status \"10b957bf4e59e08ace628ad2ef397a36521fe45050c6191b593dab647c45efbb\": rpc error: code = NotFound desc = could not find container \"10b957bf4e59e08ace628ad2ef397a36521fe45050c6191b593dab647c45efbb\": container with ID starting with 10b957bf4e59e08ace628ad2ef397a36521fe45050c6191b593dab647c45efbb not found: ID does not exist"
Nov 01 05:09:32.489048 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:32.489014    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6"]
Nov 01 05:09:32.539091 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:32.539054    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6"]
Nov 01 05:09:33.245886 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:33.245859    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6dda8a88-9e43-4686-8b10-723c098b8462" path="/var/lib/kubelet/pods/6dda8a88-9e43-4686-8b10-723c098b8462/volumes"
Nov 01 05:09:34.439921 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.439886    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv"]
Nov 01 05:09:34.440286 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.440244    4864 cpu_manager.go:414] "containerMap: removing container" podUID="6dda8a88-9e43-4686-8b10-723c098b8462" containerName="manager"
Nov 01 05:09:34.440286 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.440255    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6dda8a88-9e43-4686-8b10-723c098b8462" containerName="manager"
Nov 01 05:09:34.440371 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.440338    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="6dda8a88-9e43-4686-8b10-723c098b8462" containerName="manager"
Nov 01 05:09:34.446066 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.446045    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv"
Nov 01 05:09:34.498491 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.498459    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 01 05:09:34.547770 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.547737    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-m4grz\""
Nov 01 05:09:34.547952 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.547816    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 01 05:09:34.596673 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.596631    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/39569001-5722-4b9e-ac79-84952518f6ca-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-8qncv\" (UID: \"39569001-5722-4b9e-ac79-84952518f6ca\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv"
Nov 01 05:09:34.596860 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.596689    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39569001-5722-4b9e-ac79-84952518f6ca-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-8qncv\" (UID: \"39569001-5722-4b9e-ac79-84952518f6ca\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv"
Nov 01 05:09:34.596860 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.596842    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nnlk\" (UniqueName: \"kubernetes.io/projected/39569001-5722-4b9e-ac79-84952518f6ca-kube-api-access-9nnlk\") pod \"kserve-module-controller-manager-7566cf5fc8-8qncv\" (UID: \"39569001-5722-4b9e-ac79-84952518f6ca\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv"
Nov 01 05:09:34.596966 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.596885    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39569001-5722-4b9e-ac79-84952518f6ca-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-8qncv\" (UID: \"39569001-5722-4b9e-ac79-84952518f6ca\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv"
Nov 01 05:09:34.647279 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.647244    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Nov 01 05:09:34.649649 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.649624    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv"]
Nov 01 05:09:34.698363 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.698272    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nnlk\" (UniqueName: \"kubernetes.io/projected/39569001-5722-4b9e-ac79-84952518f6ca-kube-api-access-9nnlk\") pod \"kserve-module-controller-manager-7566cf5fc8-8qncv\" (UID: \"39569001-5722-4b9e-ac79-84952518f6ca\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv"
Nov 01 05:09:34.698363 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.698323    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39569001-5722-4b9e-ac79-84952518f6ca-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-8qncv\" (UID: \"39569001-5722-4b9e-ac79-84952518f6ca\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv"
Nov 01 05:09:34.698588 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.698561    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/39569001-5722-4b9e-ac79-84952518f6ca-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-8qncv\" (UID: \"39569001-5722-4b9e-ac79-84952518f6ca\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv"
Nov 01 05:09:34.698644 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.698610    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39569001-5722-4b9e-ac79-84952518f6ca-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-8qncv\" (UID: \"39569001-5722-4b9e-ac79-84952518f6ca\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv"
Nov 01 05:09:34.698848 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.698829    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39569001-5722-4b9e-ac79-84952518f6ca-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-8qncv\" (UID: \"39569001-5722-4b9e-ac79-84952518f6ca\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv"
Nov 01 05:09:34.698917 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.698900    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/39569001-5722-4b9e-ac79-84952518f6ca-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-8qncv\" (UID: \"39569001-5722-4b9e-ac79-84952518f6ca\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv"
Nov 01 05:09:34.699964 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.699948    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39569001-5722-4b9e-ac79-84952518f6ca-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-8qncv\" (UID: \"39569001-5722-4b9e-ac79-84952518f6ca\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv"
Nov 01 05:09:34.759971 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:34.759941    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nnlk\" (UniqueName: \"kubernetes.io/projected/39569001-5722-4b9e-ac79-84952518f6ca-kube-api-access-9nnlk\") pod \"kserve-module-controller-manager-7566cf5fc8-8qncv\" (UID: \"39569001-5722-4b9e-ac79-84952518f6ca\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv"
Nov 01 05:09:35.059117 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:35.059087    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv"
Nov 01 05:09:35.483839 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:35.483765    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv"]
Nov 01 05:09:35.486125 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:09:35.486096    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39569001_5722_4b9e_ac79_84952518f6ca.slice/crio-db8daccd36c06a9dac65e1b8b6518abbd219dbf03872642235105eebab23d623 WatchSource:0}: Error finding container db8daccd36c06a9dac65e1b8b6518abbd219dbf03872642235105eebab23d623: Status 404 returned error can't find the container with id db8daccd36c06a9dac65e1b8b6518abbd219dbf03872642235105eebab23d623
Nov 01 05:09:36.355017 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:36.354973    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv" event={"ID":"39569001-5722-4b9e-ac79-84952518f6ca","Type":"ContainerStarted","Data":"db8daccd36c06a9dac65e1b8b6518abbd219dbf03872642235105eebab23d623"}
Nov 01 05:09:39.371609 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:39.371576    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv" event={"ID":"39569001-5722-4b9e-ac79-84952518f6ca","Type":"ContainerStarted","Data":"e50800fadbe164c88097366f82c8c4d642d47f622d0fa6c51e713852473e17f8"}
Nov 01 05:09:39.372092 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:39.371706    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv"
Nov 01 05:09:39.539941 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:39.539886    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv" podStartSLOduration=2.32331073 podStartE2EDuration="5.539868719s" totalImagesPullingTime="3.216557989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:09:39.53852815 +0000 UTC m=+396.428904127" watchObservedRunningTime="2026-11-01 05:09:39.539868719 +0000 UTC m=+396.430244695"
Nov 01 05:09:52.523943 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:09:52.523863    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:09:52.843179 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:52.843140    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-crc2n"]
Nov 01 05:09:52.847451 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:52.847427    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-crc2n"
Nov 01 05:09:52.858195 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:52.858169    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/60ec3670-705a-45e2-9dbe-e71edd5fb2fc-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-crc2n\" (UID: \"60ec3670-705a-45e2-9dbe-e71edd5fb2fc\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-crc2n"
Nov 01 05:09:52.858329 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:52.858213    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/60ec3670-705a-45e2-9dbe-e71edd5fb2fc-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-crc2n\" (UID: \"60ec3670-705a-45e2-9dbe-e71edd5fb2fc\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-crc2n"
Nov 01 05:09:52.858329 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:52.858288    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76vl2\" (UniqueName: \"kubernetes.io/projected/60ec3670-705a-45e2-9dbe-e71edd5fb2fc-kube-api-access-76vl2\") pod \"kuadrant-console-plugin-66c4fb575d-crc2n\" (UID: \"60ec3670-705a-45e2-9dbe-e71edd5fb2fc\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-crc2n"
Nov 01 05:09:52.902206 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:52.902163    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Nov 01 05:09:52.904656 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:52.904630    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-crc2n"]
Nov 01 05:09:52.951842 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:52.951778    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ghcgj\""
Nov 01 05:09:52.952020 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:52.951860    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Nov 01 05:09:52.959261 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:52.959228    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/60ec3670-705a-45e2-9dbe-e71edd5fb2fc-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-crc2n\" (UID: \"60ec3670-705a-45e2-9dbe-e71edd5fb2fc\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-crc2n"
Nov 01 05:09:52.959355 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:52.959279    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76vl2\" (UniqueName: \"kubernetes.io/projected/60ec3670-705a-45e2-9dbe-e71edd5fb2fc-kube-api-access-76vl2\") pod \"kuadrant-console-plugin-66c4fb575d-crc2n\" (UID: \"60ec3670-705a-45e2-9dbe-e71edd5fb2fc\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-crc2n"
Nov 01 05:09:52.959412 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:52.959402    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/60ec3670-705a-45e2-9dbe-e71edd5fb2fc-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-crc2n\" (UID: \"60ec3670-705a-45e2-9dbe-e71edd5fb2fc\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-crc2n"
Nov 01 05:09:52.960409 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:52.960372    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/60ec3670-705a-45e2-9dbe-e71edd5fb2fc-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-crc2n\" (UID: \"60ec3670-705a-45e2-9dbe-e71edd5fb2fc\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-crc2n"
Nov 01 05:09:52.962928 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:52.962900    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/60ec3670-705a-45e2-9dbe-e71edd5fb2fc-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-crc2n\" (UID: \"60ec3670-705a-45e2-9dbe-e71edd5fb2fc\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-crc2n"
Nov 01 05:09:53.020490 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:53.020455    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76vl2\" (UniqueName: \"kubernetes.io/projected/60ec3670-705a-45e2-9dbe-e71edd5fb2fc-kube-api-access-76vl2\") pod \"kuadrant-console-plugin-66c4fb575d-crc2n\" (UID: \"60ec3670-705a-45e2-9dbe-e71edd5fb2fc\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-crc2n"
Nov 01 05:09:53.159576 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:53.159495    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-crc2n"
Nov 01 05:09:53.539642 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:53.539557    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-crc2n"]
Nov 01 05:09:53.546505 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:09:53.546479    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60ec3670_705a_45e2_9dbe_e71edd5fb2fc.slice/crio-1e44231e05ddd09c362582b65a6f0f149b1e84ec8c615a0c62588daef9f32b6d WatchSource:0}: Error finding container 1e44231e05ddd09c362582b65a6f0f149b1e84ec8c615a0c62588daef9f32b6d: Status 404 returned error can't find the container with id 1e44231e05ddd09c362582b65a6f0f149b1e84ec8c615a0c62588daef9f32b6d
Nov 01 05:09:54.451026 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:54.450987    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-crc2n" event={"ID":"60ec3670-705a-45e2-9dbe-e71edd5fb2fc","Type":"ContainerStarted","Data":"1e44231e05ddd09c362582b65a6f0f149b1e84ec8c615a0c62588daef9f32b6d"}
Nov 01 05:09:59.481730 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:59.481637    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-crc2n" event={"ID":"60ec3670-705a-45e2-9dbe-e71edd5fb2fc","Type":"ContainerStarted","Data":"6af338901c118a4b855514113602a0659d111df6a1d2caea41d97c631de73e57"}
Nov 01 05:09:59.597653 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:09:59.597594    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-crc2n" podStartSLOduration=1.9513236219999999 podStartE2EDuration="7.597577631s" totalImagesPullingTime="5.646254009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:09:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:09:59.596607945 +0000 UTC m=+416.486983926" watchObservedRunningTime="2026-11-01 05:09:59.597577631 +0000 UTC m=+416.487953607"
Nov 01 05:10:00.378927 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:00.378896    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv"
Nov 01 05:10:15.643705 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:15.643665    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f8cfc466-hdgmh"]
Nov 01 05:10:15.647533 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:15.647507    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f8cfc466-hdgmh"
Nov 01 05:10:15.706165 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:15.706118    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 01 05:10:15.706165 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:15.706146    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Nov 01 05:10:15.755332 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:15.755296    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Nov 01 05:10:15.755332 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:15.755296    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-v68zh\""
Nov 01 05:10:15.757641 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:15.757616    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f8cfc466-hdgmh"]
Nov 01 05:10:15.770004 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:15.769966    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5548g\" (UniqueName: \"kubernetes.io/projected/6a1dc6cc-81dc-4e67-ac73-da37c8e32704-kube-api-access-5548g\") pod \"maas-controller-6f8cfc466-hdgmh\" (UID: \"6a1dc6cc-81dc-4e67-ac73-da37c8e32704\") " pod="opendatahub/maas-controller-6f8cfc466-hdgmh"
Nov 01 05:10:15.770221 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:15.770056    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a1dc6cc-81dc-4e67-ac73-da37c8e32704-webhook-cert\") pod \"maas-controller-6f8cfc466-hdgmh\" (UID: \"6a1dc6cc-81dc-4e67-ac73-da37c8e32704\") " pod="opendatahub/maas-controller-6f8cfc466-hdgmh"
Nov 01 05:10:15.770221 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:15.770187    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a1dc6cc-81dc-4e67-ac73-da37c8e32704-metrics-certs\") pod \"maas-controller-6f8cfc466-hdgmh\" (UID: \"6a1dc6cc-81dc-4e67-ac73-da37c8e32704\") " pod="opendatahub/maas-controller-6f8cfc466-hdgmh"
Nov 01 05:10:15.871490 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:15.871447    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5548g\" (UniqueName: \"kubernetes.io/projected/6a1dc6cc-81dc-4e67-ac73-da37c8e32704-kube-api-access-5548g\") pod \"maas-controller-6f8cfc466-hdgmh\" (UID: \"6a1dc6cc-81dc-4e67-ac73-da37c8e32704\") " pod="opendatahub/maas-controller-6f8cfc466-hdgmh"
Nov 01 05:10:15.871711 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:15.871496    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a1dc6cc-81dc-4e67-ac73-da37c8e32704-webhook-cert\") pod \"maas-controller-6f8cfc466-hdgmh\" (UID: \"6a1dc6cc-81dc-4e67-ac73-da37c8e32704\") " pod="opendatahub/maas-controller-6f8cfc466-hdgmh"
Nov 01 05:10:15.871711 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:15.871569    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a1dc6cc-81dc-4e67-ac73-da37c8e32704-metrics-certs\") pod \"maas-controller-6f8cfc466-hdgmh\" (UID: \"6a1dc6cc-81dc-4e67-ac73-da37c8e32704\") " pod="opendatahub/maas-controller-6f8cfc466-hdgmh"
Nov 01 05:10:15.874559 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:15.874529    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a1dc6cc-81dc-4e67-ac73-da37c8e32704-webhook-cert\") pod \"maas-controller-6f8cfc466-hdgmh\" (UID: \"6a1dc6cc-81dc-4e67-ac73-da37c8e32704\") " pod="opendatahub/maas-controller-6f8cfc466-hdgmh"
Nov 01 05:10:15.874702 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:15.874529    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a1dc6cc-81dc-4e67-ac73-da37c8e32704-metrics-certs\") pod \"maas-controller-6f8cfc466-hdgmh\" (UID: \"6a1dc6cc-81dc-4e67-ac73-da37c8e32704\") " pod="opendatahub/maas-controller-6f8cfc466-hdgmh"
Nov 01 05:10:15.932648 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:15.932573    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5548g\" (UniqueName: \"kubernetes.io/projected/6a1dc6cc-81dc-4e67-ac73-da37c8e32704-kube-api-access-5548g\") pod \"maas-controller-6f8cfc466-hdgmh\" (UID: \"6a1dc6cc-81dc-4e67-ac73-da37c8e32704\") " pod="opendatahub/maas-controller-6f8cfc466-hdgmh"
Nov 01 05:10:15.960493 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:15.960441    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f8cfc466-hdgmh"
Nov 01 05:10:16.227499 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:16.227427    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74b5bc6fff-vqr9s"]
Nov 01 05:10:16.231937 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:16.231916    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74b5bc6fff-vqr9s"
Nov 01 05:10:16.291471 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:16.291442    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74b5bc6fff-vqr9s"]
Nov 01 05:10:16.353083 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:16.352924    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f8cfc466-hdgmh"]
Nov 01 05:10:16.355311 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:10:16.355278    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a1dc6cc_81dc_4e67_ac73_da37c8e32704.slice/crio-4b9883a261f54ef223a6260dcd70aacdc8cf00af22e51fb0fb15bd3953f9ae5a WatchSource:0}: Error finding container 4b9883a261f54ef223a6260dcd70aacdc8cf00af22e51fb0fb15bd3953f9ae5a: Status 404 returned error can't find the container with id 4b9883a261f54ef223a6260dcd70aacdc8cf00af22e51fb0fb15bd3953f9ae5a
Nov 01 05:10:16.376256 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:16.376215    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/90275ed4-c333-4c1f-8583-853bf05c6165-webhook-cert\") pod \"maas-controller-74b5bc6fff-vqr9s\" (UID: \"90275ed4-c333-4c1f-8583-853bf05c6165\") " pod="opendatahub/maas-controller-74b5bc6fff-vqr9s"
Nov 01 05:10:16.376441 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:16.376266    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90275ed4-c333-4c1f-8583-853bf05c6165-metrics-certs\") pod \"maas-controller-74b5bc6fff-vqr9s\" (UID: \"90275ed4-c333-4c1f-8583-853bf05c6165\") " pod="opendatahub/maas-controller-74b5bc6fff-vqr9s"
Nov 01 05:10:16.376441 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:16.376324    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lr9jz\" (UniqueName: \"kubernetes.io/projected/90275ed4-c333-4c1f-8583-853bf05c6165-kube-api-access-lr9jz\") pod \"maas-controller-74b5bc6fff-vqr9s\" (UID: \"90275ed4-c333-4c1f-8583-853bf05c6165\") " pod="opendatahub/maas-controller-74b5bc6fff-vqr9s"
Nov 01 05:10:16.477366 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:16.477321    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/90275ed4-c333-4c1f-8583-853bf05c6165-webhook-cert\") pod \"maas-controller-74b5bc6fff-vqr9s\" (UID: \"90275ed4-c333-4c1f-8583-853bf05c6165\") " pod="opendatahub/maas-controller-74b5bc6fff-vqr9s"
Nov 01 05:10:16.477366 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:16.477368    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90275ed4-c333-4c1f-8583-853bf05c6165-metrics-certs\") pod \"maas-controller-74b5bc6fff-vqr9s\" (UID: \"90275ed4-c333-4c1f-8583-853bf05c6165\") " pod="opendatahub/maas-controller-74b5bc6fff-vqr9s"
Nov 01 05:10:16.477679 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:16.477396    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lr9jz\" (UniqueName: \"kubernetes.io/projected/90275ed4-c333-4c1f-8583-853bf05c6165-kube-api-access-lr9jz\") pod \"maas-controller-74b5bc6fff-vqr9s\" (UID: \"90275ed4-c333-4c1f-8583-853bf05c6165\") " pod="opendatahub/maas-controller-74b5bc6fff-vqr9s"
Nov 01 05:10:16.480278 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:16.480252    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90275ed4-c333-4c1f-8583-853bf05c6165-metrics-certs\") pod \"maas-controller-74b5bc6fff-vqr9s\" (UID: \"90275ed4-c333-4c1f-8583-853bf05c6165\") " pod="opendatahub/maas-controller-74b5bc6fff-vqr9s"
Nov 01 05:10:16.480394 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:16.480284    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/90275ed4-c333-4c1f-8583-853bf05c6165-webhook-cert\") pod \"maas-controller-74b5bc6fff-vqr9s\" (UID: \"90275ed4-c333-4c1f-8583-853bf05c6165\") " pod="opendatahub/maas-controller-74b5bc6fff-vqr9s"
Nov 01 05:10:16.538648 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:16.538614    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr9jz\" (UniqueName: \"kubernetes.io/projected/90275ed4-c333-4c1f-8583-853bf05c6165-kube-api-access-lr9jz\") pod \"maas-controller-74b5bc6fff-vqr9s\" (UID: \"90275ed4-c333-4c1f-8583-853bf05c6165\") " pod="opendatahub/maas-controller-74b5bc6fff-vqr9s"
Nov 01 05:10:16.544437 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:16.544415    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74b5bc6fff-vqr9s"
Nov 01 05:10:16.574368 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:16.574318    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f8cfc466-hdgmh" event={"ID":"6a1dc6cc-81dc-4e67-ac73-da37c8e32704","Type":"ContainerStarted","Data":"4b9883a261f54ef223a6260dcd70aacdc8cf00af22e51fb0fb15bd3953f9ae5a"}
Nov 01 05:10:16.919653 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:16.919629    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74b5bc6fff-vqr9s"]
Nov 01 05:10:16.921349 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:10:16.921318    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90275ed4_c333_4c1f_8583_853bf05c6165.slice/crio-60b482619fc3fa7ef12f9bf9a767864e860906f949c4d65d841e7e61bb765570 WatchSource:0}: Error finding container 60b482619fc3fa7ef12f9bf9a767864e860906f949c4d65d841e7e61bb765570: Status 404 returned error can't find the container with id 60b482619fc3fa7ef12f9bf9a767864e860906f949c4d65d841e7e61bb765570
Nov 01 05:10:17.581708 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:17.581667    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74b5bc6fff-vqr9s" event={"ID":"90275ed4-c333-4c1f-8583-853bf05c6165","Type":"ContainerStarted","Data":"60b482619fc3fa7ef12f9bf9a767864e860906f949c4d65d841e7e61bb765570"}
Nov 01 05:10:22.529984 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:10:22.529942    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:10:23.623418 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:23.623375    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f8cfc466-hdgmh" event={"ID":"6a1dc6cc-81dc-4e67-ac73-da37c8e32704","Type":"ContainerStarted","Data":"2979ad66c27e791981426b009088bb47690bac20ae3cb32df8d9ca1945f7c2ff"}
Nov 01 05:10:23.623919 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:23.623510    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f8cfc466-hdgmh"
Nov 01 05:10:23.625001 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:23.624971    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74b5bc6fff-vqr9s" event={"ID":"90275ed4-c333-4c1f-8583-853bf05c6165","Type":"ContainerStarted","Data":"672b80968be43108634f54480741ede4c8d22996ddfe8ab778294a22f1928ce1"}
Nov 01 05:10:23.625125 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:23.625039    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74b5bc6fff-vqr9s"
Nov 01 05:10:23.740835 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:23.740749    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-6f8cfc466-hdgmh" podStartSLOduration=2.197237957 podStartE2EDuration="8.740736007s" totalImagesPullingTime="6.54349805s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:10:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:10:23.739999121 +0000 UTC m=+440.630375096" watchObservedRunningTime="2026-11-01 05:10:23.740736007 +0000 UTC m=+440.631111982"
Nov 01 05:10:23.883675 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:23.883570    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-74b5bc6fff-vqr9s" podStartSLOduration=1.89487582 podStartE2EDuration="7.883555872s" totalImagesPullingTime="5.988680052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:10:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:10:23.882279194 +0000 UTC m=+440.772655170" watchObservedRunningTime="2026-11-01 05:10:23.883555872 +0000 UTC m=+440.773931847"
Nov 01 05:10:26.235358 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:26.235318    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d"]
Nov 01 05:10:26.239077 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:26.239051    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d"
Nov 01 05:10:26.290861 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:26.290830    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Nov 01 05:10:26.291061 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:26.290918    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Nov 01 05:10:26.291125 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:26.291084    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Nov 01 05:10:26.340489 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:26.340455    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-phhxn\""
Nov 01 05:10:26.343187 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:26.343152    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d"]
Nov 01 05:10:26.378247 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:26.378212    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/c3884e26-630f-4983-abc1-6dc5e9cb922a-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cdt6d\" (UID: \"c3884e26-630f-4983-abc1-6dc5e9cb922a\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d"
Nov 01 05:10:26.378433 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:26.378255    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3884e26-630f-4983-abc1-6dc5e9cb922a-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cdt6d\" (UID: \"c3884e26-630f-4983-abc1-6dc5e9cb922a\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d"
Nov 01 05:10:26.378433 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:26.378297    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqbd2\" (UniqueName: \"kubernetes.io/projected/c3884e26-630f-4983-abc1-6dc5e9cb922a-kube-api-access-hqbd2\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cdt6d\" (UID: \"c3884e26-630f-4983-abc1-6dc5e9cb922a\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d"
Nov 01 05:10:26.378537 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:26.378442    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c3884e26-630f-4983-abc1-6dc5e9cb922a-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cdt6d\" (UID: \"c3884e26-630f-4983-abc1-6dc5e9cb922a\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d"
Nov 01 05:10:26.479833 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:26.479801    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/c3884e26-630f-4983-abc1-6dc5e9cb922a-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cdt6d\" (UID: \"c3884e26-630f-4983-abc1-6dc5e9cb922a\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d"
Nov 01 05:10:26.479833 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:26.479834    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3884e26-630f-4983-abc1-6dc5e9cb922a-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cdt6d\" (UID: \"c3884e26-630f-4983-abc1-6dc5e9cb922a\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d"
Nov 01 05:10:26.480096 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:26.479863    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqbd2\" (UniqueName: \"kubernetes.io/projected/c3884e26-630f-4983-abc1-6dc5e9cb922a-kube-api-access-hqbd2\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cdt6d\" (UID: \"c3884e26-630f-4983-abc1-6dc5e9cb922a\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d"
Nov 01 05:10:26.480096 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:26.479891    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c3884e26-630f-4983-abc1-6dc5e9cb922a-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cdt6d\" (UID: \"c3884e26-630f-4983-abc1-6dc5e9cb922a\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d"
Nov 01 05:10:26.480096 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:10:26.479917    4864 configmap.go:193] Couldn't get configMap opendatahub/usage-tenancy-proxy-source: configmap "usage-tenancy-proxy-source" not found
Nov 01 05:10:26.480096 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:10:26.480000    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3884e26-630f-4983-abc1-6dc5e9cb922a-proxy-source podName:c3884e26-630f-4983-abc1-6dc5e9cb922a nodeName:}" failed. No retries permitted until 2026-11-01 05:10:26.979979502 +0000 UTC m=+443.870355470 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-source" (UniqueName: "kubernetes.io/configmap/c3884e26-630f-4983-abc1-6dc5e9cb922a-proxy-source") pod "usage-logs-tenancy-proxy-9c5d49bcd-cdt6d" (UID: "c3884e26-630f-4983-abc1-6dc5e9cb922a") : configmap "usage-tenancy-proxy-source" not found
Nov 01 05:10:26.480539 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:26.480519    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3884e26-630f-4983-abc1-6dc5e9cb922a-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cdt6d\" (UID: \"c3884e26-630f-4983-abc1-6dc5e9cb922a\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d"
Nov 01 05:10:26.482422 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:26.482405    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c3884e26-630f-4983-abc1-6dc5e9cb922a-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cdt6d\" (UID: \"c3884e26-630f-4983-abc1-6dc5e9cb922a\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d"
Nov 01 05:10:26.543414 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:26.543339    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqbd2\" (UniqueName: \"kubernetes.io/projected/c3884e26-630f-4983-abc1-6dc5e9cb922a-kube-api-access-hqbd2\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cdt6d\" (UID: \"c3884e26-630f-4983-abc1-6dc5e9cb922a\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d"
Nov 01 05:10:26.984642 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:26.984600    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/c3884e26-630f-4983-abc1-6dc5e9cb922a-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cdt6d\" (UID: \"c3884e26-630f-4983-abc1-6dc5e9cb922a\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d"
Nov 01 05:10:26.985428 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:26.985408    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/c3884e26-630f-4983-abc1-6dc5e9cb922a-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-cdt6d\" (UID: \"c3884e26-630f-4983-abc1-6dc5e9cb922a\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d"
Nov 01 05:10:27.152034 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:27.151992    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d"
Nov 01 05:10:27.550948 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:27.550917    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d"]
Nov 01 05:10:27.553016 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:10:27.552977    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3884e26_630f_4983_abc1_6dc5e9cb922a.slice/crio-d2c37643ce656de41e01c2d738a4c5e1e91851deff80a5c32368a811d9287a3b WatchSource:0}: Error finding container d2c37643ce656de41e01c2d738a4c5e1e91851deff80a5c32368a811d9287a3b: Status 404 returned error can't find the container with id d2c37643ce656de41e01c2d738a4c5e1e91851deff80a5c32368a811d9287a3b
Nov 01 05:10:27.647768 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:27.647728    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d" event={"ID":"c3884e26-630f-4983-abc1-6dc5e9cb922a","Type":"ContainerStarted","Data":"d2c37643ce656de41e01c2d738a4c5e1e91851deff80a5c32368a811d9287a3b"}
Nov 01 05:10:34.633179 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:34.633143    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74b5bc6fff-vqr9s"
Nov 01 05:10:34.633694 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:34.633236    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f8cfc466-hdgmh"
Nov 01 05:10:34.862478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:34.861801    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f8cfc466-hdgmh"]
Nov 01 05:10:34.862478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:34.862040    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-6f8cfc466-hdgmh" podUID="6a1dc6cc-81dc-4e67-ac73-da37c8e32704" containerName="manager" containerID="cri-o://2979ad66c27e791981426b009088bb47690bac20ae3cb32df8d9ca1945f7c2ff" gracePeriod=30
Nov 01 05:10:35.632447 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:35.632418    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f8cfc466-hdgmh"
Nov 01 05:10:35.682697 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:35.682647    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a1dc6cc-81dc-4e67-ac73-da37c8e32704-kube-api-access-5548g\" (UniqueName: \"kubernetes.io/projected/6a1dc6cc-81dc-4e67-ac73-da37c8e32704-kube-api-access-5548g\") pod \"6a1dc6cc-81dc-4e67-ac73-da37c8e32704\" (UID: \"6a1dc6cc-81dc-4e67-ac73-da37c8e32704\") "
Nov 01 05:10:35.683205 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:35.682709    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6a1dc6cc-81dc-4e67-ac73-da37c8e32704-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a1dc6cc-81dc-4e67-ac73-da37c8e32704-metrics-certs\") pod \"6a1dc6cc-81dc-4e67-ac73-da37c8e32704\" (UID: \"6a1dc6cc-81dc-4e67-ac73-da37c8e32704\") "
Nov 01 05:10:35.683205 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:35.682818    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6a1dc6cc-81dc-4e67-ac73-da37c8e32704-webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a1dc6cc-81dc-4e67-ac73-da37c8e32704-webhook-cert\") pod \"6a1dc6cc-81dc-4e67-ac73-da37c8e32704\" (UID: \"6a1dc6cc-81dc-4e67-ac73-da37c8e32704\") "
Nov 01 05:10:35.685375 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:35.685344    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a1dc6cc-81dc-4e67-ac73-da37c8e32704-webhook-cert" pod "6a1dc6cc-81dc-4e67-ac73-da37c8e32704" (UID: "6a1dc6cc-81dc-4e67-ac73-da37c8e32704"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:10:35.685522 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:35.685487    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a1dc6cc-81dc-4e67-ac73-da37c8e32704-metrics-certs" pod "6a1dc6cc-81dc-4e67-ac73-da37c8e32704" (UID: "6a1dc6cc-81dc-4e67-ac73-da37c8e32704"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:10:35.685522 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:35.685494    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a1dc6cc-81dc-4e67-ac73-da37c8e32704-kube-api-access-5548g" pod "6a1dc6cc-81dc-4e67-ac73-da37c8e32704" (UID: "6a1dc6cc-81dc-4e67-ac73-da37c8e32704"). InnerVolumeSpecName "kube-api-access-5548g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:10:35.700666 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:35.700640    4864 generic.go:356] "Generic (PLEG): container finished" podID="6a1dc6cc-81dc-4e67-ac73-da37c8e32704" containerID="2979ad66c27e791981426b009088bb47690bac20ae3cb32df8d9ca1945f7c2ff" exitCode=0
Nov 01 05:10:35.700809 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:35.700684    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f8cfc466-hdgmh"
Nov 01 05:10:35.700809 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:35.700722    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f8cfc466-hdgmh" event={"ID":"6a1dc6cc-81dc-4e67-ac73-da37c8e32704","Type":"ContainerDied","Data":"2979ad66c27e791981426b009088bb47690bac20ae3cb32df8d9ca1945f7c2ff"}
Nov 01 05:10:35.700809 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:35.700803    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f8cfc466-hdgmh" event={"ID":"6a1dc6cc-81dc-4e67-ac73-da37c8e32704","Type":"ContainerDied","Data":"4b9883a261f54ef223a6260dcd70aacdc8cf00af22e51fb0fb15bd3953f9ae5a"}
Nov 01 05:10:35.700923 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:35.700836    4864 scope.go:122] "RemoveContainer" containerID="2979ad66c27e791981426b009088bb47690bac20ae3cb32df8d9ca1945f7c2ff"
Nov 01 05:10:35.711358 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:35.711341    4864 scope.go:122] "RemoveContainer" containerID="2979ad66c27e791981426b009088bb47690bac20ae3cb32df8d9ca1945f7c2ff"
Nov 01 05:10:35.711708 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:10:35.711690    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2979ad66c27e791981426b009088bb47690bac20ae3cb32df8d9ca1945f7c2ff\": container with ID starting with 2979ad66c27e791981426b009088bb47690bac20ae3cb32df8d9ca1945f7c2ff not found: ID does not exist" containerID="2979ad66c27e791981426b009088bb47690bac20ae3cb32df8d9ca1945f7c2ff"
Nov 01 05:10:35.711768 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:35.711717    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2979ad66c27e791981426b009088bb47690bac20ae3cb32df8d9ca1945f7c2ff"} err="failed to get container status \"2979ad66c27e791981426b009088bb47690bac20ae3cb32df8d9ca1945f7c2ff\": rpc error: code = NotFound desc = could not find container \"2979ad66c27e791981426b009088bb47690bac20ae3cb32df8d9ca1945f7c2ff\": container with ID starting with 2979ad66c27e791981426b009088bb47690bac20ae3cb32df8d9ca1945f7c2ff not found: ID does not exist"
Nov 01 05:10:35.783886 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:35.783802    4864 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a1dc6cc-81dc-4e67-ac73-da37c8e32704-webhook-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:10:35.783886 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:35.783841    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5548g\" (UniqueName: \"kubernetes.io/projected/6a1dc6cc-81dc-4e67-ac73-da37c8e32704-kube-api-access-5548g\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:10:35.783886 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:35.783857    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a1dc6cc-81dc-4e67-ac73-da37c8e32704-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:10:35.863740 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:35.863702    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f8cfc466-hdgmh"]
Nov 01 05:10:35.867616 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:35.867588    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6f8cfc466-hdgmh"]
Nov 01 05:10:37.247223 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:37.247187    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a1dc6cc-81dc-4e67-ac73-da37c8e32704" path="/var/lib/kubelet/pods/6a1dc6cc-81dc-4e67-ac73-da37c8e32704/volumes"
Nov 01 05:10:40.734196 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:40.734160    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d" event={"ID":"c3884e26-630f-4983-abc1-6dc5e9cb922a","Type":"ContainerStarted","Data":"60aa9e435477b962d7f5d0f925ee157d5104fd637c68e240f5774c31eb00c9b9"}
Nov 01 05:10:40.734584 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:40.734263    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d"
Nov 01 05:10:40.852320 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:40.852220    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d" podStartSLOduration=1.813349691 podStartE2EDuration="14.852206968s" totalImagesPullingTime="13.038857277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:10:40.850548867 +0000 UTC m=+457.740924843" watchObservedRunningTime="2026-11-01 05:10:40.852206968 +0000 UTC m=+457.742582944"
Nov 01 05:10:43.640800 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.640752    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn"]
Nov 01 05:10:43.641180 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.641165    4864 cpu_manager.go:414] "containerMap: removing container" podUID="6a1dc6cc-81dc-4e67-ac73-da37c8e32704" containerName="manager"
Nov 01 05:10:43.641180 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.641176    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a1dc6cc-81dc-4e67-ac73-da37c8e32704" containerName="manager"
Nov 01 05:10:43.641267 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.641253    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a1dc6cc-81dc-4e67-ac73-da37c8e32704" containerName="manager"
Nov 01 05:10:43.644322 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.644302    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn"
Nov 01 05:10:43.703336 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.703305    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn"]
Nov 01 05:10:43.745939 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.745909    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-r7ljx\""
Nov 01 05:10:43.745939 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.745909    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Nov 01 05:10:43.752467 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.752440    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-h2pkm"]
Nov 01 05:10:43.756094 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.756074    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-h2pkm"
Nov 01 05:10:43.763345 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.763315    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk87k\" (UniqueName: \"kubernetes.io/projected/7fb45bb3-4226-4259-84ca-eda06dd2e0d2-kube-api-access-mk87k\") pod \"llmisvc-controller-manager-869d5dcc5d-4mqpn\" (UID: \"7fb45bb3-4226-4259-84ca-eda06dd2e0d2\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn"
Nov 01 05:10:43.763570 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.763545    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7fb45bb3-4226-4259-84ca-eda06dd2e0d2-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-4mqpn\" (UID: \"7fb45bb3-4226-4259-84ca-eda06dd2e0d2\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn"
Nov 01 05:10:43.807850 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.807820    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-l9x8b\""
Nov 01 05:10:43.808070 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.807941    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Nov 01 05:10:43.814899 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.814872    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-h2pkm"]
Nov 01 05:10:43.865068 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.865032    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3340fa76-a621-451b-8c96-82b31e6b40d1-tls-certs\") pod \"model-serving-api-75bd89fdfc-h2pkm\" (UID: \"3340fa76-a621-451b-8c96-82b31e6b40d1\") " pod="opendatahub/model-serving-api-75bd89fdfc-h2pkm"
Nov 01 05:10:43.865275 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.865078    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6w9x\" (UniqueName: \"kubernetes.io/projected/3340fa76-a621-451b-8c96-82b31e6b40d1-kube-api-access-v6w9x\") pod \"model-serving-api-75bd89fdfc-h2pkm\" (UID: \"3340fa76-a621-451b-8c96-82b31e6b40d1\") " pod="opendatahub/model-serving-api-75bd89fdfc-h2pkm"
Nov 01 05:10:43.865275 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.865195    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk87k\" (UniqueName: \"kubernetes.io/projected/7fb45bb3-4226-4259-84ca-eda06dd2e0d2-kube-api-access-mk87k\") pod \"llmisvc-controller-manager-869d5dcc5d-4mqpn\" (UID: \"7fb45bb3-4226-4259-84ca-eda06dd2e0d2\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn"
Nov 01 05:10:43.865275 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.865243    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7fb45bb3-4226-4259-84ca-eda06dd2e0d2-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-4mqpn\" (UID: \"7fb45bb3-4226-4259-84ca-eda06dd2e0d2\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn"
Nov 01 05:10:43.867951 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.867925    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7fb45bb3-4226-4259-84ca-eda06dd2e0d2-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-4mqpn\" (UID: \"7fb45bb3-4226-4259-84ca-eda06dd2e0d2\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn"
Nov 01 05:10:43.925903 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.925819    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk87k\" (UniqueName: \"kubernetes.io/projected/7fb45bb3-4226-4259-84ca-eda06dd2e0d2-kube-api-access-mk87k\") pod \"llmisvc-controller-manager-869d5dcc5d-4mqpn\" (UID: \"7fb45bb3-4226-4259-84ca-eda06dd2e0d2\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn"
Nov 01 05:10:43.956854 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.956821    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn"
Nov 01 05:10:43.966054 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.966022    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3340fa76-a621-451b-8c96-82b31e6b40d1-tls-certs\") pod \"model-serving-api-75bd89fdfc-h2pkm\" (UID: \"3340fa76-a621-451b-8c96-82b31e6b40d1\") " pod="opendatahub/model-serving-api-75bd89fdfc-h2pkm"
Nov 01 05:10:43.966178 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.966070    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6w9x\" (UniqueName: \"kubernetes.io/projected/3340fa76-a621-451b-8c96-82b31e6b40d1-kube-api-access-v6w9x\") pod \"model-serving-api-75bd89fdfc-h2pkm\" (UID: \"3340fa76-a621-451b-8c96-82b31e6b40d1\") " pod="opendatahub/model-serving-api-75bd89fdfc-h2pkm"
Nov 01 05:10:43.968722 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:43.968548    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3340fa76-a621-451b-8c96-82b31e6b40d1-tls-certs\") pod \"model-serving-api-75bd89fdfc-h2pkm\" (UID: \"3340fa76-a621-451b-8c96-82b31e6b40d1\") " pod="opendatahub/model-serving-api-75bd89fdfc-h2pkm"
Nov 01 05:10:44.029393 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:44.029361    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6w9x\" (UniqueName: \"kubernetes.io/projected/3340fa76-a621-451b-8c96-82b31e6b40d1-kube-api-access-v6w9x\") pod \"model-serving-api-75bd89fdfc-h2pkm\" (UID: \"3340fa76-a621-451b-8c96-82b31e6b40d1\") " pod="opendatahub/model-serving-api-75bd89fdfc-h2pkm"
Nov 01 05:10:44.074481 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:44.074447    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-h2pkm"
Nov 01 05:10:44.331285 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:44.331260    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn"]
Nov 01 05:10:44.333086 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:10:44.333057    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7fb45bb3_4226_4259_84ca_eda06dd2e0d2.slice/crio-0172cf6937e8f0f3a37636a702d7a8f5b6fc4f869f97bdf4133e1ebc16175323 WatchSource:0}: Error finding container 0172cf6937e8f0f3a37636a702d7a8f5b6fc4f869f97bdf4133e1ebc16175323: Status 404 returned error can't find the container with id 0172cf6937e8f0f3a37636a702d7a8f5b6fc4f869f97bdf4133e1ebc16175323
Nov 01 05:10:44.450996 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:44.450972    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-h2pkm"]
Nov 01 05:10:44.452779 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:10:44.452757    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3340fa76_a621_451b_8c96_82b31e6b40d1.slice/crio-408f498e34204b9e03bced0db09f7c2a3df8d1f2aba49b0fc096a762c8f1ad5c WatchSource:0}: Error finding container 408f498e34204b9e03bced0db09f7c2a3df8d1f2aba49b0fc096a762c8f1ad5c: Status 404 returned error can't find the container with id 408f498e34204b9e03bced0db09f7c2a3df8d1f2aba49b0fc096a762c8f1ad5c
Nov 01 05:10:44.757808 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:44.757684    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn" event={"ID":"7fb45bb3-4226-4259-84ca-eda06dd2e0d2","Type":"ContainerStarted","Data":"0172cf6937e8f0f3a37636a702d7a8f5b6fc4f869f97bdf4133e1ebc16175323"}
Nov 01 05:10:44.759072 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:44.759045    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-h2pkm" event={"ID":"3340fa76-a621-451b-8c96-82b31e6b40d1","Type":"ContainerStarted","Data":"408f498e34204b9e03bced0db09f7c2a3df8d1f2aba49b0fc096a762c8f1ad5c"}
Nov 01 05:10:47.780161 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:47.780121    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn" event={"ID":"7fb45bb3-4226-4259-84ca-eda06dd2e0d2","Type":"ContainerStarted","Data":"8705967f7880352a481a3fd72db4b0a4441bc400df985b79172e20371c304812"}
Nov 01 05:10:47.780638 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:47.780266    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn"
Nov 01 05:10:47.781711 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:47.781688    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-h2pkm" event={"ID":"3340fa76-a621-451b-8c96-82b31e6b40d1","Type":"ContainerStarted","Data":"b1506326b72430945b25fc942745999ea403dc1df4be2295b8e75f9d1ccac1fb"}
Nov 01 05:10:47.781841 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:47.781740    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-h2pkm"
Nov 01 05:10:47.898098 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:47.898041    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn" podStartSLOduration=2.538869438 podStartE2EDuration="4.898026183s" totalImagesPullingTime="2.359156745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:10:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:10:47.895679895 +0000 UTC m=+464.786055868" watchObservedRunningTime="2026-11-01 05:10:47.898026183 +0000 UTC m=+464.788402158"
Nov 01 05:10:48.012692 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:48.012639    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-h2pkm" podStartSLOduration=2.7174857059999997 podStartE2EDuration="5.012624784s" totalImagesPullingTime="2.295139078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:10:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:10:48.012499243 +0000 UTC m=+464.902875220" watchObservedRunningTime="2026-11-01 05:10:48.012624784 +0000 UTC m=+464.903000760"
Nov 01 05:10:51.745437 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:51.745408    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d"
Nov 01 05:10:52.523605 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:10:52.523568    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:10:58.791871 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:10:58.791842    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-h2pkm"
Nov 01 05:11:11.860464 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:11.860429    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"]
Nov 01 05:11:12.106963 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.106878    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"]
Nov 01 05:11:12.107133 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.107008    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"
Nov 01 05:11:12.159090 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.159060    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 01 05:11:12.208391 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.208350    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9729w\""
Nov 01 05:11:12.208566 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.208424    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 01 05:11:12.208566 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.208467    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 01 05:11:12.208566 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.208483    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 01 05:11:12.223667 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.223641    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-maas-api-tls\") pod \"maas-api-7b479cdcf5-rlphj\" (UID: \"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6\") " pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"
Nov 01 05:11:12.223830 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.223674    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-openshift-service-ca\") pod \"maas-api-7b479cdcf5-rlphj\" (UID: \"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6\") " pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"
Nov 01 05:11:12.223874 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.223835    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-metrics-certs\") pod \"maas-api-7b479cdcf5-rlphj\" (UID: \"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6\") " pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"
Nov 01 05:11:12.223874 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.223866    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97cl5\" (UniqueName: \"kubernetes.io/projected/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-kube-api-access-97cl5\") pod \"maas-api-7b479cdcf5-rlphj\" (UID: \"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6\") " pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"
Nov 01 05:11:12.325054 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.325014    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-metrics-certs\") pod \"maas-api-7b479cdcf5-rlphj\" (UID: \"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6\") " pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"
Nov 01 05:11:12.325260 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.325070    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97cl5\" (UniqueName: \"kubernetes.io/projected/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-kube-api-access-97cl5\") pod \"maas-api-7b479cdcf5-rlphj\" (UID: \"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6\") " pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"
Nov 01 05:11:12.325260 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.325110    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-maas-api-tls\") pod \"maas-api-7b479cdcf5-rlphj\" (UID: \"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6\") " pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"
Nov 01 05:11:12.325260 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.325136    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-openshift-service-ca\") pod \"maas-api-7b479cdcf5-rlphj\" (UID: \"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6\") " pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"
Nov 01 05:11:12.325845 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.325822    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-openshift-service-ca\") pod \"maas-api-7b479cdcf5-rlphj\" (UID: \"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6\") " pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"
Nov 01 05:11:12.327851 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.327830    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-metrics-certs\") pod \"maas-api-7b479cdcf5-rlphj\" (UID: \"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6\") " pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"
Nov 01 05:11:12.327913 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.327872    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-maas-api-tls\") pod \"maas-api-7b479cdcf5-rlphj\" (UID: \"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6\") " pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"
Nov 01 05:11:12.386512 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.386439    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97cl5\" (UniqueName: \"kubernetes.io/projected/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-kube-api-access-97cl5\") pod \"maas-api-7b479cdcf5-rlphj\" (UID: \"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6\") " pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"
Nov 01 05:11:12.448803 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.448764    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"
Nov 01 05:11:12.845026 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.845002    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"]
Nov 01 05:11:12.846873 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:11:12.846851    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ea7d2c8_500a_4d5c_8c47_6d12ccdceec6.slice/crio-00e3507a0c91ecfa48e0ee15421cebe7a79307c3104be2a1dfc115411fa1202f WatchSource:0}: Error finding container 00e3507a0c91ecfa48e0ee15421cebe7a79307c3104be2a1dfc115411fa1202f: Status 404 returned error can't find the container with id 00e3507a0c91ecfa48e0ee15421cebe7a79307c3104be2a1dfc115411fa1202f
Nov 01 05:11:12.915606 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:12.915569    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj" event={"ID":"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6","Type":"ContainerStarted","Data":"00e3507a0c91ecfa48e0ee15421cebe7a79307c3104be2a1dfc115411fa1202f"}
Nov 01 05:11:14.928649 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:14.928558    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj" event={"ID":"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6","Type":"ContainerStarted","Data":"eeca14d8025ad0179799015d1d4d4f6c900b0e6b34c1cb9ff11b2d7ef4282bed"}
Nov 01 05:11:14.928649 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:14.928620    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"
Nov 01 05:11:15.047632 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:15.047564    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj" podStartSLOduration=2.2810547740000002 podStartE2EDuration="4.047543331s" totalImagesPullingTime="1.766488557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:11:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:11:15.046514435 +0000 UTC m=+491.936890424" watchObservedRunningTime="2026-11-01 05:11:15.047543331 +0000 UTC m=+491.937919311"
Nov 01 05:11:18.789698 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:18.789668    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn"
Nov 01 05:11:20.939630 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:20.939600    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"
Nov 01 05:11:22.524119 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:11:22.524083    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:11:52.524423 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:11:52.524377    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:11:56.417801 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:56.417749    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-8krz2"]
Nov 01 05:11:56.421617 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:56.421594    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-8krz2"
Nov 01 05:11:56.480381 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:56.480344    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-8krz2"]
Nov 01 05:11:56.523301 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:56.523262    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f4aae4c0-bc4d-4889-92e6-b5e4d22d7a06-cert\") pod \"llmisvc-controller-manager-6466c554d8-8krz2\" (UID: \"f4aae4c0-bc4d-4889-92e6-b5e4d22d7a06\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-8krz2"
Nov 01 05:11:56.523471 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:56.523313    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6wlq\" (UniqueName: \"kubernetes.io/projected/f4aae4c0-bc4d-4889-92e6-b5e4d22d7a06-kube-api-access-s6wlq\") pod \"llmisvc-controller-manager-6466c554d8-8krz2\" (UID: \"f4aae4c0-bc4d-4889-92e6-b5e4d22d7a06\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-8krz2"
Nov 01 05:11:56.624567 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:56.624532    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f4aae4c0-bc4d-4889-92e6-b5e4d22d7a06-cert\") pod \"llmisvc-controller-manager-6466c554d8-8krz2\" (UID: \"f4aae4c0-bc4d-4889-92e6-b5e4d22d7a06\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-8krz2"
Nov 01 05:11:56.624739 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:56.624581    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6wlq\" (UniqueName: \"kubernetes.io/projected/f4aae4c0-bc4d-4889-92e6-b5e4d22d7a06-kube-api-access-s6wlq\") pod \"llmisvc-controller-manager-6466c554d8-8krz2\" (UID: \"f4aae4c0-bc4d-4889-92e6-b5e4d22d7a06\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-8krz2"
Nov 01 05:11:56.627218 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:56.627182    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f4aae4c0-bc4d-4889-92e6-b5e4d22d7a06-cert\") pod \"llmisvc-controller-manager-6466c554d8-8krz2\" (UID: \"f4aae4c0-bc4d-4889-92e6-b5e4d22d7a06\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-8krz2"
Nov 01 05:11:56.685616 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:56.685540    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6wlq\" (UniqueName: \"kubernetes.io/projected/f4aae4c0-bc4d-4889-92e6-b5e4d22d7a06-kube-api-access-s6wlq\") pod \"llmisvc-controller-manager-6466c554d8-8krz2\" (UID: \"f4aae4c0-bc4d-4889-92e6-b5e4d22d7a06\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-8krz2"
Nov 01 05:11:56.734499 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:56.734464    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-8krz2"
Nov 01 05:11:57.110409 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:57.110333    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-8krz2"]
Nov 01 05:11:57.114504 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:11:57.114474    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4aae4c0_bc4d_4889_92e6_b5e4d22d7a06.slice/crio-5afa49a2067943a172f540fdf1bc06ec640c9943bfa2168b3a4780bb63f2175f WatchSource:0}: Error finding container 5afa49a2067943a172f540fdf1bc06ec640c9943bfa2168b3a4780bb63f2175f: Status 404 returned error can't find the container with id 5afa49a2067943a172f540fdf1bc06ec640c9943bfa2168b3a4780bb63f2175f
Nov 01 05:11:57.156431 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:57.156399    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-8krz2" event={"ID":"f4aae4c0-bc4d-4889-92e6-b5e4d22d7a06","Type":"ContainerStarted","Data":"5afa49a2067943a172f540fdf1bc06ec640c9943bfa2168b3a4780bb63f2175f"}
Nov 01 05:11:58.164484 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:58.164383    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-8krz2" event={"ID":"f4aae4c0-bc4d-4889-92e6-b5e4d22d7a06","Type":"ContainerStarted","Data":"4aa3b33567a3d68abffc7a7a126b654dde5f9a28b2d3adf2c1a6e2b0394c6fa3"}
Nov 01 05:11:58.164890 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:58.164504    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-8krz2"
Nov 01 05:11:58.280434 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:11:58.280378    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-8krz2" podStartSLOduration=1.540846051 podStartE2EDuration="2.280362349s" totalImagesPullingTime="739.516298ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:11:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:11:58.279223494 +0000 UTC m=+535.169599471" watchObservedRunningTime="2026-11-01 05:11:58.280362349 +0000 UTC m=+535.170738325"
Nov 01 05:12:22.523609 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:12:22.523507    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:12:29.171934 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:29.171905    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-8krz2"
Nov 01 05:12:29.339117 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:29.339085    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn"]
Nov 01 05:12:29.339550 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:29.339499    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn" podUID="7fb45bb3-4226-4259-84ca-eda06dd2e0d2" containerName="manager" containerID="cri-o://8705967f7880352a481a3fd72db4b0a4441bc400df985b79172e20371c304812" gracePeriod=30
Nov 01 05:12:29.591801 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:29.591751    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn"
Nov 01 05:12:29.717751 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:29.717679    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7fb45bb3-4226-4259-84ca-eda06dd2e0d2-kube-api-access-mk87k\" (UniqueName: \"kubernetes.io/projected/7fb45bb3-4226-4259-84ca-eda06dd2e0d2-kube-api-access-mk87k\") pod \"7fb45bb3-4226-4259-84ca-eda06dd2e0d2\" (UID: \"7fb45bb3-4226-4259-84ca-eda06dd2e0d2\") "
Nov 01 05:12:29.717908 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:29.717830    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7fb45bb3-4226-4259-84ca-eda06dd2e0d2-cert\" (UniqueName: \"kubernetes.io/secret/7fb45bb3-4226-4259-84ca-eda06dd2e0d2-cert\") pod \"7fb45bb3-4226-4259-84ca-eda06dd2e0d2\" (UID: \"7fb45bb3-4226-4259-84ca-eda06dd2e0d2\") "
Nov 01 05:12:29.719956 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:29.719923    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fb45bb3-4226-4259-84ca-eda06dd2e0d2-kube-api-access-mk87k" pod "7fb45bb3-4226-4259-84ca-eda06dd2e0d2" (UID: "7fb45bb3-4226-4259-84ca-eda06dd2e0d2"). InnerVolumeSpecName "kube-api-access-mk87k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:12:29.720068 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:29.719954    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fb45bb3-4226-4259-84ca-eda06dd2e0d2-cert" pod "7fb45bb3-4226-4259-84ca-eda06dd2e0d2" (UID: "7fb45bb3-4226-4259-84ca-eda06dd2e0d2"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:12:29.819377 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:29.819343    4864 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7fb45bb3-4226-4259-84ca-eda06dd2e0d2-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:12:29.819377 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:29.819371    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mk87k\" (UniqueName: \"kubernetes.io/projected/7fb45bb3-4226-4259-84ca-eda06dd2e0d2-kube-api-access-mk87k\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:12:30.338310 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:30.338273    4864 generic.go:356] "Generic (PLEG): container finished" podID="7fb45bb3-4226-4259-84ca-eda06dd2e0d2" containerID="8705967f7880352a481a3fd72db4b0a4441bc400df985b79172e20371c304812" exitCode=0
Nov 01 05:12:30.338310 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:30.338316    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn"
Nov 01 05:12:30.339317 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:30.338349    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn" event={"ID":"7fb45bb3-4226-4259-84ca-eda06dd2e0d2","Type":"ContainerDied","Data":"8705967f7880352a481a3fd72db4b0a4441bc400df985b79172e20371c304812"}
Nov 01 05:12:30.339317 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:30.338406    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn" event={"ID":"7fb45bb3-4226-4259-84ca-eda06dd2e0d2","Type":"ContainerDied","Data":"0172cf6937e8f0f3a37636a702d7a8f5b6fc4f869f97bdf4133e1ebc16175323"}
Nov 01 05:12:30.339317 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:30.338429    4864 scope.go:122] "RemoveContainer" containerID="8705967f7880352a481a3fd72db4b0a4441bc400df985b79172e20371c304812"
Nov 01 05:12:30.352216 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:30.352196    4864 scope.go:122] "RemoveContainer" containerID="8705967f7880352a481a3fd72db4b0a4441bc400df985b79172e20371c304812"
Nov 01 05:12:30.352642 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:12:30.352622    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8705967f7880352a481a3fd72db4b0a4441bc400df985b79172e20371c304812\": container with ID starting with 8705967f7880352a481a3fd72db4b0a4441bc400df985b79172e20371c304812 not found: ID does not exist" containerID="8705967f7880352a481a3fd72db4b0a4441bc400df985b79172e20371c304812"
Nov 01 05:12:30.352713 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:30.352652    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8705967f7880352a481a3fd72db4b0a4441bc400df985b79172e20371c304812"} err="failed to get container status \"8705967f7880352a481a3fd72db4b0a4441bc400df985b79172e20371c304812\": rpc error: code = NotFound desc = could not find container \"8705967f7880352a481a3fd72db4b0a4441bc400df985b79172e20371c304812\": container with ID starting with 8705967f7880352a481a3fd72db4b0a4441bc400df985b79172e20371c304812 not found: ID does not exist"
Nov 01 05:12:30.501214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:30.501181    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn"]
Nov 01 05:12:30.550645 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:30.550617    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn"]
Nov 01 05:12:31.247074 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:31.247044    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7fb45bb3-4226-4259-84ca-eda06dd2e0d2" path="/var/lib/kubelet/pods/7fb45bb3-4226-4259-84ca-eda06dd2e0d2/volumes"
Nov 01 05:12:47.440352 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:47.440268    4864 generic.go:356] "Generic (PLEG): container finished" podID="92982041-1d56-4b3d-8067-3d393da5cad5" containerID="b5f2cb8d051a37a2e6e90dcc06631283b736fc909bd5cb529b03394619a9336f" exitCode=0
Nov 01 05:12:47.440352 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:47.440339    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t" event={"ID":"92982041-1d56-4b3d-8067-3d393da5cad5","Type":"ContainerDied","Data":"b5f2cb8d051a37a2e6e90dcc06631283b736fc909bd5cb529b03394619a9336f"}
Nov 01 05:12:47.440835 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:47.440686    4864 scope.go:122] "RemoveContainer" containerID="b5f2cb8d051a37a2e6e90dcc06631283b736fc909bd5cb529b03394619a9336f"
Nov 01 05:12:47.441809 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:47.441763    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:12:47.441905 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:47.441851    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:12:47.441905 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:47.441882    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:12:48.447741 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:48.447706    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t" event={"ID":"92982041-1d56-4b3d-8067-3d393da5cad5","Type":"ContainerStarted","Data":"efe129f4e5819f7d48e44badc1116ff0e45dedb132a8eeb40cef6b9819bafcc4"}
Nov 01 05:12:49.190737 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:49.190705    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:12:49.191778 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:49.191747    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-ck46t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Nov 01 05:12:49.191937 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:49.191835    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t" podUID="92982041-1d56-4b3d-8067-3d393da5cad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Nov 01 05:12:50.190386 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:50.190350    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-ck46t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Nov 01 05:12:50.190852 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:50.190403    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t" podUID="92982041-1d56-4b3d-8067-3d393da5cad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Nov 01 05:12:51.190477 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:51.190380    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-ck46t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Nov 01 05:12:51.190477 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:51.190439    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t" podUID="92982041-1d56-4b3d-8067-3d393da5cad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Nov 01 05:12:51.487722 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:51.487625    4864 generic.go:356] "Generic (PLEG): container finished" podID="90275ed4-c333-4c1f-8583-853bf05c6165" containerID="672b80968be43108634f54480741ede4c8d22996ddfe8ab778294a22f1928ce1" exitCode=1
Nov 01 05:12:51.487722 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:51.487698    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74b5bc6fff-vqr9s" event={"ID":"90275ed4-c333-4c1f-8583-853bf05c6165","Type":"ContainerDied","Data":"672b80968be43108634f54480741ede4c8d22996ddfe8ab778294a22f1928ce1"}
Nov 01 05:12:51.488141 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:51.488123    4864 scope.go:122] "RemoveContainer" containerID="672b80968be43108634f54480741ede4c8d22996ddfe8ab778294a22f1928ce1"
Nov 01 05:12:51.611208 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:51.611172    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:12:52.190476 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:52.190440    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-ck46t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Nov 01 05:12:52.190933 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:52.190498    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t" podUID="92982041-1d56-4b3d-8067-3d393da5cad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Nov 01 05:12:52.495303 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:52.495277    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74b5bc6fff-vqr9s" event={"ID":"90275ed4-c333-4c1f-8583-853bf05c6165","Type":"ContainerStarted","Data":"cf2c69cc0bd7c3fe9591c90168ad0e21375c81582428a5f33056eb4efe58f217"}
Nov 01 05:12:52.495516 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:52.495485    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74b5bc6fff-vqr9s"
Nov 01 05:12:52.522670 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:12:52.522635    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:12:53.190420 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:53.190375    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-ck46t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Nov 01 05:12:53.190637 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:53.190441    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t" podUID="92982041-1d56-4b3d-8067-3d393da5cad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Nov 01 05:12:54.194829 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:54.194798    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:12:54.195709 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:12:54.195688    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t"
Nov 01 05:13:03.503154 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:03.503120    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74b5bc6fff-vqr9s"
Nov 01 05:13:03.525033 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:03.525009    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:13:03.529701 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:03.529678    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:13:03.533450 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:03.533431    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:13:03.537704 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:03.537684    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:13:22.524982 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:13:22.524943    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:13:40.636078 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.636045    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"]
Nov 01 05:13:40.636560 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.636454    4864 cpu_manager.go:414] "containerMap: removing container" podUID="7fb45bb3-4226-4259-84ca-eda06dd2e0d2" containerName="manager"
Nov 01 05:13:40.636560 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.636465    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7fb45bb3-4226-4259-84ca-eda06dd2e0d2" containerName="manager"
Nov 01 05:13:40.636560 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.636548    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="7fb45bb3-4226-4259-84ca-eda06dd2e0d2" containerName="manager"
Nov 01 05:13:40.639977 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.639961    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.686034 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.685999    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"]
Nov 01 05:13:40.690037 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.690013    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.737251 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.737213    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"]
Nov 01 05:13:40.754603 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.754564    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.754603 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.754608    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.754839 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.754629    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.754839 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.754650    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.754839 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.754757    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.754941 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.754832    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxrct\" (UniqueName: \"kubernetes.io/projected/59d81eeb-a67f-4bc9-a951-3145d53aef60-kube-api-access-gxrct\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.754941 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.754868    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.754941 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.754916    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.755034 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.754951    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.755066 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.755053    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fdqg\" (UniqueName: \"kubernetes.io/projected/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-kube-api-access-9fdqg\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.755121 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.755106    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.755198 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.755184    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.755235 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.755210    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.755267 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.755250    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.755299 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.755272    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.755332 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.755298    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.755332 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.755324    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.755395 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.755366    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59d81eeb-a67f-4bc9-a951-3145d53aef60-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.785209 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.785161    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"]
Nov 01 05:13:40.856053 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.856012    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.856053 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.856057    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.856334 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.856180    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.856334 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.856222    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.856334 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.856239    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.856334 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.856264    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59d81eeb-a67f-4bc9-a951-3145d53aef60-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.856334 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.856289    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.856334 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.856303    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.856334 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.856319    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.856679 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.856457    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.856679 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.856581    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.856679 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.856654    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.857109 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.856740    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.857109 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.856768    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.857109 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.856817    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxrct\" (UniqueName: \"kubernetes.io/projected/59d81eeb-a67f-4bc9-a951-3145d53aef60-kube-api-access-gxrct\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.857109 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.856847    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.857109 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.856881    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.857109 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.856936    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.857109 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.857004    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fdqg\" (UniqueName: \"kubernetes.io/projected/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-kube-api-access-9fdqg\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.857109 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.857060    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.857109 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.857076    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59d81eeb-a67f-4bc9-a951-3145d53aef60-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.857579 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.857116    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.857579 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.857128    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.857579 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.857334    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.857579 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.857416    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.857804 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.857635    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.857804 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.857669    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.858323 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.858297    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.858894 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.858865    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.859021 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.859001    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.859469 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.859443    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.859561 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.859541    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.918121 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.918041    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.918280 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.918136    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxrct\" (UniqueName: \"kubernetes.io/projected/59d81eeb-a67f-4bc9-a951-3145d53aef60-kube-api-access-gxrct\") pod \"maas-default-gateway-openshift-default-58d898bd55-8lzd5\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:40.918353 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.918337    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.918431 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.918413    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fdqg\" (UniqueName: \"kubernetes.io/projected/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-kube-api-access-9fdqg\") pod \"maas-default-gateway-openshift-default-58d898bd55-lpklt\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:40.953775 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:40.953744    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:41.003721 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:41.003681    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:41.383909 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:41.383886    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"]
Nov 01 05:13:41.385817 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:13:41.385758    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27a97af3_2e36_4317_9ae5_b5ee7ff7ac89.slice/crio-440baac89bc5b5259cb65f59e505f933ae910c043e6ba57dd7935ec3527e668e WatchSource:0}: Error finding container 440baac89bc5b5259cb65f59e505f933ae910c043e6ba57dd7935ec3527e668e: Status 404 returned error can't find the container with id 440baac89bc5b5259cb65f59e505f933ae910c043e6ba57dd7935ec3527e668e
Nov 01 05:13:41.388101 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:41.388072    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:13:41.388198 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:41.388133    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:13:41.388198 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:41.388167    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:13:41.433972 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:41.433948    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"]
Nov 01 05:13:41.441762 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:13:41.441731    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59d81eeb_a67f_4bc9_a951_3145d53aef60.slice/crio-5a993a0ee2e3b0bc8a0091d69389dad6ddb5423eead6f9b8a02852738c6e5586 WatchSource:0}: Error finding container 5a993a0ee2e3b0bc8a0091d69389dad6ddb5423eead6f9b8a02852738c6e5586: Status 404 returned error can't find the container with id 5a993a0ee2e3b0bc8a0091d69389dad6ddb5423eead6f9b8a02852738c6e5586
Nov 01 05:13:41.444021 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:41.443985    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:13:41.444090 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:41.444075    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:13:41.444137 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:41.444121    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:13:41.780699 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:41.777910    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt" event={"ID":"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89","Type":"ContainerStarted","Data":"44747da3fcc78d14a06394a5e019114f1ec89a1b763e830c8a6da2b5629a7574"}
Nov 01 05:13:41.780699 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:41.777986    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt" event={"ID":"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89","Type":"ContainerStarted","Data":"440baac89bc5b5259cb65f59e505f933ae910c043e6ba57dd7935ec3527e668e"}
Nov 01 05:13:41.780699 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:41.779930    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5" event={"ID":"59d81eeb-a67f-4bc9-a951-3145d53aef60","Type":"ContainerStarted","Data":"5b72186da76c62f65d774b6eda5d07981f48d09426895b19db9a1ab15b2efdef"}
Nov 01 05:13:41.780699 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:41.779986    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5" event={"ID":"59d81eeb-a67f-4bc9-a951-3145d53aef60","Type":"ContainerStarted","Data":"5a993a0ee2e3b0bc8a0091d69389dad6ddb5423eead6f9b8a02852738c6e5586"}
Nov 01 05:13:41.954094 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:41.954055    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:42.005811 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:42.005192    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:42.013809 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:42.010986    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-8lzd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 01 05:13:42.013809 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:42.011060    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5" podUID="59d81eeb-a67f-4bc9-a951-3145d53aef60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 01 05:13:42.027689 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:42.027634    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt" podStartSLOduration=2.02761714 podStartE2EDuration="2.02761714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:13:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:13:41.906076156 +0000 UTC m=+638.796452149" watchObservedRunningTime="2026-11-01 05:13:42.02761714 +0000 UTC m=+638.917993116"
Nov 01 05:13:42.028818 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:42.028762    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5" podStartSLOduration=2.028752456 podStartE2EDuration="2.028752456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:13:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:13:42.026920347 +0000 UTC m=+638.917296319" watchObservedRunningTime="2026-11-01 05:13:42.028752456 +0000 UTC m=+638.919128503"
Nov 01 05:13:42.960838 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:42.957086    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-lpklt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 01 05:13:42.960838 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:42.957150    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt" podUID="27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 01 05:13:43.007279 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:43.007215    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-8lzd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 01 05:13:43.007515 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:43.007299    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5" podUID="59d81eeb-a67f-4bc9-a951-3145d53aef60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 01 05:13:43.954612 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:43.954577    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-lpklt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 01 05:13:43.954822 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:43.954633    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt" podUID="27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 01 05:13:44.004811 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:44.004178    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-8lzd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 01 05:13:44.004811 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:44.004245    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5" podUID="59d81eeb-a67f-4bc9-a951-3145d53aef60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 01 05:13:44.955018 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:44.954982    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-lpklt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 01 05:13:44.955191 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:44.955061    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt" podUID="27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 01 05:13:45.006809 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:45.006148    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-8lzd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 01 05:13:45.006809 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:45.006211    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5" podUID="59d81eeb-a67f-4bc9-a951-3145d53aef60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 01 05:13:45.954872 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:45.954824    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-lpklt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 01 05:13:45.955090 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:45.954893    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt" podUID="27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 01 05:13:46.007203 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:46.007163    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-8lzd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 01 05:13:46.007809 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:46.007750    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5" podUID="59d81eeb-a67f-4bc9-a951-3145d53aef60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 01 05:13:46.954716 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:46.954679    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-lpklt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 01 05:13:46.954999 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:46.954737    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt" podUID="27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 01 05:13:47.006214 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:47.006171    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-8lzd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 01 05:13:47.006523 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:47.006490    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5" podUID="59d81eeb-a67f-4bc9-a951-3145d53aef60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 01 05:13:47.954302 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:47.954262    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-lpklt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 01 05:13:47.954736 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:47.954345    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt" podUID="27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 01 05:13:48.004632 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:48.004599    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-8lzd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 01 05:13:48.004846 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:48.004669    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5" podUID="59d81eeb-a67f-4bc9-a951-3145d53aef60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 01 05:13:49.955769 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:49.955713    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-lpklt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 05:13:49.956345 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:49.955802    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt" podUID="27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 05:13:50.004662 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:50.004609    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-8lzd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 05:13:50.004895 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:50.004680    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5" podUID="59d81eeb-a67f-4bc9-a951-3145d53aef60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 05:13:50.997028 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:50.996986    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-lpklt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 05:13:50.997726 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:50.997691    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt" podUID="27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 05:13:51.045987 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:51.045935    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-8lzd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 05:13:51.046197 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:51.046015    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5" podUID="59d81eeb-a67f-4bc9-a951-3145d53aef60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 05:13:51.678429 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:51.678395    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:51.679584 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:51.679559    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:51.680669 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:51.680645    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:13:51.803142 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:51.803106    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:51.803910 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:51.803892    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:51.804802 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:13:51.804763    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:13:52.534848 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:13:52.534735    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:14:22.523491 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:14:22.523404    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:14:52.524282 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:14:52.524238    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:15:11.831495 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:11.831457    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"]
Nov 01 05:15:11.835392 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:11.835373    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"
Nov 01 05:15:11.889634 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:11.889602    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-a48099-serving-cert\""
Nov 01 05:15:11.941905 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:11.941871    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"]
Nov 01 05:15:11.974211 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:11.974174    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2143de82-6f2b-403c-a702-f248c5013318-maas-api-tls\") pod \"maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q\" (UID: \"2143de82-6f2b-403c-a702-f248c5013318\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"
Nov 01 05:15:11.974418 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:11.974336    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vln7s\" (UniqueName: \"kubernetes.io/projected/2143de82-6f2b-403c-a702-f248c5013318-kube-api-access-vln7s\") pod \"maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q\" (UID: \"2143de82-6f2b-403c-a702-f248c5013318\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"
Nov 01 05:15:11.974494 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:11.974432    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2143de82-6f2b-403c-a702-f248c5013318-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q\" (UID: \"2143de82-6f2b-403c-a702-f248c5013318\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"
Nov 01 05:15:11.974550 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:11.974493    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2143de82-6f2b-403c-a702-f248c5013318-metrics-certs\") pod \"maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q\" (UID: \"2143de82-6f2b-403c-a702-f248c5013318\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"
Nov 01 05:15:12.075625 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:12.075591    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2143de82-6f2b-403c-a702-f248c5013318-maas-api-tls\") pod \"maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q\" (UID: \"2143de82-6f2b-403c-a702-f248c5013318\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"
Nov 01 05:15:12.075907 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:12.075680    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vln7s\" (UniqueName: \"kubernetes.io/projected/2143de82-6f2b-403c-a702-f248c5013318-kube-api-access-vln7s\") pod \"maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q\" (UID: \"2143de82-6f2b-403c-a702-f248c5013318\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"
Nov 01 05:15:12.075907 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:12.075726    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2143de82-6f2b-403c-a702-f248c5013318-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q\" (UID: \"2143de82-6f2b-403c-a702-f248c5013318\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"
Nov 01 05:15:12.075907 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:15:12.075755    4864 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-serving-cert: secret "maas-api-e2e-worker-w4-a48099-serving-cert" not found
Nov 01 05:15:12.075907 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:12.075777    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2143de82-6f2b-403c-a702-f248c5013318-metrics-certs\") pod \"maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q\" (UID: \"2143de82-6f2b-403c-a702-f248c5013318\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"
Nov 01 05:15:12.075907 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:15:12.075856    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2143de82-6f2b-403c-a702-f248c5013318-maas-api-tls podName:2143de82-6f2b-403c-a702-f248c5013318 nodeName:}" failed. No retries permitted until 2026-11-01 05:15:12.575833115 +0000 UTC m=+729.466209092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2143de82-6f2b-403c-a702-f248c5013318-maas-api-tls") pod "maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q" (UID: "2143de82-6f2b-403c-a702-f248c5013318") : secret "maas-api-e2e-worker-w4-a48099-serving-cert" not found
Nov 01 05:15:12.076661 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:12.076633    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2143de82-6f2b-403c-a702-f248c5013318-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q\" (UID: \"2143de82-6f2b-403c-a702-f248c5013318\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"
Nov 01 05:15:12.078529 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:12.078506    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2143de82-6f2b-403c-a702-f248c5013318-metrics-certs\") pod \"maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q\" (UID: \"2143de82-6f2b-403c-a702-f248c5013318\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"
Nov 01 05:15:12.139419 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:12.139342    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vln7s\" (UniqueName: \"kubernetes.io/projected/2143de82-6f2b-403c-a702-f248c5013318-kube-api-access-vln7s\") pod \"maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q\" (UID: \"2143de82-6f2b-403c-a702-f248c5013318\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"
Nov 01 05:15:12.581344 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:12.581311    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2143de82-6f2b-403c-a702-f248c5013318-maas-api-tls\") pod \"maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q\" (UID: \"2143de82-6f2b-403c-a702-f248c5013318\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"
Nov 01 05:15:12.583932 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:12.583902    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2143de82-6f2b-403c-a702-f248c5013318-maas-api-tls\") pod \"maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q\" (UID: \"2143de82-6f2b-403c-a702-f248c5013318\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"
Nov 01 05:15:12.748373 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:12.748335    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"
Nov 01 05:15:13.150189 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:13.150157    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"]
Nov 01 05:15:13.150614 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:15:13.150408    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2143de82_6f2b_403c_a702_f248c5013318.slice/crio-ee8d08466ba846b7e42607593565f0561b8449069adab1140c79fb3d0bf3dd63 WatchSource:0}: Error finding container ee8d08466ba846b7e42607593565f0561b8449069adab1140c79fb3d0bf3dd63: Status 404 returned error can't find the container with id ee8d08466ba846b7e42607593565f0561b8449069adab1140c79fb3d0bf3dd63
Nov 01 05:15:13.151914 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:13.151894    4864 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 05:15:13.336379 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:13.336345    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q" event={"ID":"2143de82-6f2b-403c-a702-f248c5013318","Type":"ContainerStarted","Data":"ee8d08466ba846b7e42607593565f0561b8449069adab1140c79fb3d0bf3dd63"}
Nov 01 05:15:14.343601 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:14.343560    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q" event={"ID":"2143de82-6f2b-403c-a702-f248c5013318","Type":"ContainerStarted","Data":"c588e1cee2c779b3c031622eedf53fd68dea3939d6d526d055ac3895a48c8711"}
Nov 01 05:15:14.344063 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:14.343649    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"
Nov 01 05:15:20.356593 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:20.356562    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"
Nov 01 05:15:20.474528 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:20.474477    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q" podStartSLOduration=8.64791689 podStartE2EDuration="9.474462637s" totalImagesPullingTime="826.545747ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:15:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:15:14.460428803 +0000 UTC m=+731.350804782" watchObservedRunningTime="2026-11-01 05:15:20.474462637 +0000 UTC m=+737.364838612"
Nov 01 05:15:22.594810 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:15:22.591841    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:15:22.850075 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:22.849975    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-lpklt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 05:15:22.850075 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:15:22.850059    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt" podUID="27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 05:15:52.524264 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:15:52.524173    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:16:11.028560 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:11.028521    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"]
Nov 01 05:16:11.032616 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:11.032594    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"
Nov 01 05:16:11.084741 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:11.084706    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-2ea125-serving-cert\""
Nov 01 05:16:11.136432 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:11.136401    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"]
Nov 01 05:16:11.182402 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:11.182357    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8951da34-16d1-4f50-b68b-51dbde0ecc32-maas-api-tls\") pod \"maas-api-e2e-inf-a-2ea125-f4975f846-f2btg\" (UID: \"8951da34-16d1-4f50-b68b-51dbde0ecc32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"
Nov 01 05:16:11.182587 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:11.182533    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8951da34-16d1-4f50-b68b-51dbde0ecc32-metrics-certs\") pod \"maas-api-e2e-inf-a-2ea125-f4975f846-f2btg\" (UID: \"8951da34-16d1-4f50-b68b-51dbde0ecc32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"
Nov 01 05:16:11.182656 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:11.182580    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmb57\" (UniqueName: \"kubernetes.io/projected/8951da34-16d1-4f50-b68b-51dbde0ecc32-kube-api-access-xmb57\") pod \"maas-api-e2e-inf-a-2ea125-f4975f846-f2btg\" (UID: \"8951da34-16d1-4f50-b68b-51dbde0ecc32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"
Nov 01 05:16:11.182754 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:11.182738    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8951da34-16d1-4f50-b68b-51dbde0ecc32-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2ea125-f4975f846-f2btg\" (UID: \"8951da34-16d1-4f50-b68b-51dbde0ecc32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"
Nov 01 05:16:11.283774 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:11.283674    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8951da34-16d1-4f50-b68b-51dbde0ecc32-metrics-certs\") pod \"maas-api-e2e-inf-a-2ea125-f4975f846-f2btg\" (UID: \"8951da34-16d1-4f50-b68b-51dbde0ecc32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"
Nov 01 05:16:11.283774 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:11.283735    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmb57\" (UniqueName: \"kubernetes.io/projected/8951da34-16d1-4f50-b68b-51dbde0ecc32-kube-api-access-xmb57\") pod \"maas-api-e2e-inf-a-2ea125-f4975f846-f2btg\" (UID: \"8951da34-16d1-4f50-b68b-51dbde0ecc32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"
Nov 01 05:16:11.283774 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:11.283776    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8951da34-16d1-4f50-b68b-51dbde0ecc32-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2ea125-f4975f846-f2btg\" (UID: \"8951da34-16d1-4f50-b68b-51dbde0ecc32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"
Nov 01 05:16:11.284093 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:11.283900    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8951da34-16d1-4f50-b68b-51dbde0ecc32-maas-api-tls\") pod \"maas-api-e2e-inf-a-2ea125-f4975f846-f2btg\" (UID: \"8951da34-16d1-4f50-b68b-51dbde0ecc32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"
Nov 01 05:16:11.284093 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:16:11.284024    4864 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-serving-cert: secret "maas-api-e2e-inf-a-2ea125-serving-cert" not found
Nov 01 05:16:11.284203 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:16:11.284097    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8951da34-16d1-4f50-b68b-51dbde0ecc32-maas-api-tls podName:8951da34-16d1-4f50-b68b-51dbde0ecc32 nodeName:}" failed. No retries permitted until 2026-11-01 05:16:11.784075903 +0000 UTC m=+788.674451870 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8951da34-16d1-4f50-b68b-51dbde0ecc32-maas-api-tls") pod "maas-api-e2e-inf-a-2ea125-f4975f846-f2btg" (UID: "8951da34-16d1-4f50-b68b-51dbde0ecc32") : secret "maas-api-e2e-inf-a-2ea125-serving-cert" not found
Nov 01 05:16:11.285134 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:11.285063    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8951da34-16d1-4f50-b68b-51dbde0ecc32-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2ea125-f4975f846-f2btg\" (UID: \"8951da34-16d1-4f50-b68b-51dbde0ecc32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"
Nov 01 05:16:11.286901 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:11.286874    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8951da34-16d1-4f50-b68b-51dbde0ecc32-metrics-certs\") pod \"maas-api-e2e-inf-a-2ea125-f4975f846-f2btg\" (UID: \"8951da34-16d1-4f50-b68b-51dbde0ecc32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"
Nov 01 05:16:11.346837 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:11.346805    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmb57\" (UniqueName: \"kubernetes.io/projected/8951da34-16d1-4f50-b68b-51dbde0ecc32-kube-api-access-xmb57\") pod \"maas-api-e2e-inf-a-2ea125-f4975f846-f2btg\" (UID: \"8951da34-16d1-4f50-b68b-51dbde0ecc32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"
Nov 01 05:16:11.789224 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:11.789181    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8951da34-16d1-4f50-b68b-51dbde0ecc32-maas-api-tls\") pod \"maas-api-e2e-inf-a-2ea125-f4975f846-f2btg\" (UID: \"8951da34-16d1-4f50-b68b-51dbde0ecc32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"
Nov 01 05:16:11.791942 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:11.791910    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8951da34-16d1-4f50-b68b-51dbde0ecc32-maas-api-tls\") pod \"maas-api-e2e-inf-a-2ea125-f4975f846-f2btg\" (UID: \"8951da34-16d1-4f50-b68b-51dbde0ecc32\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"
Nov 01 05:16:11.946955 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:11.946923    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"
Nov 01 05:16:12.325182 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:12.325151    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"]
Nov 01 05:16:12.326665 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:16:12.326632    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8951da34_16d1_4f50_b68b_51dbde0ecc32.slice/crio-ed953b5149158ece179d2c00084a34741a51b83e3a804e5e92a85767c50b6691 WatchSource:0}: Error finding container ed953b5149158ece179d2c00084a34741a51b83e3a804e5e92a85767c50b6691: Status 404 returned error can't find the container with id ed953b5149158ece179d2c00084a34741a51b83e3a804e5e92a85767c50b6691
Nov 01 05:16:12.743088 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:12.743005    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg" event={"ID":"8951da34-16d1-4f50-b68b-51dbde0ecc32","Type":"ContainerStarted","Data":"ed953b5149158ece179d2c00084a34741a51b83e3a804e5e92a85767c50b6691"}
Nov 01 05:16:13.750098 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:13.750063    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg" event={"ID":"8951da34-16d1-4f50-b68b-51dbde0ecc32","Type":"ContainerStarted","Data":"e4a6bae87310a51248b246580b8f6bbe23fbcd5f33a22c01321856c0a8448d9a"}
Nov 01 05:16:13.750519 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:13.750186    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"
Nov 01 05:16:13.868079 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:13.868012    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg" podStartSLOduration=2.995942436 podStartE2EDuration="3.867991722s" totalImagesPullingTime="872.049286ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:16:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:16:13.866473304 +0000 UTC m=+790.756849281" watchObservedRunningTime="2026-11-01 05:16:13.867991722 +0000 UTC m=+790.758367699"
Nov 01 05:16:19.763891 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:19.763856    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"
Nov 01 05:16:22.523719 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:16:22.523683    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:16:44.674977 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.674943    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"]
Nov 01 05:16:44.678836 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.678814    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:44.780307 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.780274    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-3d70ff\"/\"premium-w2-3d70ff-kserve-self-signed-certs\""
Nov 01 05:16:44.780504 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.780379    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-3d70ff\"/\"default-dockercfg-rz4dl\""
Nov 01 05:16:44.780589 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.780575    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-3d70ff\"/\"openshift-service-ca.crt\""
Nov 01 05:16:44.780654 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.780612    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-3d70ff\"/\"kube-root-ca.crt\""
Nov 01 05:16:44.783968 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.783947    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"]
Nov 01 05:16:44.813160 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.813129    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b28a4792-6e4e-4f9b-80e0-737dc18ba934-tls-certs\") pod \"premium-w2-3d70ff-kserve-c975d498d-fhrkk\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") " pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:44.813338 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.813313    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-model-cache\") pod \"premium-w2-3d70ff-kserve-c975d498d-fhrkk\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") " pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:44.813423 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.813379    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-dshm\") pod \"premium-w2-3d70ff-kserve-c975d498d-fhrkk\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") " pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:44.813524 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.813495    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-home\") pod \"premium-w2-3d70ff-kserve-c975d498d-fhrkk\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") " pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:44.813663 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.813566    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-tmp-dir\") pod \"premium-w2-3d70ff-kserve-c975d498d-fhrkk\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") " pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:44.813663 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.813612    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk6lj\" (UniqueName: \"kubernetes.io/projected/b28a4792-6e4e-4f9b-80e0-737dc18ba934-kube-api-access-dk6lj\") pod \"premium-w2-3d70ff-kserve-c975d498d-fhrkk\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") " pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:44.914895 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.914855    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b28a4792-6e4e-4f9b-80e0-737dc18ba934-tls-certs\") pod \"premium-w2-3d70ff-kserve-c975d498d-fhrkk\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") " pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:44.915088 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.914918    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-model-cache\") pod \"premium-w2-3d70ff-kserve-c975d498d-fhrkk\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") " pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:44.915088 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.914975    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-dshm\") pod \"premium-w2-3d70ff-kserve-c975d498d-fhrkk\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") " pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:44.915088 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.915010    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-home\") pod \"premium-w2-3d70ff-kserve-c975d498d-fhrkk\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") " pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:44.915088 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.915063    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-tmp-dir\") pod \"premium-w2-3d70ff-kserve-c975d498d-fhrkk\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") " pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:44.915294 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.915111    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk6lj\" (UniqueName: \"kubernetes.io/projected/b28a4792-6e4e-4f9b-80e0-737dc18ba934-kube-api-access-dk6lj\") pod \"premium-w2-3d70ff-kserve-c975d498d-fhrkk\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") " pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:44.915455 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.915425    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-model-cache\") pod \"premium-w2-3d70ff-kserve-c975d498d-fhrkk\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") " pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:44.915665 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.915556    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-tmp-dir\") pod \"premium-w2-3d70ff-kserve-c975d498d-fhrkk\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") " pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:44.915986 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.915961    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-home\") pod \"premium-w2-3d70ff-kserve-c975d498d-fhrkk\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") " pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:44.917546 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.917520    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-dshm\") pod \"premium-w2-3d70ff-kserve-c975d498d-fhrkk\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") " pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:44.917814 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.917779    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b28a4792-6e4e-4f9b-80e0-737dc18ba934-tls-certs\") pod \"premium-w2-3d70ff-kserve-c975d498d-fhrkk\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") " pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:44.976290 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.976196    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk6lj\" (UniqueName: \"kubernetes.io/projected/b28a4792-6e4e-4f9b-80e0-737dc18ba934-kube-api-access-dk6lj\") pod \"premium-w2-3d70ff-kserve-c975d498d-fhrkk\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") " pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:44.991013 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:44.990985    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:45.406820 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:45.406752    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"]
Nov 01 05:16:45.408407 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:16:45.408381    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb28a4792_6e4e_4f9b_80e0_737dc18ba934.slice/crio-9a8db4b32e877236690a52bd3f79626d6d8bcc6eaee2c46b8a6931061d5b5b3c WatchSource:0}: Error finding container 9a8db4b32e877236690a52bd3f79626d6d8bcc6eaee2c46b8a6931061d5b5b3c: Status 404 returned error can't find the container with id 9a8db4b32e877236690a52bd3f79626d6d8bcc6eaee2c46b8a6931061d5b5b3c
Nov 01 05:16:45.943933 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:45.943898    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk" event={"ID":"b28a4792-6e4e-4f9b-80e0-737dc18ba934","Type":"ContainerStarted","Data":"9a8db4b32e877236690a52bd3f79626d6d8bcc6eaee2c46b8a6931061d5b5b3c"}
Nov 01 05:16:47.958499 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:47.958464    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk" event={"ID":"b28a4792-6e4e-4f9b-80e0-737dc18ba934","Type":"ContainerStarted","Data":"2d915a46cfc677786c61f627e76381ce8080fdb5150570722c35bd363c7ff0f5"}
Nov 01 05:16:47.991191 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:47.991144    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:48.104540 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:48.104439    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk" podStartSLOduration=1.689810913 podStartE2EDuration="4.104423049s" totalImagesPullingTime="2.414612136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:16:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:16:48.103034504 +0000 UTC m=+824.993410481" watchObservedRunningTime="2026-11-01 05:16:48.104423049 +0000 UTC m=+824.994799038"
Nov 01 05:16:52.524926 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:16:52.524890    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:16:54.992129 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:54.992088    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:55.008623 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:55.008597    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:16:55.019980 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:16:55.019957    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:17:07.363510 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.363474    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"]
Nov 01 05:17:07.367387 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.367371    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:07.419530 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.419492    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-a48099\"/\"kube-root-ca.crt\""
Nov 01 05:17:07.436371 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.436338    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-home\") pod \"premium-w4-a48099-kserve-6ccd45bcff-tp7mx\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") " pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:07.436371 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.436380    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-tmp-dir\") pod \"premium-w4-a48099-kserve-6ccd45bcff-tp7mx\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") " pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:07.436611 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.436502    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-dshm\") pod \"premium-w4-a48099-kserve-6ccd45bcff-tp7mx\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") " pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:07.436611 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.436546    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-model-cache\") pod \"premium-w4-a48099-kserve-6ccd45bcff-tp7mx\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") " pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:07.437901 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.437769    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b847150-a3e8-4210-af71-4f2ec0010c03-tls-certs\") pod \"premium-w4-a48099-kserve-6ccd45bcff-tp7mx\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") " pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:07.438150 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.437873    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn8wq\" (UniqueName: \"kubernetes.io/projected/7b847150-a3e8-4210-af71-4f2ec0010c03-kube-api-access-kn8wq\") pod \"premium-w4-a48099-kserve-6ccd45bcff-tp7mx\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") " pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:07.469050 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.469028    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-a48099\"/\"default-dockercfg-r7p77\""
Nov 01 05:17:07.469181 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.469161    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-a48099\"/\"premium-w4-a48099-kserve-self-signed-certs\""
Nov 01 05:17:07.474192 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.474173    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"]
Nov 01 05:17:07.519595 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.519569    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-a48099\"/\"openshift-service-ca.crt\""
Nov 01 05:17:07.538510 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.538474    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-model-cache\") pod \"premium-w4-a48099-kserve-6ccd45bcff-tp7mx\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") " pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:07.538510 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.538511    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b847150-a3e8-4210-af71-4f2ec0010c03-tls-certs\") pod \"premium-w4-a48099-kserve-6ccd45bcff-tp7mx\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") " pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:07.538776 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.538534    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn8wq\" (UniqueName: \"kubernetes.io/projected/7b847150-a3e8-4210-af71-4f2ec0010c03-kube-api-access-kn8wq\") pod \"premium-w4-a48099-kserve-6ccd45bcff-tp7mx\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") " pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:07.538776 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.538612    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-home\") pod \"premium-w4-a48099-kserve-6ccd45bcff-tp7mx\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") " pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:07.538776 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.538635    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-tmp-dir\") pod \"premium-w4-a48099-kserve-6ccd45bcff-tp7mx\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") " pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:07.538776 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.538705    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-dshm\") pod \"premium-w4-a48099-kserve-6ccd45bcff-tp7mx\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") " pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:07.539016 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.538978    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-model-cache\") pod \"premium-w4-a48099-kserve-6ccd45bcff-tp7mx\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") " pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:07.539110 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.539093    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-home\") pod \"premium-w4-a48099-kserve-6ccd45bcff-tp7mx\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") " pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:07.539190 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.539144    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-tmp-dir\") pod \"premium-w4-a48099-kserve-6ccd45bcff-tp7mx\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") " pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:07.541206 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.541181    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-dshm\") pod \"premium-w4-a48099-kserve-6ccd45bcff-tp7mx\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") " pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:07.541399 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.541378    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b847150-a3e8-4210-af71-4f2ec0010c03-tls-certs\") pod \"premium-w4-a48099-kserve-6ccd45bcff-tp7mx\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") " pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:07.600007 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.599967    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn8wq\" (UniqueName: \"kubernetes.io/projected/7b847150-a3e8-4210-af71-4f2ec0010c03-kube-api-access-kn8wq\") pod \"premium-w4-a48099-kserve-6ccd45bcff-tp7mx\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") " pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:07.706913 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:07.706828    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:08.096515 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:08.096495    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"]
Nov 01 05:17:08.098088 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:17:08.098065    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b847150_a3e8_4210_af71_4f2ec0010c03.slice/crio-6fdc72a3113734ca7a0f21771e646fe5460a2b1392c50b907522c4fe4bb547cb WatchSource:0}: Error finding container 6fdc72a3113734ca7a0f21771e646fe5460a2b1392c50b907522c4fe4bb547cb: Status 404 returned error can't find the container with id 6fdc72a3113734ca7a0f21771e646fe5460a2b1392c50b907522c4fe4bb547cb
Nov 01 05:17:09.013117 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:09.013084    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7443-a7f194"]
Nov 01 05:17:09.016666 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:09.016645    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-a7f194"
Nov 01 05:17:09.055498 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:09.055465    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prqdp\" (UniqueName: \"kubernetes.io/projected/2056f6bd-fc21-4909-a585-fe613ab32485-kube-api-access-prqdp\") pod \"test-curl-7443-a7f194\" (UID: \"2056f6bd-fc21-4909-a585-fe613ab32485\") " pod="opendatahub/test-curl-7443-a7f194"
Nov 01 05:17:09.071006 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:09.070979    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7443-a7f194"]
Nov 01 05:17:09.087678 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:09.087647    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx" event={"ID":"7b847150-a3e8-4210-af71-4f2ec0010c03","Type":"ContainerStarted","Data":"b8b9666ddce88d036d5be635b865379341eba436f0356259664609abd9f4dd43"}
Nov 01 05:17:09.087889 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:09.087688    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx" event={"ID":"7b847150-a3e8-4210-af71-4f2ec0010c03","Type":"ContainerStarted","Data":"6fdc72a3113734ca7a0f21771e646fe5460a2b1392c50b907522c4fe4bb547cb"}
Nov 01 05:17:09.117190 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:09.117156    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-wfzxk\""
Nov 01 05:17:09.157015 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:09.156973    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prqdp\" (UniqueName: \"kubernetes.io/projected/2056f6bd-fc21-4909-a585-fe613ab32485-kube-api-access-prqdp\") pod \"test-curl-7443-a7f194\" (UID: \"2056f6bd-fc21-4909-a585-fe613ab32485\") " pod="opendatahub/test-curl-7443-a7f194"
Nov 01 05:17:09.218066 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:09.218031    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prqdp\" (UniqueName: \"kubernetes.io/projected/2056f6bd-fc21-4909-a585-fe613ab32485-kube-api-access-prqdp\") pod \"test-curl-7443-a7f194\" (UID: \"2056f6bd-fc21-4909-a585-fe613ab32485\") " pod="opendatahub/test-curl-7443-a7f194"
Nov 01 05:17:09.245007 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:09.244948    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx" podStartSLOduration=2.244931809 podStartE2EDuration="2.244931809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:17:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:17:09.242755413 +0000 UTC m=+846.133131388" watchObservedRunningTime="2026-11-01 05:17:09.244931809 +0000 UTC m=+846.135307785"
Nov 01 05:17:09.329729 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:09.329687    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-a7f194"
Nov 01 05:17:09.707060 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:09.706955    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:09.709964 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:09.709941    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7443-a7f194"]
Nov 01 05:17:09.712097 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:17:09.712068    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2056f6bd_fc21_4909_a585_fe613ab32485.slice/crio-8206d03964f6793d9c3e5290b2403ff6f804557ea873b036031d323958ec0d4a WatchSource:0}: Error finding container 8206d03964f6793d9c3e5290b2403ff6f804557ea873b036031d323958ec0d4a: Status 404 returned error can't find the container with id 8206d03964f6793d9c3e5290b2403ff6f804557ea873b036031d323958ec0d4a
Nov 01 05:17:10.094512 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:10.094470    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7443-a7f194" event={"ID":"2056f6bd-fc21-4909-a585-fe613ab32485","Type":"ContainerStarted","Data":"8206d03964f6793d9c3e5290b2403ff6f804557ea873b036031d323958ec0d4a"}
Nov 01 05:17:13.115898 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:13.115843    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7443-a7f194" event={"ID":"2056f6bd-fc21-4909-a585-fe613ab32485","Type":"ContainerStarted","Data":"d4b448625353e498d1a5a25af2d192984e420bdcfc03ea8a3b21eb5c8df88ea9"}
Nov 01 05:17:13.228408 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:13.228281    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7443-a7f194" podStartSLOduration=2.076005333 podStartE2EDuration="5.228266456s" totalImagesPullingTime="3.152261123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:17:13.227373123 +0000 UTC m=+850.117749099" watchObservedRunningTime="2026-11-01 05:17:13.228266456 +0000 UTC m=+850.118642431"
Nov 01 05:17:14.651187 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:14.651148    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7443-a7f194"]
Nov 01 05:17:14.655139 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:14.655113    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7443-a7f194"]
Nov 01 05:17:14.810741 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:14.810706    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7443-2bc023"]
Nov 01 05:17:14.811156 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:14.811143    4864 cpu_manager.go:414] "containerMap: removing container" podUID="2056f6bd-fc21-4909-a585-fe613ab32485" containerName="test-curl-7443-a7f194"
Nov 01 05:17:14.811195 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:14.811157    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2056f6bd-fc21-4909-a585-fe613ab32485" containerName="test-curl-7443-a7f194"
Nov 01 05:17:14.811267 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:14.811258    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="2056f6bd-fc21-4909-a585-fe613ab32485" containerName="test-curl-7443-a7f194"
Nov 01 05:17:14.814690 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:14.814663    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-2bc023"
Nov 01 05:17:14.870859 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:14.870833    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7443-2bc023"]
Nov 01 05:17:14.921531 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:14.921449    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz4kk\" (UniqueName: \"kubernetes.io/projected/317e0b46-8a7d-40c5-acc5-34384d5d76ab-kube-api-access-gz4kk\") pod \"test-curl-7443-2bc023\" (UID: \"317e0b46-8a7d-40c5-acc5-34384d5d76ab\") " pod="opendatahub/test-curl-7443-2bc023"
Nov 01 05:17:15.022404 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:15.022374    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz4kk\" (UniqueName: \"kubernetes.io/projected/317e0b46-8a7d-40c5-acc5-34384d5d76ab-kube-api-access-gz4kk\") pod \"test-curl-7443-2bc023\" (UID: \"317e0b46-8a7d-40c5-acc5-34384d5d76ab\") " pod="opendatahub/test-curl-7443-2bc023"
Nov 01 05:17:15.082747 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:15.082713    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz4kk\" (UniqueName: \"kubernetes.io/projected/317e0b46-8a7d-40c5-acc5-34384d5d76ab-kube-api-access-gz4kk\") pod \"test-curl-7443-2bc023\" (UID: \"317e0b46-8a7d-40c5-acc5-34384d5d76ab\") " pod="opendatahub/test-curl-7443-2bc023"
Nov 01 05:17:15.125969 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:15.125938    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-2bc023"
Nov 01 05:17:15.128802 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:15.128756    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7443-a7f194" podUID="2056f6bd-fc21-4909-a585-fe613ab32485" containerName="test-curl-7443-a7f194" containerID="cri-o://d4b448625353e498d1a5a25af2d192984e420bdcfc03ea8a3b21eb5c8df88ea9" gracePeriod=2
Nov 01 05:17:15.179576 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:17:15.179527    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7443-a7f194\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="2056f6bd-fc21-4909-a585-fe613ab32485" pod="opendatahub/test-curl-7443-a7f194"
Nov 01 05:17:15.498072 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:15.498038    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7443-2bc023"]
Nov 01 05:17:15.499297 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:17:15.499256    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod317e0b46_8a7d_40c5_acc5_34384d5d76ab.slice/crio-5d0dbcbe963cf4ad68d8f17995edd52b856aa1ed1dce966c91192fc655730fa3 WatchSource:0}: Error finding container 5d0dbcbe963cf4ad68d8f17995edd52b856aa1ed1dce966c91192fc655730fa3: Status 404 returned error can't find the container with id 5d0dbcbe963cf4ad68d8f17995edd52b856aa1ed1dce966c91192fc655730fa3
Nov 01 05:17:16.136997 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:16.136962    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7443-2bc023" event={"ID":"317e0b46-8a7d-40c5-acc5-34384d5d76ab","Type":"ContainerStarted","Data":"b425b58e9bdcf22cad2e4123a3a3e71098b6a4cd5738e6d7bbe7470cd454842c"}
Nov 01 05:17:16.136997 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:16.137004    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7443-2bc023" event={"ID":"317e0b46-8a7d-40c5-acc5-34384d5d76ab","Type":"ContainerStarted","Data":"5d0dbcbe963cf4ad68d8f17995edd52b856aa1ed1dce966c91192fc655730fa3"}
Nov 01 05:17:16.281638 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:16.281570    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7443-2bc023" podStartSLOduration=2.281554813 podStartE2EDuration="2.281554813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:17:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:17:16.281165408 +0000 UTC m=+853.171541384" watchObservedRunningTime="2026-11-01 05:17:16.281554813 +0000 UTC m=+853.171930787"
Nov 01 05:17:17.349075 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:17.349048    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-a7f194"
Nov 01 05:17:17.448552 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:17.448463    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2056f6bd-fc21-4909-a585-fe613ab32485-kube-api-access-prqdp\" (UniqueName: \"kubernetes.io/projected/2056f6bd-fc21-4909-a585-fe613ab32485-kube-api-access-prqdp\") pod \"2056f6bd-fc21-4909-a585-fe613ab32485\" (UID: \"2056f6bd-fc21-4909-a585-fe613ab32485\") "
Nov 01 05:17:17.450795 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:17.450751    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2056f6bd-fc21-4909-a585-fe613ab32485-kube-api-access-prqdp" pod "2056f6bd-fc21-4909-a585-fe613ab32485" (UID: "2056f6bd-fc21-4909-a585-fe613ab32485"). InnerVolumeSpecName "kube-api-access-prqdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:17:17.549907 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:17.549874    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prqdp\" (UniqueName: \"kubernetes.io/projected/2056f6bd-fc21-4909-a585-fe613ab32485-kube-api-access-prqdp\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:17:17.707446 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:17.707357    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:17.726148 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:17.726122    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:17.831842 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:17.831811    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7443-2bc023"]
Nov 01 05:17:17.834958 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:17.834929    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7443-2bc023"]
Nov 01 05:17:18.149880 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:18.149841    4864 generic.go:356] "Generic (PLEG): container finished" podID="2056f6bd-fc21-4909-a585-fe613ab32485" containerID="d4b448625353e498d1a5a25af2d192984e420bdcfc03ea8a3b21eb5c8df88ea9" exitCode=137
Nov 01 05:17:18.150088 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:18.149889    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-a7f194"
Nov 01 05:17:18.150088 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:18.149937    4864 scope.go:122] "RemoveContainer" containerID="d4b448625353e498d1a5a25af2d192984e420bdcfc03ea8a3b21eb5c8df88ea9"
Nov 01 05:17:18.150220 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:18.150113    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7443-2bc023" podUID="317e0b46-8a7d-40c5-acc5-34384d5d76ab" containerName="test-curl-7443-2bc023" containerID="cri-o://b425b58e9bdcf22cad2e4123a3a3e71098b6a4cd5738e6d7bbe7470cd454842c" gracePeriod=2
Nov 01 05:17:18.162239 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:18.162216    4864 scope.go:122] "RemoveContainer" containerID="d4b448625353e498d1a5a25af2d192984e420bdcfc03ea8a3b21eb5c8df88ea9"
Nov 01 05:17:18.162522 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:17:18.162499    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4b448625353e498d1a5a25af2d192984e420bdcfc03ea8a3b21eb5c8df88ea9\": container with ID starting with d4b448625353e498d1a5a25af2d192984e420bdcfc03ea8a3b21eb5c8df88ea9 not found: ID does not exist" containerID="d4b448625353e498d1a5a25af2d192984e420bdcfc03ea8a3b21eb5c8df88ea9"
Nov 01 05:17:18.162587 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:18.162533    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4b448625353e498d1a5a25af2d192984e420bdcfc03ea8a3b21eb5c8df88ea9"} err="failed to get container status \"d4b448625353e498d1a5a25af2d192984e420bdcfc03ea8a3b21eb5c8df88ea9\": rpc error: code = NotFound desc = could not find container \"d4b448625353e498d1a5a25af2d192984e420bdcfc03ea8a3b21eb5c8df88ea9\": container with ID starting with d4b448625353e498d1a5a25af2d192984e420bdcfc03ea8a3b21eb5c8df88ea9 not found: ID does not exist"
Nov 01 05:17:18.164428 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:18.164411    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 05:17:18.200874 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:17:18.200846    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7443-2bc023\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="317e0b46-8a7d-40c5-acc5-34384d5d76ab" pod="opendatahub/test-curl-7443-2bc023"
Nov 01 05:17:18.251396 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:17:18.251367    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7443-2bc023\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="317e0b46-8a7d-40c5-acc5-34384d5d76ab" pod="opendatahub/test-curl-7443-2bc023"
Nov 01 05:17:18.301988 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:17:18.301953    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7443-a7f194\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="2056f6bd-fc21-4909-a585-fe613ab32485" pod="opendatahub/test-curl-7443-a7f194"
Nov 01 05:17:19.247452 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:19.247414    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2056f6bd-fc21-4909-a585-fe613ab32485" path="/var/lib/kubelet/pods/2056f6bd-fc21-4909-a585-fe613ab32485/volumes"
Nov 01 05:17:20.448774 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:20.448743    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-2bc023"
Nov 01 05:17:20.582867 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:20.582830    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/317e0b46-8a7d-40c5-acc5-34384d5d76ab-kube-api-access-gz4kk\" (UniqueName: \"kubernetes.io/projected/317e0b46-8a7d-40c5-acc5-34384d5d76ab-kube-api-access-gz4kk\") pod \"317e0b46-8a7d-40c5-acc5-34384d5d76ab\" (UID: \"317e0b46-8a7d-40c5-acc5-34384d5d76ab\") "
Nov 01 05:17:20.585085 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:20.585058    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/317e0b46-8a7d-40c5-acc5-34384d5d76ab-kube-api-access-gz4kk" pod "317e0b46-8a7d-40c5-acc5-34384d5d76ab" (UID: "317e0b46-8a7d-40c5-acc5-34384d5d76ab"). InnerVolumeSpecName "kube-api-access-gz4kk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:17:20.684034 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:20.683946    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gz4kk\" (UniqueName: \"kubernetes.io/projected/317e0b46-8a7d-40c5-acc5-34384d5d76ab-kube-api-access-gz4kk\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:17:21.173580 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:21.173546    4864 generic.go:356] "Generic (PLEG): container finished" podID="317e0b46-8a7d-40c5-acc5-34384d5d76ab" containerID="b425b58e9bdcf22cad2e4123a3a3e71098b6a4cd5738e6d7bbe7470cd454842c" exitCode=137
Nov 01 05:17:21.173806 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:21.173595    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-2bc023"
Nov 01 05:17:21.173806 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:21.173634    4864 scope.go:122] "RemoveContainer" containerID="b425b58e9bdcf22cad2e4123a3a3e71098b6a4cd5738e6d7bbe7470cd454842c"
Nov 01 05:17:21.183219 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:21.183202    4864 scope.go:122] "RemoveContainer" containerID="b425b58e9bdcf22cad2e4123a3a3e71098b6a4cd5738e6d7bbe7470cd454842c"
Nov 01 05:17:21.183481 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:17:21.183462    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b425b58e9bdcf22cad2e4123a3a3e71098b6a4cd5738e6d7bbe7470cd454842c\": container with ID starting with b425b58e9bdcf22cad2e4123a3a3e71098b6a4cd5738e6d7bbe7470cd454842c not found: ID does not exist" containerID="b425b58e9bdcf22cad2e4123a3a3e71098b6a4cd5738e6d7bbe7470cd454842c"
Nov 01 05:17:21.183522 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:21.183490    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b425b58e9bdcf22cad2e4123a3a3e71098b6a4cd5738e6d7bbe7470cd454842c"} err="failed to get container status \"b425b58e9bdcf22cad2e4123a3a3e71098b6a4cd5738e6d7bbe7470cd454842c\": rpc error: code = NotFound desc = could not find container \"b425b58e9bdcf22cad2e4123a3a3e71098b6a4cd5738e6d7bbe7470cd454842c\": container with ID starting with b425b58e9bdcf22cad2e4123a3a3e71098b6a4cd5738e6d7bbe7470cd454842c not found: ID does not exist"
Nov 01 05:17:21.232778 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:17:21.232739    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7443-2bc023\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="317e0b46-8a7d-40c5-acc5-34384d5d76ab" pod="opendatahub/test-curl-7443-2bc023"
Nov 01 05:17:21.246692 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:21.246657    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="317e0b46-8a7d-40c5-acc5-34384d5d76ab" path="/var/lib/kubelet/pods/317e0b46-8a7d-40c5-acc5-34384d5d76ab/volumes"
Nov 01 05:17:22.524946 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:17:22.524893    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:17:52.536942 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:17:52.536905    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:17:53.627034 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.627007    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"]
Nov 01 05:17:53.627456 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.627443    4864 cpu_manager.go:414] "containerMap: removing container" podUID="317e0b46-8a7d-40c5-acc5-34384d5d76ab" containerName="test-curl-7443-2bc023"
Nov 01 05:17:53.627500 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.627456    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="317e0b46-8a7d-40c5-acc5-34384d5d76ab" containerName="test-curl-7443-2bc023"
Nov 01 05:17:53.627538 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.627523    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="317e0b46-8a7d-40c5-acc5-34384d5d76ab" containerName="test-curl-7443-2bc023"
Nov 01 05:17:53.630921 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.630905    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"
Nov 01 05:17:53.682815 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.682795    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-a12c35-serving-cert\""
Nov 01 05:17:53.711809 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.711755    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22f93dd6-caf5-4352-a81f-98878a3c485f-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw\" (UID: \"22f93dd6-caf5-4352-a81f-98878a3c485f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"
Nov 01 05:17:53.711939 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.711864    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22f93dd6-caf5-4352-a81f-98878a3c485f-maas-api-tls\") pod \"maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw\" (UID: \"22f93dd6-caf5-4352-a81f-98878a3c485f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"
Nov 01 05:17:53.711986 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.711952    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5qrj\" (UniqueName: \"kubernetes.io/projected/22f93dd6-caf5-4352-a81f-98878a3c485f-kube-api-access-f5qrj\") pod \"maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw\" (UID: \"22f93dd6-caf5-4352-a81f-98878a3c485f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"
Nov 01 05:17:53.712021 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.711990    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22f93dd6-caf5-4352-a81f-98878a3c485f-metrics-certs\") pod \"maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw\" (UID: \"22f93dd6-caf5-4352-a81f-98878a3c485f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"
Nov 01 05:17:53.735313 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.735283    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"]
Nov 01 05:17:53.737485 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.737466    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"]
Nov 01 05:17:53.741156 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.741140    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:17:53.793622 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.793593    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-2ea125\"/\"kube-root-ca.crt\""
Nov 01 05:17:53.794815 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.794793    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-2ea125\"/\"openshift-service-ca.crt\""
Nov 01 05:17:53.795098 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.795070    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-2ea125\"/\"default-dockercfg-vqq8z\""
Nov 01 05:17:53.795219 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.795133    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-2ea125\"/\"test-model-2ea125-kserve-self-signed-certs\""
Nov 01 05:17:53.806221 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.806197    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"]
Nov 01 05:17:53.813376 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.813353    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-dshm\") pod \"test-model-2ea125-kserve-7f7f6dc9bc-46snp\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") " pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:17:53.813487 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.813387    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22f93dd6-caf5-4352-a81f-98878a3c485f-metrics-certs\") pod \"maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw\" (UID: \"22f93dd6-caf5-4352-a81f-98878a3c485f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"
Nov 01 05:17:53.813555 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.813481    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22f93dd6-caf5-4352-a81f-98878a3c485f-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw\" (UID: \"22f93dd6-caf5-4352-a81f-98878a3c485f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"
Nov 01 05:17:53.813555 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.813540    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22f93dd6-caf5-4352-a81f-98878a3c485f-maas-api-tls\") pod \"maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw\" (UID: \"22f93dd6-caf5-4352-a81f-98878a3c485f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"
Nov 01 05:17:53.813677 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:17:53.813629    4864 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-serving-cert: secret "maas-api-e2e-extmod-a-a12c35-serving-cert" not found
Nov 01 05:17:53.813677 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.813663    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-home\") pod \"test-model-2ea125-kserve-7f7f6dc9bc-46snp\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") " pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:17:53.813799 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:17:53.813683    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22f93dd6-caf5-4352-a81f-98878a3c485f-maas-api-tls podName:22f93dd6-caf5-4352-a81f-98878a3c485f nodeName:}" failed. No retries permitted until 2026-11-01 05:17:54.313664507 +0000 UTC m=+891.204040462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/22f93dd6-caf5-4352-a81f-98878a3c485f-maas-api-tls") pod "maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw" (UID: "22f93dd6-caf5-4352-a81f-98878a3c485f") : secret "maas-api-e2e-extmod-a-a12c35-serving-cert" not found
Nov 01 05:17:53.813799 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.813769    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-tmp-dir\") pod \"test-model-2ea125-kserve-7f7f6dc9bc-46snp\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") " pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:17:53.813919 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.813833    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-model-cache\") pod \"test-model-2ea125-kserve-7f7f6dc9bc-46snp\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") " pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:17:53.813919 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.813867    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp4g5\" (UniqueName: \"kubernetes.io/projected/52b5e868-dd1f-490d-821e-b178350e7ea2-kube-api-access-mp4g5\") pod \"test-model-2ea125-kserve-7f7f6dc9bc-46snp\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") " pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:17:53.814029 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.813954    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5qrj\" (UniqueName: \"kubernetes.io/projected/22f93dd6-caf5-4352-a81f-98878a3c485f-kube-api-access-f5qrj\") pod \"maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw\" (UID: \"22f93dd6-caf5-4352-a81f-98878a3c485f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"
Nov 01 05:17:53.814029 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.813971    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52b5e868-dd1f-490d-821e-b178350e7ea2-tls-certs\") pod \"test-model-2ea125-kserve-7f7f6dc9bc-46snp\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") " pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:17:53.814614 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.814589    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22f93dd6-caf5-4352-a81f-98878a3c485f-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw\" (UID: \"22f93dd6-caf5-4352-a81f-98878a3c485f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"
Nov 01 05:17:53.817013 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.816989    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22f93dd6-caf5-4352-a81f-98878a3c485f-metrics-certs\") pod \"maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw\" (UID: \"22f93dd6-caf5-4352-a81f-98878a3c485f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"
Nov 01 05:17:53.874773 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.874746    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5qrj\" (UniqueName: \"kubernetes.io/projected/22f93dd6-caf5-4352-a81f-98878a3c485f-kube-api-access-f5qrj\") pod \"maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw\" (UID: \"22f93dd6-caf5-4352-a81f-98878a3c485f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"
Nov 01 05:17:53.915248 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.915157    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-home\") pod \"test-model-2ea125-kserve-7f7f6dc9bc-46snp\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") " pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:17:53.915248 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.915208    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-tmp-dir\") pod \"test-model-2ea125-kserve-7f7f6dc9bc-46snp\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") " pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:17:53.915248 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.915236    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-model-cache\") pod \"test-model-2ea125-kserve-7f7f6dc9bc-46snp\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") " pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:17:53.915511 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.915263    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp4g5\" (UniqueName: \"kubernetes.io/projected/52b5e868-dd1f-490d-821e-b178350e7ea2-kube-api-access-mp4g5\") pod \"test-model-2ea125-kserve-7f7f6dc9bc-46snp\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") " pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:17:53.915511 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.915289    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52b5e868-dd1f-490d-821e-b178350e7ea2-tls-certs\") pod \"test-model-2ea125-kserve-7f7f6dc9bc-46snp\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") " pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:17:53.915511 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.915306    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-dshm\") pod \"test-model-2ea125-kserve-7f7f6dc9bc-46snp\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") " pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:17:53.918609 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.916030    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-model-cache\") pod \"test-model-2ea125-kserve-7f7f6dc9bc-46snp\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") " pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:17:53.918609 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.916068    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-home\") pod \"test-model-2ea125-kserve-7f7f6dc9bc-46snp\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") " pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:17:53.918609 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.916091    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-tmp-dir\") pod \"test-model-2ea125-kserve-7f7f6dc9bc-46snp\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") " pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:17:53.923355 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.920116    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52b5e868-dd1f-490d-821e-b178350e7ea2-tls-certs\") pod \"test-model-2ea125-kserve-7f7f6dc9bc-46snp\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") " pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:17:53.925734 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.925712    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-dshm\") pod \"test-model-2ea125-kserve-7f7f6dc9bc-46snp\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") " pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:17:53.977211 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:53.977179    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp4g5\" (UniqueName: \"kubernetes.io/projected/52b5e868-dd1f-490d-821e-b178350e7ea2-kube-api-access-mp4g5\") pod \"test-model-2ea125-kserve-7f7f6dc9bc-46snp\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") " pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:17:54.053118 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:54.053091    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:17:54.319472 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:54.319441    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22f93dd6-caf5-4352-a81f-98878a3c485f-maas-api-tls\") pod \"maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw\" (UID: \"22f93dd6-caf5-4352-a81f-98878a3c485f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"
Nov 01 05:17:54.321977 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:54.321948    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22f93dd6-caf5-4352-a81f-98878a3c485f-maas-api-tls\") pod \"maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw\" (UID: \"22f93dd6-caf5-4352-a81f-98878a3c485f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"
Nov 01 05:17:54.436752 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:54.436728    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"]
Nov 01 05:17:54.438891 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:17:54.438866    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52b5e868_dd1f_490d_821e_b178350e7ea2.slice/crio-34e81d25b3a0e830839e88ed501ae4f715b987989fe36ac11cbdca1b477eecf5 WatchSource:0}: Error finding container 34e81d25b3a0e830839e88ed501ae4f715b987989fe36ac11cbdca1b477eecf5: Status 404 returned error can't find the container with id 34e81d25b3a0e830839e88ed501ae4f715b987989fe36ac11cbdca1b477eecf5
Nov 01 05:17:54.543927 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:54.543892    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"
Nov 01 05:17:54.929649 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:54.929627    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"]
Nov 01 05:17:54.931261 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:17:54.931227    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22f93dd6_caf5_4352_a81f_98878a3c485f.slice/crio-c620e5907fcb569506c0ef0ab515290f3d975ddc78b22c16f870902700f3e01e WatchSource:0}: Error finding container c620e5907fcb569506c0ef0ab515290f3d975ddc78b22c16f870902700f3e01e: Status 404 returned error can't find the container with id c620e5907fcb569506c0ef0ab515290f3d975ddc78b22c16f870902700f3e01e
Nov 01 05:17:55.394778 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:55.394745    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp" event={"ID":"52b5e868-dd1f-490d-821e-b178350e7ea2","Type":"ContainerStarted","Data":"c827c2e0f7afa3e3b5355099527b5a5a8fad186d1646c244c0d27f7c52da6db7"}
Nov 01 05:17:55.394956 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:55.394816    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp" event={"ID":"52b5e868-dd1f-490d-821e-b178350e7ea2","Type":"ContainerStarted","Data":"34e81d25b3a0e830839e88ed501ae4f715b987989fe36ac11cbdca1b477eecf5"}
Nov 01 05:17:55.396067 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:55.396041    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw" event={"ID":"22f93dd6-caf5-4352-a81f-98878a3c485f","Type":"ContainerStarted","Data":"c620e5907fcb569506c0ef0ab515290f3d975ddc78b22c16f870902700f3e01e"}
Nov 01 05:17:55.513161 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:55.513115    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp" podStartSLOduration=2.513101927 podStartE2EDuration="2.513101927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:17:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:17:55.510347521 +0000 UTC m=+892.400723501" watchObservedRunningTime="2026-11-01 05:17:55.513101927 +0000 UTC m=+892.403477947"
Nov 01 05:17:56.053459 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:56.053430    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:17:56.402845 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:56.402747    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw" event={"ID":"22f93dd6-caf5-4352-a81f-98878a3c485f","Type":"ContainerStarted","Data":"afe00036852b503cb69bae64c9c52c0006dddb37ee3877cdeb47063ef419d49b"}
Nov 01 05:17:56.402845 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:56.402832    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"
Nov 01 05:17:56.521871 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:17:56.521821    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw" podStartSLOduration=2.740147705 podStartE2EDuration="3.521804541s" totalImagesPullingTime="781.656836ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:17:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:17:56.520208417 +0000 UTC m=+893.410584386" watchObservedRunningTime="2026-11-01 05:17:56.521804541 +0000 UTC m=+893.412180517"
Nov 01 05:18:01.263368 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.263338    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"]
Nov 01 05:18:01.267387 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.267357    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:01.325448 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.325418    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"]
Nov 01 05:18:01.371279 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.371230    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-af8051\"/\"kube-root-ca.crt\""
Nov 01 05:18:01.371450 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.371268    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-af8051\"/\"openshift-service-ca.crt\""
Nov 01 05:18:01.371515 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.371455    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-af8051\"/\"default-dockercfg-mpzqx\""
Nov 01 05:18:01.371704 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.371689    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-af8051\"/\"test-model-af8051-kserve-self-signed-certs\""
Nov 01 05:18:01.389099 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.389078    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-tls-certs\") pod \"test-model-af8051-kserve-576468f797-8zpxh\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") " pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:01.389167 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.389119    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-home\") pod \"test-model-af8051-kserve-576468f797-8zpxh\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") " pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:01.389167 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.389139    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-tmp-dir\") pod \"test-model-af8051-kserve-576468f797-8zpxh\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") " pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:01.389253 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.389240    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxzln\" (UniqueName: \"kubernetes.io/projected/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-kube-api-access-pxzln\") pod \"test-model-af8051-kserve-576468f797-8zpxh\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") " pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:01.389287 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.389265    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-dshm\") pod \"test-model-af8051-kserve-576468f797-8zpxh\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") " pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:01.389336 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.389286    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-model-cache\") pod \"test-model-af8051-kserve-576468f797-8zpxh\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") " pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:01.490152 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.490112    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-tls-certs\") pod \"test-model-af8051-kserve-576468f797-8zpxh\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") " pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:01.490386 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.490180    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-home\") pod \"test-model-af8051-kserve-576468f797-8zpxh\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") " pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:01.490386 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.490214    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-tmp-dir\") pod \"test-model-af8051-kserve-576468f797-8zpxh\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") " pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:01.490386 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.490263    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxzln\" (UniqueName: \"kubernetes.io/projected/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-kube-api-access-pxzln\") pod \"test-model-af8051-kserve-576468f797-8zpxh\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") " pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:01.490386 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.490287    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-dshm\") pod \"test-model-af8051-kserve-576468f797-8zpxh\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") " pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:01.490386 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.490314    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-model-cache\") pod \"test-model-af8051-kserve-576468f797-8zpxh\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") " pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:01.490717 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.490677    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-tmp-dir\") pod \"test-model-af8051-kserve-576468f797-8zpxh\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") " pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:01.490880 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.490775    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-model-cache\") pod \"test-model-af8051-kserve-576468f797-8zpxh\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") " pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:01.491268 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.491246    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-home\") pod \"test-model-af8051-kserve-576468f797-8zpxh\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") " pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:01.492761 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.492733    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-dshm\") pod \"test-model-af8051-kserve-576468f797-8zpxh\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") " pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:01.493034 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.493018    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-tls-certs\") pod \"test-model-af8051-kserve-576468f797-8zpxh\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") " pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:01.552040 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.551946    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxzln\" (UniqueName: \"kubernetes.io/projected/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-kube-api-access-pxzln\") pod \"test-model-af8051-kserve-576468f797-8zpxh\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") " pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:01.579513 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.579472    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:01.969891 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:01.969868    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"]
Nov 01 05:18:01.972037 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:18:01.972007    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb13c463a_f27e_49a8_af7f_ee945dc1b9b8.slice/crio-3cd55acfa7acdae4bcc46f87bca9b11205cda4095fd39f5bb717316c21560bfa WatchSource:0}: Error finding container 3cd55acfa7acdae4bcc46f87bca9b11205cda4095fd39f5bb717316c21560bfa: Status 404 returned error can't find the container with id 3cd55acfa7acdae4bcc46f87bca9b11205cda4095fd39f5bb717316c21560bfa
Nov 01 05:18:02.416951 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:02.416922    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"
Nov 01 05:18:02.441001 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:02.440964    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh" event={"ID":"b13c463a-f27e-49a8-af7f-ee945dc1b9b8","Type":"ContainerStarted","Data":"44e4fabb17f10f29dd54aef82c3cda3ecf64eee6a6dbdf8ca87a372e88e697c3"}
Nov 01 05:18:02.441001 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:02.441006    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh" event={"ID":"b13c463a-f27e-49a8-af7f-ee945dc1b9b8","Type":"ContainerStarted","Data":"3cd55acfa7acdae4bcc46f87bca9b11205cda4095fd39f5bb717316c21560bfa"}
Nov 01 05:18:02.580145 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:02.580111    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:02.652025 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:02.651974    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh" podStartSLOduration=1.6519557360000001 podStartE2EDuration="1.651955736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:18:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:18:02.650243741 +0000 UTC m=+899.540619741" watchObservedRunningTime="2026-11-01 05:18:02.651955736 +0000 UTC m=+899.542331715"
Nov 01 05:18:03.568653 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:03.568619    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:18:03.573380 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:03.573346    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:18:03.577470 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:03.577450    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:18:03.581581 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:03.581563    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:18:04.054267 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:04.054236    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:18:04.067155 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:04.067124    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:18:04.464042 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:04.463952    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:18:11.580326 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:11.580288    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:11.593283 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:11.593255    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:12.521627 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:18:12.521594    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:18:22.526480 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:18:22.526443    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:18:52.525346 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:18:52.525262    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:19:01.459418 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.459370    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"]
Nov 01 05:19:01.464258 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.464232    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:01.565972 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.565940    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-a48099\"/\"distinct-2-w4-a48099-kserve-self-signed-certs\""
Nov 01 05:19:01.570615 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.570583    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"]
Nov 01 05:19:01.578539 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.578506    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-dshm\") pod \"distinct-2-w4-a48099-kserve-8649ddd5-w7cz8\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") " pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:01.578682 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.578561    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/071c5c98-548c-439b-b8f6-3b07c70df932-tls-certs\") pod \"distinct-2-w4-a48099-kserve-8649ddd5-w7cz8\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") " pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:01.578746 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.578730    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-tmp-dir\") pod \"distinct-2-w4-a48099-kserve-8649ddd5-w7cz8\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") " pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:01.578960 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.578939    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-home\") pod \"distinct-2-w4-a48099-kserve-8649ddd5-w7cz8\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") " pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:01.579093 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.578981    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-449jj\" (UniqueName: \"kubernetes.io/projected/071c5c98-548c-439b-b8f6-3b07c70df932-kube-api-access-449jj\") pod \"distinct-2-w4-a48099-kserve-8649ddd5-w7cz8\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") " pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:01.579093 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.579026    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-model-cache\") pod \"distinct-2-w4-a48099-kserve-8649ddd5-w7cz8\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") " pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:01.680717 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.680672    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-model-cache\") pod \"distinct-2-w4-a48099-kserve-8649ddd5-w7cz8\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") " pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:01.680958 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.680804    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-dshm\") pod \"distinct-2-w4-a48099-kserve-8649ddd5-w7cz8\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") " pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:01.680958 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.680854    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/071c5c98-548c-439b-b8f6-3b07c70df932-tls-certs\") pod \"distinct-2-w4-a48099-kserve-8649ddd5-w7cz8\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") " pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:01.680958 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.680896    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-tmp-dir\") pod \"distinct-2-w4-a48099-kserve-8649ddd5-w7cz8\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") " pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:01.681134 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.681030    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-home\") pod \"distinct-2-w4-a48099-kserve-8649ddd5-w7cz8\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") " pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:01.681255 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.681215    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-model-cache\") pod \"distinct-2-w4-a48099-kserve-8649ddd5-w7cz8\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") " pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:01.681342 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.681319    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-tmp-dir\") pod \"distinct-2-w4-a48099-kserve-8649ddd5-w7cz8\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") " pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:01.681452 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.681428    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-449jj\" (UniqueName: \"kubernetes.io/projected/071c5c98-548c-439b-b8f6-3b07c70df932-kube-api-access-449jj\") pod \"distinct-2-w4-a48099-kserve-8649ddd5-w7cz8\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") " pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:01.681845 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.681820    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-home\") pod \"distinct-2-w4-a48099-kserve-8649ddd5-w7cz8\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") " pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:01.683684 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.683654    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-dshm\") pod \"distinct-2-w4-a48099-kserve-8649ddd5-w7cz8\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") " pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:01.683838 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.683804    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/071c5c98-548c-439b-b8f6-3b07c70df932-tls-certs\") pod \"distinct-2-w4-a48099-kserve-8649ddd5-w7cz8\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") " pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:01.743348 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.743270    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-449jj\" (UniqueName: \"kubernetes.io/projected/071c5c98-548c-439b-b8f6-3b07c70df932-kube-api-access-449jj\") pod \"distinct-2-w4-a48099-kserve-8649ddd5-w7cz8\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") " pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:01.779504 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:01.779458    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:02.188296 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:02.188268    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"]
Nov 01 05:19:02.190351 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:19:02.190316    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod071c5c98_548c_439b_b8f6_3b07c70df932.slice/crio-e92d7fc255b0a1a837b5043814ec5bc161b235f3e7320878aca7e9042f32f0ce WatchSource:0}: Error finding container e92d7fc255b0a1a837b5043814ec5bc161b235f3e7320878aca7e9042f32f0ce: Status 404 returned error can't find the container with id e92d7fc255b0a1a837b5043814ec5bc161b235f3e7320878aca7e9042f32f0ce
Nov 01 05:19:02.835178 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:02.835142    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8" event={"ID":"071c5c98-548c-439b-b8f6-3b07c70df932","Type":"ContainerStarted","Data":"4862b1f9f66e9423b2516a3e1a9f5edfa6857d83c9dc9a59cf40c8d326e25fa0"}
Nov 01 05:19:02.835178 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:02.835185    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8" event={"ID":"071c5c98-548c-439b-b8f6-3b07c70df932","Type":"ContainerStarted","Data":"e92d7fc255b0a1a837b5043814ec5bc161b235f3e7320878aca7e9042f32f0ce"}
Nov 01 05:19:02.952534 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:02.952477    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8" podStartSLOduration=1.952462058 podStartE2EDuration="1.952462058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:19:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:19:02.9503614 +0000 UTC m=+959.840737376" watchObservedRunningTime="2026-11-01 05:19:02.952462058 +0000 UTC m=+959.842838034"
Nov 01 05:19:03.779969 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:03.779931    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:11.780531 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:11.780496    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:11.793649 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:11.793627    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:11.906352 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:11.906324    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 05:19:22.524883 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:19:22.524845    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:19:39.188260 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.188216    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"]
Nov 01 05:19:39.192962 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.192936    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.245423 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.245385    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-3d70ff-openshift-default-dockercfg-x8xrx\""
Nov 01 05:19:39.299918 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.299876    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"]
Nov 01 05:19:39.367461 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.367410    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-workload-certs\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.367660 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.367466    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-istio-data\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.367660 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.367559    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-credential-socket\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.367660 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.367595    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0acea0d8-ee39-4724-8c86-4396234a5654-istio-podinfo\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.367660 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.367623    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0acea0d8-ee39-4724-8c86-4396234a5654-istiod-ca-cert\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.367929 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.367700    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0acea0d8-ee39-4724-8c86-4396234a5654-istio-token\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.367929 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.367754    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrzbz\" (UniqueName: \"kubernetes.io/projected/0acea0d8-ee39-4724-8c86-4396234a5654-kube-api-access-xrzbz\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.367929 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.367803    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-workload-socket\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.367929 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.367849    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-istio-envoy\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.469261 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.469174    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-istio-envoy\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.469427 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.469292    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-workload-certs\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.469427 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.469345    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-istio-data\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.469427 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.469378    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-credential-socket\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.469427 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.469401    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0acea0d8-ee39-4724-8c86-4396234a5654-istio-podinfo\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.469735 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.469430    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0acea0d8-ee39-4724-8c86-4396234a5654-istiod-ca-cert\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.469735 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.469504    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0acea0d8-ee39-4724-8c86-4396234a5654-istio-token\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.469735 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.469561    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrzbz\" (UniqueName: \"kubernetes.io/projected/0acea0d8-ee39-4724-8c86-4396234a5654-kube-api-access-xrzbz\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.469735 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.469593    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-workload-socket\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.470136 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.469885    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-istio-data\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.470136 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.469959    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-workload-socket\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.470929 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.470572    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-credential-socket\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.471039 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.471015    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-workload-certs\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.473117 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.473089    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0acea0d8-ee39-4724-8c86-4396234a5654-istio-podinfo\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.473511 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.473479    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0acea0d8-ee39-4724-8c86-4396234a5654-istiod-ca-cert\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.473675 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.473656    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-istio-envoy\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.530903 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.530870    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0acea0d8-ee39-4724-8c86-4396234a5654-istio-token\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.531440 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.531416    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrzbz\" (UniqueName: \"kubernetes.io/projected/0acea0d8-ee39-4724-8c86-4396234a5654-kube-api-access-xrzbz\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:39.806750 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:39.806720    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:40.238038 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:40.238013    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"]
Nov 01 05:19:40.238631 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:19:40.238607    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0acea0d8_ee39_4724_8c86_4396234a5654.slice/crio-d9175050e1b7b27a90f0e559f21a690900d545c857207cb23c1b23ca03254224 WatchSource:0}: Error finding container d9175050e1b7b27a90f0e559f21a690900d545c857207cb23c1b23ca03254224: Status 404 returned error can't find the container with id d9175050e1b7b27a90f0e559f21a690900d545c857207cb23c1b23ca03254224
Nov 01 05:19:40.240883 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:40.240847    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:19:40.240994 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:40.240925    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:19:40.240994 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:40.240968    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:19:41.089978 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:41.089942    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8" event={"ID":"0acea0d8-ee39-4724-8c86-4396234a5654","Type":"ContainerStarted","Data":"f9f9c0e58476f13f6c249b5f2a3f1cf0ef955149b4d887d31a93e7eafb507960"}
Nov 01 05:19:41.090207 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:41.090005    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8" event={"ID":"0acea0d8-ee39-4724-8c86-4396234a5654","Type":"ContainerStarted","Data":"d9175050e1b7b27a90f0e559f21a690900d545c857207cb23c1b23ca03254224"}
Nov 01 05:19:41.211042 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:41.210992    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8" podStartSLOduration=2.2109791850000002 podStartE2EDuration="2.210979185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:19:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:19:41.209095072 +0000 UTC m=+998.099471049" watchObservedRunningTime="2026-11-01 05:19:41.210979185 +0000 UTC m=+998.101355168"
Nov 01 05:19:41.807448 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:41.807414    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:41.809211 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:41.809183    4864 patch_prober.go:29] interesting pod/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 01 05:19:41.809328 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:41.809241    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8" podUID="0acea0d8-ee39-4724-8c86-4396234a5654" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 01 05:19:42.807361 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:42.807293    4864 patch_prober.go:29] interesting pod/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 01 05:19:42.807539 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:42.807354    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8" podUID="0acea0d8-ee39-4724-8c86-4396234a5654" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 01 05:19:43.807301 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:43.807271    4864 patch_prober.go:29] interesting pod/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 01 05:19:43.807474 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:43.807327    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8" podUID="0acea0d8-ee39-4724-8c86-4396234a5654" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 01 05:19:44.807308 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:44.807265    4864 patch_prober.go:29] interesting pod/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 01 05:19:44.807893 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:44.807325    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8" podUID="0acea0d8-ee39-4724-8c86-4396234a5654" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 01 05:19:45.807876 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:45.807826    4864 patch_prober.go:29] interesting pod/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 01 05:19:45.808419 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:45.807898    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8" podUID="0acea0d8-ee39-4724-8c86-4396234a5654" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 01 05:19:47.074010 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:47.073973    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:47.074756 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:47.074737    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:47.075590 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:19:47.075567    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:19:52.524184 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:19:52.524135    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:20:06.862425 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:06.862386    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"]
Nov 01 05:20:06.867237 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:06.867211    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:06.962453 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:06.962399    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgnrz\" (UniqueName: \"kubernetes.io/projected/d5d265aa-c0b7-4df3-a8ca-238c1e002561-kube-api-access-tgnrz\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:06.962664 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:06.962547    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-credential-socket\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:06.962738 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:06.962670    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-podinfo\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:06.962839 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:06.962818    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-data\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:06.962935 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:06.962918    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-envoy\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:06.963020 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:06.962989    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-workload-certs\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:06.963156 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:06.963072    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istiod-ca-cert\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:06.963156 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:06.963113    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-workload-socket\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:06.963156 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:06.963140    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-token\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:06.974113 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:06.974083    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"]
Nov 01 05:20:07.022247 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.022221    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-7d1ec3-openshift-default-dockercfg-thhxl\""
Nov 01 05:20:07.064160 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.064120    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-credential-socket\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:07.064358 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.064172    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-podinfo\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:07.064358 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.064219    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-data\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:07.064358 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.064266    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-envoy\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:07.064358 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.064316    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-workload-certs\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:07.064358 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.064339    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istiod-ca-cert\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:07.064601 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.064364    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-workload-socket\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:07.064601 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.064388    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-token\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:07.064601 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.064418    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgnrz\" (UniqueName: \"kubernetes.io/projected/d5d265aa-c0b7-4df3-a8ca-238c1e002561-kube-api-access-tgnrz\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:07.064601 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.064585    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-data\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:07.065244 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.064585    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-credential-socket\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:07.065438 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.065203    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-workload-certs\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:07.065438 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.065234    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-workload-socket\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:07.065622 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.065477    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istiod-ca-cert\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:07.067426 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.067405    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-podinfo\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:07.067656 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.067638    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-envoy\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:07.128434 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.128358    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgnrz\" (UniqueName: \"kubernetes.io/projected/d5d265aa-c0b7-4df3-a8ca-238c1e002561-kube-api-access-tgnrz\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:07.128434 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.128361    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-token\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:07.187819 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.185932    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:07.643110 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:20:07.643062    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d265aa_c0b7_4df3_a8ca_238c1e002561.slice/crio-43afabb98eaa8a5dda8cf180fb706787e7d352145019669ec8bbc2a2e5ec192e WatchSource:0}: Error finding container 43afabb98eaa8a5dda8cf180fb706787e7d352145019669ec8bbc2a2e5ec192e: Status 404 returned error can't find the container with id 43afabb98eaa8a5dda8cf180fb706787e7d352145019669ec8bbc2a2e5ec192e
Nov 01 05:20:07.644888 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.644860    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"]
Nov 01 05:20:07.646261 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.646230    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:20:07.646376 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.646304    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:20:07.646376 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:07.646347    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:20:08.266479 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:08.266440    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn" event={"ID":"d5d265aa-c0b7-4df3-a8ca-238c1e002561","Type":"ContainerStarted","Data":"8c0d51bb913a6154f95c9f2eb34efe901b99431326b1262163de6d6453e0c82e"}
Nov 01 05:20:08.266960 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:08.266491    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn" event={"ID":"d5d265aa-c0b7-4df3-a8ca-238c1e002561","Type":"ContainerStarted","Data":"43afabb98eaa8a5dda8cf180fb706787e7d352145019669ec8bbc2a2e5ec192e"}
Nov 01 05:20:08.392731 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:08.392675    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn" podStartSLOduration=2.3926616640000002 podStartE2EDuration="2.392661664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:20:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:20:08.390121112 +0000 UTC m=+1025.280497089" watchObservedRunningTime="2026-11-01 05:20:08.392661664 +0000 UTC m=+1025.283037641"
Nov 01 05:20:09.188410 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:09.188100    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:09.190271 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:09.190241    4864 patch_prober.go:29] interesting pod/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 01 05:20:09.190450 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:09.190291    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn" podUID="d5d265aa-c0b7-4df3-a8ca-238c1e002561" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 01 05:20:10.188383 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:10.188348    4864 patch_prober.go:29] interesting pod/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 01 05:20:10.188918 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:10.188410    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn" podUID="d5d265aa-c0b7-4df3-a8ca-238c1e002561" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 01 05:20:11.188149 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:11.188097    4864 patch_prober.go:29] interesting pod/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 01 05:20:11.188337 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:11.188170    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn" podUID="d5d265aa-c0b7-4df3-a8ca-238c1e002561" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 01 05:20:12.189219 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:12.189135    4864 patch_prober.go:29] interesting pod/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 01 05:20:12.189219 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:12.189205    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn" podUID="d5d265aa-c0b7-4df3-a8ca-238c1e002561" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 01 05:20:13.188211 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:13.188168    4864 patch_prober.go:29] interesting pod/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 01 05:20:13.188462 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:13.188253    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn" podUID="d5d265aa-c0b7-4df3-a8ca-238c1e002561" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 01 05:20:14.188225 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:14.188165    4864 patch_prober.go:29] interesting pod/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 01 05:20:14.188772 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:14.188249    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn" podUID="d5d265aa-c0b7-4df3-a8ca-238c1e002561" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 01 05:20:15.196685 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:15.196651    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:15.197502 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:15.197476    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:15.198487 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:15.198468    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 05:20:22.526695 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:20:22.526650    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:20:38.574584 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.574540    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"]
Nov 01 05:20:38.578509 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.578493    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.631395 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.631356    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-a48099-openshift-default-dockercfg-8mhxp\""
Nov 01 05:20:38.641513 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.641481    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"]
Nov 01 05:20:38.710345 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.710313    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-workload-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.710345 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.710347    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-token\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.710552 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.710460    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-envoy\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.710552 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.710494    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a25440ad-796b-4ef8-bf27-18ef58ca955a-istiod-ca-cert\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.710626 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.710556    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt4zw\" (UniqueName: \"kubernetes.io/projected/a25440ad-796b-4ef8-bf27-18ef58ca955a-kube-api-access-bt4zw\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.710626 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.710607    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-workload-certs\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.710695 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.710653    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-data\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.710760 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.710747    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-podinfo\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.710825 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.710780    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-credential-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.812337 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.812298    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-workload-certs\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.812543 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.812357    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-data\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.812543 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.812396    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-podinfo\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.812543 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.812421    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-credential-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.812543 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.812469    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-workload-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.812543 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.812485    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-token\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.812543 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.812524    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-envoy\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.812964 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.812625    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a25440ad-796b-4ef8-bf27-18ef58ca955a-istiod-ca-cert\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.812964 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.812681    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt4zw\" (UniqueName: \"kubernetes.io/projected/a25440ad-796b-4ef8-bf27-18ef58ca955a-kube-api-access-bt4zw\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.812964 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.812835    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-workload-certs\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.812964 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.812861    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-data\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.813118 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.812962    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-workload-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.813118 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.812982    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-credential-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.813453 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.813393    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a25440ad-796b-4ef8-bf27-18ef58ca955a-istiod-ca-cert\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.815094 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.815059    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-envoy\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.815346 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.815328    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-podinfo\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.874125 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.874054    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-token\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.874259 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.874194    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt4zw\" (UniqueName: \"kubernetes.io/projected/a25440ad-796b-4ef8-bf27-18ef58ca955a-kube-api-access-bt4zw\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-wl8rv\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:38.892050 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:38.892021    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:39.329278 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:39.329239    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"]
Nov 01 05:20:39.330777 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:20:39.330739    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25440ad_796b_4ef8_bf27_18ef58ca955a.slice/crio-a4027ce001f913ce8a1b1ad6287600d5cdf9ee6c309901b22321185f3bc4ea6e WatchSource:0}: Error finding container a4027ce001f913ce8a1b1ad6287600d5cdf9ee6c309901b22321185f3bc4ea6e: Status 404 returned error can't find the container with id a4027ce001f913ce8a1b1ad6287600d5cdf9ee6c309901b22321185f3bc4ea6e
Nov 01 05:20:39.333605 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:39.333571    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:20:39.333719 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:39.333638    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:20:39.333719 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:39.333665    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:20:39.509110 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:39.509072    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv" event={"ID":"a25440ad-796b-4ef8-bf27-18ef58ca955a","Type":"ContainerStarted","Data":"8eed477bd81d2f951d6489cbadbb9012104989091f7e99fc4116964b831808ca"}
Nov 01 05:20:39.509301 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:39.509134    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv" event={"ID":"a25440ad-796b-4ef8-bf27-18ef58ca955a","Type":"ContainerStarted","Data":"a4027ce001f913ce8a1b1ad6287600d5cdf9ee6c309901b22321185f3bc4ea6e"}
Nov 01 05:20:39.630992 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:39.630890    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv" podStartSLOduration=1.630876852 podStartE2EDuration="1.630876852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:20:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:20:39.628986032 +0000 UTC m=+1056.519362011" watchObservedRunningTime="2026-11-01 05:20:39.630876852 +0000 UTC m=+1056.521252825"
Nov 01 05:20:39.892405 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:39.892334    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:40.893910 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:40.893874    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 01 05:20:40.894378 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:40.893931    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv" podUID="a25440ad-796b-4ef8-bf27-18ef58ca955a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 01 05:20:41.892472 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:41.892436    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 01 05:20:41.892740 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:41.892491    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv" podUID="a25440ad-796b-4ef8-bf27-18ef58ca955a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 01 05:20:42.893347 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:42.893313    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 01 05:20:42.893776 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:42.893366    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv" podUID="a25440ad-796b-4ef8-bf27-18ef58ca955a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 01 05:20:43.892624 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:43.892588    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 01 05:20:43.892959 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:43.892930    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv" podUID="a25440ad-796b-4ef8-bf27-18ef58ca955a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 01 05:20:44.893190 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:44.893152    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 01 05:20:44.893630 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:44.893219    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv" podUID="a25440ad-796b-4ef8-bf27-18ef58ca955a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 01 05:20:45.302101 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.302012    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"]
Nov 01 05:20:45.308719 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.308516    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:45.364764 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.364733    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"]
Nov 01 05:20:45.409713 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.409675    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d94a1258-dbb1-44b5-bf57-1b44b66fe286-tls-certs\") pod \"distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") " pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:45.409933 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.409743    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-home\") pod \"distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") " pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:45.409933 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.409863    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-dshm\") pod \"distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") " pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:45.409933 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.409891    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-tmp-dir\") pod \"distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") " pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:45.410069 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.409958    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqb7r\" (UniqueName: \"kubernetes.io/projected/d94a1258-dbb1-44b5-bf57-1b44b66fe286-kube-api-access-dqb7r\") pod \"distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") " pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:45.410117 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.410072    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-model-cache\") pod \"distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") " pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:45.411044 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.411025    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-3d70ff\"/\"distinct-w2-3d70ff-kserve-self-signed-certs\""
Nov 01 05:20:45.513481 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.513444    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-model-cache\") pod \"distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") " pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:45.513663 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.513566    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d94a1258-dbb1-44b5-bf57-1b44b66fe286-tls-certs\") pod \"distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") " pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:45.513663 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.513610    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-home\") pod \"distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") " pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:45.513663 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.513638    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-dshm\") pod \"distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") " pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:45.513831 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.513663    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-tmp-dir\") pod \"distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") " pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:45.513831 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.513705    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqb7r\" (UniqueName: \"kubernetes.io/projected/d94a1258-dbb1-44b5-bf57-1b44b66fe286-kube-api-access-dqb7r\") pod \"distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") " pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:45.513986 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.513962    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-model-cache\") pod \"distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") " pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:45.514229 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.514210    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-home\") pod \"distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") " pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:45.514616 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.514585    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-tmp-dir\") pod \"distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") " pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:45.516617 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.516584    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-dshm\") pod \"distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") " pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:45.517252 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.517230    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d94a1258-dbb1-44b5-bf57-1b44b66fe286-tls-certs\") pod \"distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") " pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:45.577301 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.577272    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqb7r\" (UniqueName: \"kubernetes.io/projected/d94a1258-dbb1-44b5-bf57-1b44b66fe286-kube-api-access-dqb7r\") pod \"distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") " pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:45.625420 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.625386    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:45.892638 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.892552    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 01 05:20:45.892638 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:45.892624    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv" podUID="a25440ad-796b-4ef8-bf27-18ef58ca955a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 01 05:20:46.040248 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:46.040218    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"]
Nov 01 05:20:46.042213 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:20:46.042182    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd94a1258_dbb1_44b5_bf57_1b44b66fe286.slice/crio-8a8d4e39f94220b5e7969bc087908c6c61ec30b16473377f2f1ddd3256bbdced WatchSource:0}: Error finding container 8a8d4e39f94220b5e7969bc087908c6c61ec30b16473377f2f1ddd3256bbdced: Status 404 returned error can't find the container with id 8a8d4e39f94220b5e7969bc087908c6c61ec30b16473377f2f1ddd3256bbdced
Nov 01 05:20:46.559610 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:46.559567    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx" event={"ID":"d94a1258-dbb1-44b5-bf57-1b44b66fe286","Type":"ContainerStarted","Data":"f64d7e83d7bc3f7ceb844e1ec23b486eb49272a5c50623b99156f59562c601b7"}
Nov 01 05:20:46.559843 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:46.559618    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx" event={"ID":"d94a1258-dbb1-44b5-bf57-1b44b66fe286","Type":"ContainerStarted","Data":"8a8d4e39f94220b5e7969bc087908c6c61ec30b16473377f2f1ddd3256bbdced"}
Nov 01 05:20:46.626001 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:46.625964    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:46.678389 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:46.678319    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx" podStartSLOduration=1.678297683 podStartE2EDuration="1.678297683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:20:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:20:46.675462609 +0000 UTC m=+1063.565838586" watchObservedRunningTime="2026-11-01 05:20:46.678297683 +0000 UTC m=+1063.568673661"
Nov 01 05:20:46.897509 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:46.897414    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:46.897831 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:46.897812    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:46.898820 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:46.898768    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:20:52.529919 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:20:52.529883    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:20:55.626459 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:55.626409    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:55.645385 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:55.645303    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:55.656746 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:55.656720    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:20:59.001451 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:59.001413    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"]
Nov 01 05:20:59.001978 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:20:59.001631    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt" podUID="27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" containerName="istio-proxy" containerID="cri-o://44747da3fcc78d14a06394a5e019114f1ec89a1b763e830c8a6da2b5629a7574" gracePeriod=30
Nov 01 05:21:04.265565 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.265536    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:21:04.362363 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.362329    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-podinfo\") pod \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") "
Nov 01 05:21:04.362363 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.362367    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-workload-certs\") pod \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") "
Nov 01 05:21:04.362624 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.362390    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-credential-socket\") pod \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") "
Nov 01 05:21:04.362624 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.362442    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-kube-api-access-9fdqg\" (UniqueName: \"kubernetes.io/projected/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-kube-api-access-9fdqg\") pod \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") "
Nov 01 05:21:04.362624 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.362458    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-workload-socket\") pod \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") "
Nov 01 05:21:04.362624 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.362485    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-envoy\") pod \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") "
Nov 01 05:21:04.362624 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.362597    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-token\" (UniqueName: \"kubernetes.io/projected/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-token\") pod \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") "
Nov 01 05:21:04.362915 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.362660    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-data\") pod \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") "
Nov 01 05:21:04.362915 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.362764    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istiod-ca-cert\") pod \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\" (UID: \"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89\") "
Nov 01 05:21:04.362915 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.362887    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-workload-certs" pod "27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" (UID: "27a97af3-2e36-4317-9ae5-b5ee7ff7ac89"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:21:04.363068 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.362910    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-workload-socket" pod "27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" (UID: "27a97af3-2e36-4317-9ae5-b5ee7ff7ac89"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:21:04.363135 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.363060    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-data" pod "27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" (UID: "27a97af3-2e36-4317-9ae5-b5ee7ff7ac89"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:21:04.363135 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.363081    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-credential-socket" pod "27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" (UID: "27a97af3-2e36-4317-9ae5-b5ee7ff7ac89"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:21:04.363515 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.363449    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istiod-ca-cert" pod "27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" (UID: "27a97af3-2e36-4317-9ae5-b5ee7ff7ac89"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:21:04.363628 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.363580    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:21:04.363628 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.363603    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:21:04.363628 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.363617    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:21:04.363820 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.363631    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:21:04.365086 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.365062    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-podinfo" pod "27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" (UID: "27a97af3-2e36-4317-9ae5-b5ee7ff7ac89"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 05:21:04.365246 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.365223    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-token" pod "27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" (UID: "27a97af3-2e36-4317-9ae5-b5ee7ff7ac89"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:21:04.365314 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.365268    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-kube-api-access-9fdqg" pod "27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" (UID: "27a97af3-2e36-4317-9ae5-b5ee7ff7ac89"). InnerVolumeSpecName "kube-api-access-9fdqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:21:04.365368 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.365332    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-envoy" pod "27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" (UID: "27a97af3-2e36-4317-9ae5-b5ee7ff7ac89"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:21:04.465565 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.465533    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:21:04.465565 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.465563    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:21:04.465565 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.465573    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:21:04.465850 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.465582    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fdqg\" (UniqueName: \"kubernetes.io/projected/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-kube-api-access-9fdqg\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:21:04.465850 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.465598    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:21:04.700940 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.700847    4864 generic.go:356] "Generic (PLEG): container finished" podID="27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" containerID="44747da3fcc78d14a06394a5e019114f1ec89a1b763e830c8a6da2b5629a7574" exitCode=0
Nov 01 05:21:04.700940 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.700894    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"
Nov 01 05:21:04.701179 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.700931    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt" event={"ID":"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89","Type":"ContainerDied","Data":"44747da3fcc78d14a06394a5e019114f1ec89a1b763e830c8a6da2b5629a7574"}
Nov 01 05:21:04.701179 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.700994    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt" event={"ID":"27a97af3-2e36-4317-9ae5-b5ee7ff7ac89","Type":"ContainerDied","Data":"440baac89bc5b5259cb65f59e505f933ae910c043e6ba57dd7935ec3527e668e"}
Nov 01 05:21:04.701179 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.701024    4864 scope.go:122] "RemoveContainer" containerID="44747da3fcc78d14a06394a5e019114f1ec89a1b763e830c8a6da2b5629a7574"
Nov 01 05:21:04.711877 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.711862    4864 scope.go:122] "RemoveContainer" containerID="44747da3fcc78d14a06394a5e019114f1ec89a1b763e830c8a6da2b5629a7574"
Nov 01 05:21:04.712131 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:21:04.712116    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44747da3fcc78d14a06394a5e019114f1ec89a1b763e830c8a6da2b5629a7574\": container with ID starting with 44747da3fcc78d14a06394a5e019114f1ec89a1b763e830c8a6da2b5629a7574 not found: ID does not exist" containerID="44747da3fcc78d14a06394a5e019114f1ec89a1b763e830c8a6da2b5629a7574"
Nov 01 05:21:04.712171 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.712141    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44747da3fcc78d14a06394a5e019114f1ec89a1b763e830c8a6da2b5629a7574"} err="failed to get container status \"44747da3fcc78d14a06394a5e019114f1ec89a1b763e830c8a6da2b5629a7574\": rpc error: code = NotFound desc = could not find container \"44747da3fcc78d14a06394a5e019114f1ec89a1b763e830c8a6da2b5629a7574\": container with ID starting with 44747da3fcc78d14a06394a5e019114f1ec89a1b763e830c8a6da2b5629a7574 not found: ID does not exist"
Nov 01 05:21:04.869715 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.869673    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"]
Nov 01 05:21:04.920998 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:04.920964    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt"]
Nov 01 05:21:05.248004 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:05.247975    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" path="/var/lib/kubelet/pods/27a97af3-2e36-4317-9ae5-b5ee7ff7ac89/volumes"
Nov 01 05:21:09.902682 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:09.902630    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"]
Nov 01 05:21:09.903244 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:09.903152    4864 cpu_manager.go:414] "containerMap: removing container" podUID="27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" containerName="istio-proxy"
Nov 01 05:21:09.903244 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:09.903165    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" containerName="istio-proxy"
Nov 01 05:21:09.903364 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:09.903260    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="27a97af3-2e36-4317-9ae5-b5ee7ff7ac89" containerName="istio-proxy"
Nov 01 05:21:09.907841 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:09.907816    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.003249 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.003209    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"]
Nov 01 05:21:10.028416 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.028376    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-podinfo\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.028617 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.028506    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvdnv\" (UniqueName: \"kubernetes.io/projected/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-kube-api-access-fvdnv\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.028697 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.028627    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-data\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.028760 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.028693    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istiod-ca-cert\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.028760 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.028736    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-workload-socket\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.028974 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.028950    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-token\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.029084 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.028992    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-credential-socket\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.029216 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.029100    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-envoy\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.029282 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.029266    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-workload-certs\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.130304 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.130269    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvdnv\" (UniqueName: \"kubernetes.io/projected/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-kube-api-access-fvdnv\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.130490 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.130322    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-data\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.130490 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.130356    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istiod-ca-cert\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.130490 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.130379    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-workload-socket\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.130490 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.130419    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-token\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.130490 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.130441    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-credential-socket\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.130490 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.130474    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-envoy\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.130853 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.130520    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-workload-certs\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.130853 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.130578    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-podinfo\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.130941 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.130857    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-data\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.130973 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.130954    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-workload-socket\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.131317 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.131290    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istiod-ca-cert\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.131563 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.131358    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-credential-socket\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.131563 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.131537    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-workload-certs\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.133260 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.133234    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-envoy\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.133503 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.133474    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-podinfo\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.194226 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.194147    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-token\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.194444 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.194419    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvdnv\" (UniqueName: \"kubernetes.io/projected/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-kube-api-access-fvdnv\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.230205 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.230163    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:10.663717 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.663696    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"]
Nov 01 05:21:10.665402 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:21:10.665378    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb7c1a34_f9cf_445b_b14a_058fc937a6cb.slice/crio-5ba6fc1776bfc3b72f22ad863cd900d2211fbf858fbfde617538f42500ce87e5 WatchSource:0}: Error finding container 5ba6fc1776bfc3b72f22ad863cd900d2211fbf858fbfde617538f42500ce87e5: Status 404 returned error can't find the container with id 5ba6fc1776bfc3b72f22ad863cd900d2211fbf858fbfde617538f42500ce87e5
Nov 01 05:21:10.667687 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.667657    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:21:10.667775 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.667720    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:21:10.667775 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.667751    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:21:10.740533 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:10.740502    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm" event={"ID":"fb7c1a34-f9cf-445b-b14a-058fc937a6cb","Type":"ContainerStarted","Data":"5ba6fc1776bfc3b72f22ad863cd900d2211fbf858fbfde617538f42500ce87e5"}
Nov 01 05:21:11.749183 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:11.749139    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm" event={"ID":"fb7c1a34-f9cf-445b-b14a-058fc937a6cb","Type":"ContainerStarted","Data":"f2fde46837394499cdacc51fb4b7fe04c32877c814486e331690753aa5d8248a"}
Nov 01 05:21:11.925897 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:11.925826    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm" podStartSLOduration=2.9258027049999997 podStartE2EDuration="2.925802705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:21:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:21:11.92386255 +0000 UTC m=+1088.814238526" watchObservedRunningTime="2026-11-01 05:21:11.925802705 +0000 UTC m=+1088.816178678"
Nov 01 05:21:12.230901 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:12.230862    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:12.232723 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:12.232697    4864 patch_prober.go:29] interesting pod/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 01 05:21:12.232887 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:12.232749    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm" podUID="fb7c1a34-f9cf-445b-b14a-058fc937a6cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 01 05:21:13.231192 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:13.231156    4864 patch_prober.go:29] interesting pod/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 01 05:21:13.231680 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:13.231210    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm" podUID="fb7c1a34-f9cf-445b-b14a-058fc937a6cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 01 05:21:14.230586 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:14.230554    4864 patch_prober.go:29] interesting pod/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 01 05:21:14.230775 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:14.230607    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm" podUID="fb7c1a34-f9cf-445b-b14a-058fc937a6cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 01 05:21:15.231044 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:15.231006    4864 patch_prober.go:29] interesting pod/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 01 05:21:15.231459 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:15.231072    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm" podUID="fb7c1a34-f9cf-445b-b14a-058fc937a6cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 01 05:21:16.231268 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:16.231207    4864 patch_prober.go:29] interesting pod/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 01 05:21:16.231664 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:16.231297    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm" podUID="fb7c1a34-f9cf-445b-b14a-058fc937a6cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 01 05:21:17.230674 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:17.230619    4864 patch_prober.go:29] interesting pod/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 01 05:21:17.230910 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:17.230702    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm" podUID="fb7c1a34-f9cf-445b-b14a-058fc937a6cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 01 05:21:18.235404 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:18.235375    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:18.235845 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:18.235826    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:18.236533 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:18.236506    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:21:22.525355 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:21:22.525311    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:21:39.046888 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.046854    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"]
Nov 01 05:21:39.050685 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.050666    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.156658 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.156626    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"]
Nov 01 05:21:39.198293 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.198253    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-data\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.198476 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.198339    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-token\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.198476 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.198425    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-podinfo\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.198580 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.198479    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-envoy\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.198580 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.198514    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istiod-ca-cert\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.198711 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.198689    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-workload-certs\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.198772 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.198757    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-workload-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.198845 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.198808    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rch6c\" (UniqueName: \"kubernetes.io/projected/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-kube-api-access-rch6c\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.199030 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.199009    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-credential-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.300371 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.300273    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-envoy\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.300371 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.300323    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istiod-ca-cert\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.300371 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.300370    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-workload-certs\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.300663 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.300404    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-workload-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.300663 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.300429    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rch6c\" (UniqueName: \"kubernetes.io/projected/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-kube-api-access-rch6c\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.300663 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.300576    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-credential-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.300663 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.300643    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-data\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.300918 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.300674    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-token\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.300918 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.300704    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-podinfo\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.300918 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.300773    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-workload-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.300918 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.300855    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-workload-certs\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.301125 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.301060    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-data\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.301125 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.301103    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-credential-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.301546 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.301524    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istiod-ca-cert\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.302916 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.302892    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-envoy\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.303296 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.303278    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-podinfo\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.362941 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.362908    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rch6c\" (UniqueName: \"kubernetes.io/projected/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-kube-api-access-rch6c\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.371665 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.371602    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-token\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-j7t9j\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:39.665919 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:39.665887    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:40.102229 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:21:40.102194    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03518f25_1bdb_4021_aa8c_6d1e9083c8fe.slice/crio-007b8741a739c1901e84458c446cb7f1b5e65096e862142d5995303c308e750f WatchSource:0}: Error finding container 007b8741a739c1901e84458c446cb7f1b5e65096e862142d5995303c308e750f: Status 404 returned error can't find the container with id 007b8741a739c1901e84458c446cb7f1b5e65096e862142d5995303c308e750f
Nov 01 05:21:40.102572 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:40.102376    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"]
Nov 01 05:21:40.104529 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:40.104500    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:21:40.104620 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:40.104581    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:21:40.104620 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:40.104608    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:21:40.978058 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:40.978015    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j" event={"ID":"03518f25-1bdb-4021-aa8c-6d1e9083c8fe","Type":"ContainerStarted","Data":"aa8a28ca172c0d6bec7371311e815d50df0a551dde4aa2ccbc6f458ba9faba72"}
Nov 01 05:21:40.978374 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:40.978349    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j" event={"ID":"03518f25-1bdb-4021-aa8c-6d1e9083c8fe","Type":"ContainerStarted","Data":"007b8741a739c1901e84458c446cb7f1b5e65096e862142d5995303c308e750f"}
Nov 01 05:21:41.193807 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:41.188347    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j" podStartSLOduration=2.188321507 podStartE2EDuration="2.188321507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:21:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:21:41.166383171 +0000 UTC m=+1118.056759159" watchObservedRunningTime="2026-11-01 05:21:41.188321507 +0000 UTC m=+1118.078697484"
Nov 01 05:21:41.667486 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:41.667447    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:41.667874 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:41.667851    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 01 05:21:41.668086 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:41.668051    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j" podUID="03518f25-1bdb-4021-aa8c-6d1e9083c8fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 01 05:21:42.675405 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:42.674651    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 01 05:21:42.675405 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:42.674714    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j" podUID="03518f25-1bdb-4021-aa8c-6d1e9083c8fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 01 05:21:43.688889 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:43.683362    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 01 05:21:43.688889 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:43.683429    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j" podUID="03518f25-1bdb-4021-aa8c-6d1e9083c8fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 01 05:21:44.675202 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:44.675167    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 01 05:21:44.675617 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:44.675582    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j" podUID="03518f25-1bdb-4021-aa8c-6d1e9083c8fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 01 05:21:45.671212 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:45.671177    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 01 05:21:45.671862 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:45.671830    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j" podUID="03518f25-1bdb-4021-aa8c-6d1e9083c8fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 01 05:21:46.666658 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:46.666611    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 01 05:21:46.666901 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:46.666691    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j" podUID="03518f25-1bdb-4021-aa8c-6d1e9083c8fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 01 05:21:48.707983 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:48.707934    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 05:21:48.708525 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:48.708008    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j" podUID="03518f25-1bdb-4021-aa8c-6d1e9083c8fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 05:21:48.916313 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:48.916275    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:48.917167 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:48.917136    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:48.917912 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:21:48.917891    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:21:52.525541 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:21:52.525503    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:22:00.201844 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:00.201803    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"]
Nov 01 05:22:10.364805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.364755    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"]
Nov 01 05:22:10.370096 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.370073    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.419104 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.419059    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-data\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.419295 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.419165    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-token\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.419295 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.419209    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffz58\" (UniqueName: \"kubernetes.io/projected/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-kube-api-access-ffz58\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.419393 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.419305    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istiod-ca-cert\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.419393 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.419345    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-podinfo\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.419393 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.419373    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-workload-certs\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.419510 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.419489    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-workload-socket\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.419547 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.419526    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-credential-socket\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.419675 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.419649    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-envoy\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.520743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.520704    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-workload-certs\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.520925 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.520846    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-workload-socket\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.520925 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.520886    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-credential-socket\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.521032 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.520964    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-envoy\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.521086 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.521026    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-data\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.521086 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.521064    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-token\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.521205 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.521096    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffz58\" (UniqueName: \"kubernetes.io/projected/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-kube-api-access-ffz58\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.521205 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.521104    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-workload-certs\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.521315 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.521234    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-workload-socket\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.521315 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.521303    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-credential-socket\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.521461 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.521439    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-data\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.521531 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.521456    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istiod-ca-cert\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.521531 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.521516    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-podinfo\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.522036 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.522015    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istiod-ca-cert\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.523576 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.523547    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-envoy\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.524198 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.524175    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-podinfo\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.574906 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.574872    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"]
Nov 01 05:22:10.583910 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.583880    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-token\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.584816 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.584707    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffz58\" (UniqueName: \"kubernetes.io/projected/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-kube-api-access-ffz58\") pod \"e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") " pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:10.685143 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:10.685061    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:11.114600 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:11.114558    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"]
Nov 01 05:22:11.152203 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:22:11.152173    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d57abe1_4de8_49e9_a864_2c48cc5ce6be.slice/crio-954fb1595bab469d13e4dafb914203e4198cfdc231e7fc2bdc396d069d3bb208 WatchSource:0}: Error finding container 954fb1595bab469d13e4dafb914203e4198cfdc231e7fc2bdc396d069d3bb208: Status 404 returned error can't find the container with id 954fb1595bab469d13e4dafb914203e4198cfdc231e7fc2bdc396d069d3bb208
Nov 01 05:22:11.154987 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:11.154956    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:22:11.155091 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:11.155025    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:22:11.155091 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:11.155054    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:22:11.202515 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:11.202473    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj" event={"ID":"5d57abe1-4de8-49e9-a864-2c48cc5ce6be","Type":"ContainerStarted","Data":"954fb1595bab469d13e4dafb914203e4198cfdc231e7fc2bdc396d069d3bb208"}
Nov 01 05:22:12.228431 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:12.228399    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj" event={"ID":"5d57abe1-4de8-49e9-a864-2c48cc5ce6be","Type":"ContainerStarted","Data":"de24428edcff99a22ad6149894ac3c62ca327d27fd3295c05918f13ce7a37c28"}
Nov 01 05:22:12.406809 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:12.402617    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj" podStartSLOduration=2.402597574 podStartE2EDuration="2.402597574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:22:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:22:12.399578464 +0000 UTC m=+1149.289954441" watchObservedRunningTime="2026-11-01 05:22:12.402597574 +0000 UTC m=+1149.292973549"
Nov 01 05:22:12.686148 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:12.686114    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:12.690070 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:12.690041    4864 patch_prober.go:29] interesting pod/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 01 05:22:12.690276 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:12.690254    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj" podUID="5d57abe1-4de8-49e9-a864-2c48cc5ce6be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 01 05:22:13.687275 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:13.687235    4864 patch_prober.go:29] interesting pod/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 01 05:22:13.687919 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:13.687869    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj" podUID="5d57abe1-4de8-49e9-a864-2c48cc5ce6be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 01 05:22:14.686211 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:14.686172    4864 patch_prober.go:29] interesting pod/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 01 05:22:14.686551 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:14.686523    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj" podUID="5d57abe1-4de8-49e9-a864-2c48cc5ce6be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 01 05:22:15.223440 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.223370    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp" podUID="52b5e868-dd1f-490d-821e-b178350e7ea2" containerName="main" containerID="cri-o://c827c2e0f7afa3e3b5355099527b5a5a8fad186d1646c244c0d27f7c52da6db7" gracePeriod=45
Nov 01 05:22:15.648312 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.648015    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:22:15.686672 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.686207    4864 patch_prober.go:29] interesting pod/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 01 05:22:15.686672 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.686277    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj" podUID="5d57abe1-4de8-49e9-a864-2c48cc5ce6be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 01 05:22:15.720778 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.720737    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-home\" (UniqueName: \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-home\") pod \"52b5e868-dd1f-490d-821e-b178350e7ea2\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") "
Nov 01 05:22:15.720992 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.720850    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-dshm\") pod \"52b5e868-dd1f-490d-821e-b178350e7ea2\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") "
Nov 01 05:22:15.720992 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.720947    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-tmp-dir\") pod \"52b5e868-dd1f-490d-821e-b178350e7ea2\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") "
Nov 01 05:22:15.721116 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.721005    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-model-cache\") pod \"52b5e868-dd1f-490d-821e-b178350e7ea2\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") "
Nov 01 05:22:15.721116 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.721052    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-home" pod "52b5e868-dd1f-490d-821e-b178350e7ea2" (UID: "52b5e868-dd1f-490d-821e-b178350e7ea2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:22:15.721116 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.721098    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52b5e868-dd1f-490d-821e-b178350e7ea2-kube-api-access-mp4g5\" (UniqueName: \"kubernetes.io/projected/52b5e868-dd1f-490d-821e-b178350e7ea2-kube-api-access-mp4g5\") pod \"52b5e868-dd1f-490d-821e-b178350e7ea2\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") "
Nov 01 05:22:15.721270 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.721129    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/52b5e868-dd1f-490d-821e-b178350e7ea2-tls-certs\" (UniqueName: \"kubernetes.io/secret/52b5e868-dd1f-490d-821e-b178350e7ea2-tls-certs\") pod \"52b5e868-dd1f-490d-821e-b178350e7ea2\" (UID: \"52b5e868-dd1f-490d-821e-b178350e7ea2\") "
Nov 01 05:22:15.721270 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.721214    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-tmp-dir" pod "52b5e868-dd1f-490d-821e-b178350e7ea2" (UID: "52b5e868-dd1f-490d-821e-b178350e7ea2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:22:15.721387 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.721283    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-model-cache" pod "52b5e868-dd1f-490d-821e-b178350e7ea2" (UID: "52b5e868-dd1f-490d-821e-b178350e7ea2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:22:15.721590 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.721563    4864 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-tmp-dir\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:22:15.721763 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.721746    4864 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-model-cache\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:22:15.721911 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.721895    4864 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-home\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:22:15.723462 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.723440    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52b5e868-dd1f-490d-821e-b178350e7ea2-kube-api-access-mp4g5" pod "52b5e868-dd1f-490d-821e-b178350e7ea2" (UID: "52b5e868-dd1f-490d-821e-b178350e7ea2"). InnerVolumeSpecName "kube-api-access-mp4g5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:22:15.723574 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.723477    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52b5e868-dd1f-490d-821e-b178350e7ea2-tls-certs" pod "52b5e868-dd1f-490d-821e-b178350e7ea2" (UID: "52b5e868-dd1f-490d-821e-b178350e7ea2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:22:15.723927 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.723909    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-dshm" pod "52b5e868-dd1f-490d-821e-b178350e7ea2" (UID: "52b5e868-dd1f-490d-821e-b178350e7ea2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:22:15.823402 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.823354    4864 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52b5e868-dd1f-490d-821e-b178350e7ea2-dshm\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:22:15.823402 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.823397    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mp4g5\" (UniqueName: \"kubernetes.io/projected/52b5e868-dd1f-490d-821e-b178350e7ea2-kube-api-access-mp4g5\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:22:15.823673 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:15.823416    4864 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52b5e868-dd1f-490d-821e-b178350e7ea2-tls-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:22:16.256304 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:16.256274    4864 generic.go:356] "Generic (PLEG): container finished" podID="52b5e868-dd1f-490d-821e-b178350e7ea2" containerID="c827c2e0f7afa3e3b5355099527b5a5a8fad186d1646c244c0d27f7c52da6db7" exitCode=0
Nov 01 05:22:16.256702 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:16.256325    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"
Nov 01 05:22:16.256702 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:16.256365    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp" event={"ID":"52b5e868-dd1f-490d-821e-b178350e7ea2","Type":"ContainerDied","Data":"c827c2e0f7afa3e3b5355099527b5a5a8fad186d1646c244c0d27f7c52da6db7"}
Nov 01 05:22:16.256702 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:16.256422    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp" event={"ID":"52b5e868-dd1f-490d-821e-b178350e7ea2","Type":"ContainerDied","Data":"34e81d25b3a0e830839e88ed501ae4f715b987989fe36ac11cbdca1b477eecf5"}
Nov 01 05:22:16.256702 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:16.256444    4864 scope.go:122] "RemoveContainer" containerID="c827c2e0f7afa3e3b5355099527b5a5a8fad186d1646c244c0d27f7c52da6db7"
Nov 01 05:22:16.332267 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:16.328953    4864 scope.go:122] "RemoveContainer" containerID="c827c2e0f7afa3e3b5355099527b5a5a8fad186d1646c244c0d27f7c52da6db7"
Nov 01 05:22:16.333422 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:22:16.333231    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c827c2e0f7afa3e3b5355099527b5a5a8fad186d1646c244c0d27f7c52da6db7\": container with ID starting with c827c2e0f7afa3e3b5355099527b5a5a8fad186d1646c244c0d27f7c52da6db7 not found: ID does not exist" containerID="c827c2e0f7afa3e3b5355099527b5a5a8fad186d1646c244c0d27f7c52da6db7"
Nov 01 05:22:16.333422 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:16.333274    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c827c2e0f7afa3e3b5355099527b5a5a8fad186d1646c244c0d27f7c52da6db7"} err="failed to get container status \"c827c2e0f7afa3e3b5355099527b5a5a8fad186d1646c244c0d27f7c52da6db7\": rpc error: code = NotFound desc = could not find container \"c827c2e0f7afa3e3b5355099527b5a5a8fad186d1646c244c0d27f7c52da6db7\": container with ID starting with c827c2e0f7afa3e3b5355099527b5a5a8fad186d1646c244c0d27f7c52da6db7 not found: ID does not exist"
Nov 01 05:22:16.450410 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:16.450341    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"]
Nov 01 05:22:16.455544 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:16.455304    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp" podUID="52b5e868-dd1f-490d-821e-b178350e7ea2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 05:22:16.519308 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:16.519209    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-2ea125/test-model-2ea125-kserve-7f7f6dc9bc-46snp"]
Nov 01 05:22:16.685773 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:16.685736    4864 patch_prober.go:29] interesting pod/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 01 05:22:16.686437 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:16.686053    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj" podUID="5d57abe1-4de8-49e9-a864-2c48cc5ce6be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 01 05:22:17.247565 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:17.247530    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="52b5e868-dd1f-490d-821e-b178350e7ea2" path="/var/lib/kubelet/pods/52b5e868-dd1f-490d-821e-b178350e7ea2/volumes"
Nov 01 05:22:17.685934 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:17.685901    4864 patch_prober.go:29] interesting pod/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 01 05:22:17.686324 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:17.685963    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj" podUID="5d57abe1-4de8-49e9-a864-2c48cc5ce6be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 01 05:22:18.825166 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:18.825123    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:18.825679 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:18.825579    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:18.826423 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:18.826403    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:22:22.527625 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:22:22.527587    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:22:31.324653 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:31.324611    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"]
Nov 01 05:22:33.424629 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:33.424594    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"]
Nov 01 05:22:33.425134 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:33.425119    4864 cpu_manager.go:414] "containerMap: removing container" podUID="52b5e868-dd1f-490d-821e-b178350e7ea2" containerName="main"
Nov 01 05:22:33.425182 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:33.425135    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="52b5e868-dd1f-490d-821e-b178350e7ea2" containerName="main"
Nov 01 05:22:33.425249 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:33.425241    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="52b5e868-dd1f-490d-821e-b178350e7ea2" containerName="main"
Nov 01 05:22:33.428843 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:33.428824    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"
Nov 01 05:22:33.533566 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:33.533533    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-5451cb-serving-cert\""
Nov 01 05:22:33.536599 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:33.536563    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rchx\" (UniqueName: \"kubernetes.io/projected/d74d58db-55bb-40f4-8aa9-588fab2d1a49-kube-api-access-5rchx\") pod \"maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z\" (UID: \"d74d58db-55bb-40f4-8aa9-588fab2d1a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"
Nov 01 05:22:33.536738 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:33.536615    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d74d58db-55bb-40f4-8aa9-588fab2d1a49-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z\" (UID: \"d74d58db-55bb-40f4-8aa9-588fab2d1a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"
Nov 01 05:22:33.536854 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:33.536826    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d74d58db-55bb-40f4-8aa9-588fab2d1a49-metrics-certs\") pod \"maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z\" (UID: \"d74d58db-55bb-40f4-8aa9-588fab2d1a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"
Nov 01 05:22:33.536923 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:33.536867    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d74d58db-55bb-40f4-8aa9-588fab2d1a49-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z\" (UID: \"d74d58db-55bb-40f4-8aa9-588fab2d1a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"
Nov 01 05:22:33.537052 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:33.537021    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"]
Nov 01 05:22:33.637764 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:33.637730    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d74d58db-55bb-40f4-8aa9-588fab2d1a49-metrics-certs\") pod \"maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z\" (UID: \"d74d58db-55bb-40f4-8aa9-588fab2d1a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"
Nov 01 05:22:33.637988 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:33.637770    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d74d58db-55bb-40f4-8aa9-588fab2d1a49-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z\" (UID: \"d74d58db-55bb-40f4-8aa9-588fab2d1a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"
Nov 01 05:22:33.637988 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:33.637843    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rchx\" (UniqueName: \"kubernetes.io/projected/d74d58db-55bb-40f4-8aa9-588fab2d1a49-kube-api-access-5rchx\") pod \"maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z\" (UID: \"d74d58db-55bb-40f4-8aa9-588fab2d1a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"
Nov 01 05:22:33.637988 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:33.637867    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d74d58db-55bb-40f4-8aa9-588fab2d1a49-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z\" (UID: \"d74d58db-55bb-40f4-8aa9-588fab2d1a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"
Nov 01 05:22:33.637988 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:22:33.637966    4864 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-serving-cert: secret "maas-api-e2e-ipp-b-5451cb-serving-cert" not found
Nov 01 05:22:33.638232 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:22:33.638048    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d74d58db-55bb-40f4-8aa9-588fab2d1a49-maas-api-tls podName:d74d58db-55bb-40f4-8aa9-588fab2d1a49 nodeName:}" failed. No retries permitted until 2026-11-01 05:22:34.138027419 +0000 UTC m=+1171.028403388 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d74d58db-55bb-40f4-8aa9-588fab2d1a49-maas-api-tls") pod "maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z" (UID: "d74d58db-55bb-40f4-8aa9-588fab2d1a49") : secret "maas-api-e2e-ipp-b-5451cb-serving-cert" not found
Nov 01 05:22:33.638523 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:33.638501    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d74d58db-55bb-40f4-8aa9-588fab2d1a49-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z\" (UID: \"d74d58db-55bb-40f4-8aa9-588fab2d1a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"
Nov 01 05:22:33.640567 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:33.640544    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d74d58db-55bb-40f4-8aa9-588fab2d1a49-metrics-certs\") pod \"maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z\" (UID: \"d74d58db-55bb-40f4-8aa9-588fab2d1a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"
Nov 01 05:22:33.700629 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:33.700548    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rchx\" (UniqueName: \"kubernetes.io/projected/d74d58db-55bb-40f4-8aa9-588fab2d1a49-kube-api-access-5rchx\") pod \"maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z\" (UID: \"d74d58db-55bb-40f4-8aa9-588fab2d1a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"
Nov 01 05:22:34.144311 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:34.144273    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d74d58db-55bb-40f4-8aa9-588fab2d1a49-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z\" (UID: \"d74d58db-55bb-40f4-8aa9-588fab2d1a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"
Nov 01 05:22:34.146951 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:34.146927    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d74d58db-55bb-40f4-8aa9-588fab2d1a49-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z\" (UID: \"d74d58db-55bb-40f4-8aa9-588fab2d1a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"
Nov 01 05:22:34.344296 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:34.344260    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"
Nov 01 05:22:34.784213 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:34.784162    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"]
Nov 01 05:22:34.793816 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:22:34.787409    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd74d58db_55bb_40f4_8aa9_588fab2d1a49.slice/crio-45181d63831ae0e7a33a3b04f437007ec21adf7e5f8af24f32ffffad7e97f08d WatchSource:0}: Error finding container 45181d63831ae0e7a33a3b04f437007ec21adf7e5f8af24f32ffffad7e97f08d: Status 404 returned error can't find the container with id 45181d63831ae0e7a33a3b04f437007ec21adf7e5f8af24f32ffffad7e97f08d
Nov 01 05:22:34.793816 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:34.789277    4864 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 05:22:35.646630 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:35.646591    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z" event={"ID":"d74d58db-55bb-40f4-8aa9-588fab2d1a49","Type":"ContainerStarted","Data":"45181d63831ae0e7a33a3b04f437007ec21adf7e5f8af24f32ffffad7e97f08d"}
Nov 01 05:22:36.654625 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:36.654587    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z" event={"ID":"d74d58db-55bb-40f4-8aa9-588fab2d1a49","Type":"ContainerStarted","Data":"89b101106cd61429d126ee5acf9fcb19f8ec2805e27308d0b34f89665ddaabfd"}
Nov 01 05:22:36.655191 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:36.654675    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"
Nov 01 05:22:36.772227 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:36.772172    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z" podStartSLOduration=2.653470619 podStartE2EDuration="3.772157492s" totalImagesPullingTime="1.118686873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:22:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:22:36.771452759 +0000 UTC m=+1173.661828735" watchObservedRunningTime="2026-11-01 05:22:36.772157492 +0000 UTC m=+1173.662533469"
Nov 01 05:22:39.524601 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.524571    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"]
Nov 01 05:22:39.528469 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.528445    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.634293 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.634256    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-envoy\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.634487 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.634321    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-podinfo\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.634487 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.634360    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-credential-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.634487 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.634406    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8lhk\" (UniqueName: \"kubernetes.io/projected/59915eb9-7731-4c07-925d-d9d1a2789d9a-kube-api-access-q8lhk\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.634487 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.634439    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-workload-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.634705 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.634487    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-token\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.634705 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.634546    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-data\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.634705 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.634569    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59915eb9-7731-4c07-925d-d9d1a2789d9a-istiod-ca-cert\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.634705 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.634637    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-workload-certs\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.634994 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.634968    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"]
Nov 01 05:22:39.735502 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.735461    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-workload-certs\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.735698 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.735591    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-envoy\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.735698 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.735636    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-podinfo\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.735698 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.735671    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-credential-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.735925 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.735743    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8lhk\" (UniqueName: \"kubernetes.io/projected/59915eb9-7731-4c07-925d-d9d1a2789d9a-kube-api-access-q8lhk\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.735925 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.735815    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-workload-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.735925 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.735866    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-token\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.736084 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.735927    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-data\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.736084 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.735963    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59915eb9-7731-4c07-925d-d9d1a2789d9a-istiod-ca-cert\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.736409 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.736329    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-workload-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.736578 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.736557    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-data\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.736652 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.736613    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-workload-certs\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.736652 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.736624    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-credential-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.736817 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.736774    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59915eb9-7731-4c07-925d-d9d1a2789d9a-istiod-ca-cert\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.738742 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.738719    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-envoy\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.738967 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.738950    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-podinfo\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.803311 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.797895    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-token\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.803311 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.798095    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8lhk\" (UniqueName: \"kubernetes.io/projected/59915eb9-7731-4c07-925d-d9d1a2789d9a-kube-api-access-q8lhk\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-4rqx7\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:39.843530 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:39.843490    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:40.275771 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:40.275738    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"]
Nov 01 05:22:40.276902 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:22:40.276871    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59915eb9_7731_4c07_925d_d9d1a2789d9a.slice/crio-1fff466b47bea8e4aa7090b45e979d13175f9df0f4c603dd9ee169aab99b9080 WatchSource:0}: Error finding container 1fff466b47bea8e4aa7090b45e979d13175f9df0f4c603dd9ee169aab99b9080: Status 404 returned error can't find the container with id 1fff466b47bea8e4aa7090b45e979d13175f9df0f4c603dd9ee169aab99b9080
Nov 01 05:22:40.279493 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:40.279457    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:22:40.279605 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:40.279539    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:22:40.279605 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:40.279584    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:22:40.684556 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:40.684515    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7" event={"ID":"59915eb9-7731-4c07-925d-d9d1a2789d9a","Type":"ContainerStarted","Data":"b4922312f39c4e7d1b2cf2b8635eb66b27793a86ceea482a36cd858f5451902a"}
Nov 01 05:22:40.685104 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:40.684574    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7" event={"ID":"59915eb9-7731-4c07-925d-d9d1a2789d9a","Type":"ContainerStarted","Data":"1fff466b47bea8e4aa7090b45e979d13175f9df0f4c603dd9ee169aab99b9080"}
Nov 01 05:22:40.812514 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:40.812441    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7" podStartSLOduration=1.8124184030000001 podStartE2EDuration="1.812418403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:22:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:22:40.810326911 +0000 UTC m=+1177.700702894" watchObservedRunningTime="2026-11-01 05:22:40.812418403 +0000 UTC m=+1177.702794374"
Nov 01 05:22:40.844471 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:40.844434    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:41.846233 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:41.846191    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 05:22:41.846694 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:41.846270    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7" podUID="59915eb9-7731-4c07-925d-d9d1a2789d9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 05:22:42.684956 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:42.684923    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"
Nov 01 05:22:42.844220 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:42.844181    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 05:22:42.844409 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:42.844283    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7" podUID="59915eb9-7731-4c07-925d-d9d1a2789d9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 05:22:43.844944 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:43.844905    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 05:22:43.845459 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:43.844990    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7" podUID="59915eb9-7731-4c07-925d-d9d1a2789d9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 05:22:44.844237 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:44.844195    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 05:22:44.844413 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:44.844263    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7" podUID="59915eb9-7731-4c07-925d-d9d1a2789d9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 05:22:45.844194 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:45.844159    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 05:22:45.844584 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:45.844218    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7" podUID="59915eb9-7731-4c07-925d-d9d1a2789d9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 05:22:46.347659 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.347619    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh" podUID="b13c463a-f27e-49a8-af7f-ee945dc1b9b8" containerName="main" containerID="cri-o://44e4fabb17f10f29dd54aef82c3cda3ecf64eee6a6dbdf8ca87a372e88e697c3" gracePeriod=45
Nov 01 05:22:46.509550 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.509504    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh" podUID="b13c463a-f27e-49a8-af7f-ee945dc1b9b8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": dial tcp 10.132.0.63:8000: connect: connection refused"
Nov 01 05:22:46.726851 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.726822    4864 generic.go:356] "Generic (PLEG): container finished" podID="b13c463a-f27e-49a8-af7f-ee945dc1b9b8" containerID="44e4fabb17f10f29dd54aef82c3cda3ecf64eee6a6dbdf8ca87a372e88e697c3" exitCode=0
Nov 01 05:22:46.726851 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.726835    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh" event={"ID":"b13c463a-f27e-49a8-af7f-ee945dc1b9b8","Type":"ContainerDied","Data":"44e4fabb17f10f29dd54aef82c3cda3ecf64eee6a6dbdf8ca87a372e88e697c3"}
Nov 01 05:22:46.727135 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.726887    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh" event={"ID":"b13c463a-f27e-49a8-af7f-ee945dc1b9b8","Type":"ContainerDied","Data":"3cd55acfa7acdae4bcc46f87bca9b11205cda4095fd39f5bb717316c21560bfa"}
Nov 01 05:22:46.727135 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.726899    4864 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3cd55acfa7acdae4bcc46f87bca9b11205cda4095fd39f5bb717316c21560bfa"
Nov 01 05:22:46.738698 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.738674    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:22:46.840463 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.840422    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-model-cache\") pod \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") "
Nov 01 05:22:46.840767 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.840520    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-tmp-dir\") pod \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") "
Nov 01 05:22:46.840767 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.840555    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-tls-certs\" (UniqueName: \"kubernetes.io/secret/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-tls-certs\") pod \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") "
Nov 01 05:22:46.840767 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.840580    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-dshm\") pod \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") "
Nov 01 05:22:46.840767 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.840678    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-kube-api-access-pxzln\" (UniqueName: \"kubernetes.io/projected/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-kube-api-access-pxzln\") pod \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") "
Nov 01 05:22:46.840767 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.840711    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-model-cache" pod "b13c463a-f27e-49a8-af7f-ee945dc1b9b8" (UID: "b13c463a-f27e-49a8-af7f-ee945dc1b9b8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:22:46.840767 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.840739    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-home\" (UniqueName: \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-home\") pod \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\" (UID: \"b13c463a-f27e-49a8-af7f-ee945dc1b9b8\") "
Nov 01 05:22:46.841147 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.840848    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-tmp-dir" pod "b13c463a-f27e-49a8-af7f-ee945dc1b9b8" (UID: "b13c463a-f27e-49a8-af7f-ee945dc1b9b8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:22:46.841147 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.840988    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-home" pod "b13c463a-f27e-49a8-af7f-ee945dc1b9b8" (UID: "b13c463a-f27e-49a8-af7f-ee945dc1b9b8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:22:46.841257 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.841230    4864 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-model-cache\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:22:46.841257 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.841245    4864 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-tmp-dir\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:22:46.841257 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.841256    4864 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-home\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:22:46.843118 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.843081    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-dshm" pod "b13c463a-f27e-49a8-af7f-ee945dc1b9b8" (UID: "b13c463a-f27e-49a8-af7f-ee945dc1b9b8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:22:46.843219 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.843131    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-tls-certs" pod "b13c463a-f27e-49a8-af7f-ee945dc1b9b8" (UID: "b13c463a-f27e-49a8-af7f-ee945dc1b9b8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:22:46.843416 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.843395    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-kube-api-access-pxzln" pod "b13c463a-f27e-49a8-af7f-ee945dc1b9b8" (UID: "b13c463a-f27e-49a8-af7f-ee945dc1b9b8"). InnerVolumeSpecName "kube-api-access-pxzln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:22:46.844634 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.844617    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 05:22:46.844948 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.844678    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7" podUID="59915eb9-7731-4c07-925d-d9d1a2789d9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 05:22:46.943215 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.943172    4864 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-tls-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:22:46.943215 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.943213    4864 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-dshm\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:22:46.943424 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:46.943230    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxzln\" (UniqueName: \"kubernetes.io/projected/b13c463a-f27e-49a8-af7f-ee945dc1b9b8-kube-api-access-pxzln\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:22:47.741724 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:47.740869    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"
Nov 01 05:22:47.900019 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:47.899978    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:47.901269 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:47.901242    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:47.902441 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:47.902417    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:22:47.911530 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:47.911502    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"]
Nov 01 05:22:47.961130 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:47.961053    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-af8051/test-model-af8051-kserve-576468f797-8zpxh"]
Nov 01 05:22:48.123538 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.123505    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"]
Nov 01 05:22:48.123737 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.123711    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg" podUID="8951da34-16d1-4f50-b68b-51dbde0ecc32" containerName="maas-api" containerID="cri-o://e4a6bae87310a51248b246580b8f6bbe23fbcd5f33a22c01321856c0a8448d9a" gracePeriod=30
Nov 01 05:22:48.440468 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.440436    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"
Nov 01 05:22:48.489617 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.489586    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8951da34-16d1-4f50-b68b-51dbde0ecc32-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8951da34-16d1-4f50-b68b-51dbde0ecc32-openshift-service-ca\") pod \"8951da34-16d1-4f50-b68b-51dbde0ecc32\" (UID: \"8951da34-16d1-4f50-b68b-51dbde0ecc32\") "
Nov 01 05:22:48.489857 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.489701    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8951da34-16d1-4f50-b68b-51dbde0ecc32-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8951da34-16d1-4f50-b68b-51dbde0ecc32-metrics-certs\") pod \"8951da34-16d1-4f50-b68b-51dbde0ecc32\" (UID: \"8951da34-16d1-4f50-b68b-51dbde0ecc32\") "
Nov 01 05:22:48.489857 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.489755    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8951da34-16d1-4f50-b68b-51dbde0ecc32-kube-api-access-xmb57\" (UniqueName: \"kubernetes.io/projected/8951da34-16d1-4f50-b68b-51dbde0ecc32-kube-api-access-xmb57\") pod \"8951da34-16d1-4f50-b68b-51dbde0ecc32\" (UID: \"8951da34-16d1-4f50-b68b-51dbde0ecc32\") "
Nov 01 05:22:48.489857 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.489833    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8951da34-16d1-4f50-b68b-51dbde0ecc32-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8951da34-16d1-4f50-b68b-51dbde0ecc32-maas-api-tls\") pod \"8951da34-16d1-4f50-b68b-51dbde0ecc32\" (UID: \"8951da34-16d1-4f50-b68b-51dbde0ecc32\") "
Nov 01 05:22:48.490102 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.490076    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8951da34-16d1-4f50-b68b-51dbde0ecc32-openshift-service-ca" pod "8951da34-16d1-4f50-b68b-51dbde0ecc32" (UID: "8951da34-16d1-4f50-b68b-51dbde0ecc32"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:22:48.490255 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.490238    4864 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8951da34-16d1-4f50-b68b-51dbde0ecc32-openshift-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:22:48.492314 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.492291    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8951da34-16d1-4f50-b68b-51dbde0ecc32-maas-api-tls" pod "8951da34-16d1-4f50-b68b-51dbde0ecc32" (UID: "8951da34-16d1-4f50-b68b-51dbde0ecc32"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:22:48.492521 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.492489    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8951da34-16d1-4f50-b68b-51dbde0ecc32-metrics-certs" pod "8951da34-16d1-4f50-b68b-51dbde0ecc32" (UID: "8951da34-16d1-4f50-b68b-51dbde0ecc32"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:22:48.492624 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.492520    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8951da34-16d1-4f50-b68b-51dbde0ecc32-kube-api-access-xmb57" pod "8951da34-16d1-4f50-b68b-51dbde0ecc32" (UID: "8951da34-16d1-4f50-b68b-51dbde0ecc32"). InnerVolumeSpecName "kube-api-access-xmb57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:22:48.591385 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.591349    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmb57\" (UniqueName: \"kubernetes.io/projected/8951da34-16d1-4f50-b68b-51dbde0ecc32-kube-api-access-xmb57\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:22:48.591385 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.591380    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8951da34-16d1-4f50-b68b-51dbde0ecc32-maas-api-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:22:48.591385 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.591389    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8951da34-16d1-4f50-b68b-51dbde0ecc32-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:22:48.756585 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.756487    4864 generic.go:356] "Generic (PLEG): container finished" podID="8951da34-16d1-4f50-b68b-51dbde0ecc32" containerID="e4a6bae87310a51248b246580b8f6bbe23fbcd5f33a22c01321856c0a8448d9a" exitCode=0
Nov 01 05:22:48.757859 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.757835    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"
Nov 01 05:22:48.761358 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.761317    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg" event={"ID":"8951da34-16d1-4f50-b68b-51dbde0ecc32","Type":"ContainerDied","Data":"e4a6bae87310a51248b246580b8f6bbe23fbcd5f33a22c01321856c0a8448d9a"}
Nov 01 05:22:48.761508 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.761385    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg" event={"ID":"8951da34-16d1-4f50-b68b-51dbde0ecc32","Type":"ContainerDied","Data":"ed953b5149158ece179d2c00084a34741a51b83e3a804e5e92a85767c50b6691"}
Nov 01 05:22:48.761508 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.761415    4864 scope.go:122] "RemoveContainer" containerID="e4a6bae87310a51248b246580b8f6bbe23fbcd5f33a22c01321856c0a8448d9a"
Nov 01 05:22:48.778663 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.778634    4864 scope.go:122] "RemoveContainer" containerID="e4a6bae87310a51248b246580b8f6bbe23fbcd5f33a22c01321856c0a8448d9a"
Nov 01 05:22:48.779049 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:22:48.779025    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4a6bae87310a51248b246580b8f6bbe23fbcd5f33a22c01321856c0a8448d9a\": container with ID starting with e4a6bae87310a51248b246580b8f6bbe23fbcd5f33a22c01321856c0a8448d9a not found: ID does not exist" containerID="e4a6bae87310a51248b246580b8f6bbe23fbcd5f33a22c01321856c0a8448d9a"
Nov 01 05:22:48.779151 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.779058    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4a6bae87310a51248b246580b8f6bbe23fbcd5f33a22c01321856c0a8448d9a"} err="failed to get container status \"e4a6bae87310a51248b246580b8f6bbe23fbcd5f33a22c01321856c0a8448d9a\": rpc error: code = NotFound desc = could not find container \"e4a6bae87310a51248b246580b8f6bbe23fbcd5f33a22c01321856c0a8448d9a\": container with ID starting with e4a6bae87310a51248b246580b8f6bbe23fbcd5f33a22c01321856c0a8448d9a not found: ID does not exist"
Nov 01 05:22:48.952388 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.952351    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"]
Nov 01 05:22:48.974855 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:48.974818    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2ea125-f4975f846-f2btg"]
Nov 01 05:22:49.262809 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:49.259484    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8951da34-16d1-4f50-b68b-51dbde0ecc32" path="/var/lib/kubelet/pods/8951da34-16d1-4f50-b68b-51dbde0ecc32/volumes"
Nov 01 05:22:49.262809 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:22:49.260135    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b13c463a-f27e-49a8-af7f-ee945dc1b9b8" path="/var/lib/kubelet/pods/b13c463a-f27e-49a8-af7f-ee945dc1b9b8/volumes"
Nov 01 05:22:52.523958 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:22:52.523923    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:23:03.615757 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:03.615727    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:23:03.620683 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:03.620656    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:23:03.624943 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:03.624922    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:23:03.629451 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:03.629433    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:23:22.524897 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:23:22.524815    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:23:28.024541 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:28.023897    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk" podUID="b28a4792-6e4e-4f9b-80e0-737dc18ba934" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 05:23:31.862031 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:31.861994    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-rv56j-mw2kw"]
Nov 01 05:23:31.862483 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:31.862470    4864 cpu_manager.go:414] "containerMap: removing container" podUID="b13c463a-f27e-49a8-af7f-ee945dc1b9b8" containerName="main"
Nov 01 05:23:31.862531 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:31.862484    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b13c463a-f27e-49a8-af7f-ee945dc1b9b8" containerName="main"
Nov 01 05:23:31.862531 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:31.862500    4864 cpu_manager.go:414] "containerMap: removing container" podUID="8951da34-16d1-4f50-b68b-51dbde0ecc32" containerName="maas-api"
Nov 01 05:23:31.862531 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:31.862505    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8951da34-16d1-4f50-b68b-51dbde0ecc32" containerName="maas-api"
Nov 01 05:23:31.862669 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:31.862604    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="b13c463a-f27e-49a8-af7f-ee945dc1b9b8" containerName="main"
Nov 01 05:23:31.862669 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:31.862613    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="8951da34-16d1-4f50-b68b-51dbde0ecc32" containerName="maas-api"
Nov 01 05:23:31.866049 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:31.866029    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-rv56j-mw2kw"
Nov 01 05:23:31.921036 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:31.921006    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-slwj6\""
Nov 01 05:23:31.973856 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:31.973819    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-rv56j-mw2kw"]
Nov 01 05:23:31.977186 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:31.977156    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/899c1e60-3610-4825-8803-854638f8f9fd-archives-path\") pod \"periodic-gathering-rv56j-mw2kw\" (UID: \"899c1e60-3610-4825-8803-854638f8f9fd\") " pod="openshift-insights/periodic-gathering-rv56j-mw2kw"
Nov 01 05:23:31.977303 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:31.977193    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/899c1e60-3610-4825-8803-854638f8f9fd-trusted-ca-bundle\") pod \"periodic-gathering-rv56j-mw2kw\" (UID: \"899c1e60-3610-4825-8803-854638f8f9fd\") " pod="openshift-insights/periodic-gathering-rv56j-mw2kw"
Nov 01 05:23:31.977303 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:31.977229    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/899c1e60-3610-4825-8803-854638f8f9fd-service-ca-bundle\") pod \"periodic-gathering-rv56j-mw2kw\" (UID: \"899c1e60-3610-4825-8803-854638f8f9fd\") " pod="openshift-insights/periodic-gathering-rv56j-mw2kw"
Nov 01 05:23:31.977379 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:31.977345    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c5vr\" (UniqueName: \"kubernetes.io/projected/899c1e60-3610-4825-8803-854638f8f9fd-kube-api-access-9c5vr\") pod \"periodic-gathering-rv56j-mw2kw\" (UID: \"899c1e60-3610-4825-8803-854638f8f9fd\") " pod="openshift-insights/periodic-gathering-rv56j-mw2kw"
Nov 01 05:23:32.079380 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:32.079342    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/899c1e60-3610-4825-8803-854638f8f9fd-archives-path\") pod \"periodic-gathering-rv56j-mw2kw\" (UID: \"899c1e60-3610-4825-8803-854638f8f9fd\") " pod="openshift-insights/periodic-gathering-rv56j-mw2kw"
Nov 01 05:23:32.079380 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:32.079393    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/899c1e60-3610-4825-8803-854638f8f9fd-trusted-ca-bundle\") pod \"periodic-gathering-rv56j-mw2kw\" (UID: \"899c1e60-3610-4825-8803-854638f8f9fd\") " pod="openshift-insights/periodic-gathering-rv56j-mw2kw"
Nov 01 05:23:32.079646 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:32.079427    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/899c1e60-3610-4825-8803-854638f8f9fd-service-ca-bundle\") pod \"periodic-gathering-rv56j-mw2kw\" (UID: \"899c1e60-3610-4825-8803-854638f8f9fd\") " pod="openshift-insights/periodic-gathering-rv56j-mw2kw"
Nov 01 05:23:32.079646 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:32.079459    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c5vr\" (UniqueName: \"kubernetes.io/projected/899c1e60-3610-4825-8803-854638f8f9fd-kube-api-access-9c5vr\") pod \"periodic-gathering-rv56j-mw2kw\" (UID: \"899c1e60-3610-4825-8803-854638f8f9fd\") " pod="openshift-insights/periodic-gathering-rv56j-mw2kw"
Nov 01 05:23:32.080412 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:32.080389    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/899c1e60-3610-4825-8803-854638f8f9fd-trusted-ca-bundle\") pod \"periodic-gathering-rv56j-mw2kw\" (UID: \"899c1e60-3610-4825-8803-854638f8f9fd\") " pod="openshift-insights/periodic-gathering-rv56j-mw2kw"
Nov 01 05:23:32.081034 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:32.081011    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/899c1e60-3610-4825-8803-854638f8f9fd-archives-path\") pod \"periodic-gathering-rv56j-mw2kw\" (UID: \"899c1e60-3610-4825-8803-854638f8f9fd\") " pod="openshift-insights/periodic-gathering-rv56j-mw2kw"
Nov 01 05:23:32.083063 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:32.083035    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/899c1e60-3610-4825-8803-854638f8f9fd-service-ca-bundle\") pod \"periodic-gathering-rv56j-mw2kw\" (UID: \"899c1e60-3610-4825-8803-854638f8f9fd\") " pod="openshift-insights/periodic-gathering-rv56j-mw2kw"
Nov 01 05:23:32.145693 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:32.145608    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c5vr\" (UniqueName: \"kubernetes.io/projected/899c1e60-3610-4825-8803-854638f8f9fd-kube-api-access-9c5vr\") pod \"periodic-gathering-rv56j-mw2kw\" (UID: \"899c1e60-3610-4825-8803-854638f8f9fd\") " pod="openshift-insights/periodic-gathering-rv56j-mw2kw"
Nov 01 05:23:32.178672 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:32.178644    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-rv56j-mw2kw"
Nov 01 05:23:32.557251 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:32.557217    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-rv56j-mw2kw"]
Nov 01 05:23:32.558655 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:23:32.558611    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod899c1e60_3610_4825_8803_854638f8f9fd.slice/crio-0442fb16b3b72ada69e9ef16b11d273171cea16fd54e1454720d4186a066264c WatchSource:0}: Error finding container 0442fb16b3b72ada69e9ef16b11d273171cea16fd54e1454720d4186a066264c: Status 404 returned error can't find the container with id 0442fb16b3b72ada69e9ef16b11d273171cea16fd54e1454720d4186a066264c
Nov 01 05:23:33.267484 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:33.267445    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-rv56j-mw2kw" event={"ID":"899c1e60-3610-4825-8803-854638f8f9fd","Type":"ContainerStarted","Data":"33da4e010b7c363acc2fe6453972d71c7b5b8723f7bc4c97952ebf0956e22444"}
Nov 01 05:23:33.267933 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:33.267506    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-rv56j-mw2kw" event={"ID":"899c1e60-3610-4825-8803-854638f8f9fd","Type":"ContainerStarted","Data":"0442fb16b3b72ada69e9ef16b11d273171cea16fd54e1454720d4186a066264c"}
Nov 01 05:23:33.386146 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:33.386080    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-rv56j-mw2kw" podStartSLOduration=2.386061597 podStartE2EDuration="2.386061597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:23:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:23:33.38307194 +0000 UTC m=+1230.273447918" watchObservedRunningTime="2026-11-01 05:23:33.386061597 +0000 UTC m=+1230.276437574"
Nov 01 05:23:39.507426 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.507382    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"]
Nov 01 05:23:39.515431 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.515410    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:39.571383 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.571348    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"]
Nov 01 05:23:39.617119 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.617084    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-2864b6\"/\"kube-root-ca.crt\""
Nov 01 05:23:39.617366 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.617098    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-2864b6\"/\"openshift-service-ca.crt\""
Nov 01 05:23:39.617366 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.617162    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-2864b6\"/\"default-dockercfg-twrsd\""
Nov 01 05:23:39.617652 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.617635    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-2864b6\"/\"ipp-route-2864b6-kserve-self-signed-certs\""
Nov 01 05:23:39.672848 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.672776    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-model-cache\") pod \"ipp-route-2864b6-kserve-74f7784cd9-7dlpb\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") " pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:39.673024 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.672903    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-home\") pod \"ipp-route-2864b6-kserve-74f7784cd9-7dlpb\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") " pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:39.673024 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.672944    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-tls-certs\") pod \"ipp-route-2864b6-kserve-74f7784cd9-7dlpb\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") " pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:39.673134 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.673110    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-tmp-dir\") pod \"ipp-route-2864b6-kserve-74f7784cd9-7dlpb\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") " pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:39.673244 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.673228    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-dshm\") pod \"ipp-route-2864b6-kserve-74f7784cd9-7dlpb\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") " pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:39.673324 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.673279    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztxll\" (UniqueName: \"kubernetes.io/projected/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-kube-api-access-ztxll\") pod \"ipp-route-2864b6-kserve-74f7784cd9-7dlpb\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") " pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:39.774364 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.774277    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-tls-certs\") pod \"ipp-route-2864b6-kserve-74f7784cd9-7dlpb\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") " pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:39.774364 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.774344    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-tmp-dir\") pod \"ipp-route-2864b6-kserve-74f7784cd9-7dlpb\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") " pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:39.774587 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.774438    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-dshm\") pod \"ipp-route-2864b6-kserve-74f7784cd9-7dlpb\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") " pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:39.774587 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.774487    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztxll\" (UniqueName: \"kubernetes.io/projected/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-kube-api-access-ztxll\") pod \"ipp-route-2864b6-kserve-74f7784cd9-7dlpb\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") " pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:39.774587 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.774523    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-model-cache\") pod \"ipp-route-2864b6-kserve-74f7784cd9-7dlpb\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") " pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:39.774755 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.774596    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-home\") pod \"ipp-route-2864b6-kserve-74f7784cd9-7dlpb\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") " pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:39.775089 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.775060    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-tmp-dir\") pod \"ipp-route-2864b6-kserve-74f7784cd9-7dlpb\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") " pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:39.775089 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.775081    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-home\") pod \"ipp-route-2864b6-kserve-74f7784cd9-7dlpb\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") " pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:39.775305 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.775192    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-model-cache\") pod \"ipp-route-2864b6-kserve-74f7784cd9-7dlpb\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") " pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:39.776959 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.776930    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-dshm\") pod \"ipp-route-2864b6-kserve-74f7784cd9-7dlpb\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") " pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:39.777102 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.777085    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-tls-certs\") pod \"ipp-route-2864b6-kserve-74f7784cd9-7dlpb\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") " pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:39.842243 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:39.842209    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztxll\" (UniqueName: \"kubernetes.io/projected/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-kube-api-access-ztxll\") pod \"ipp-route-2864b6-kserve-74f7784cd9-7dlpb\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") " pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:40.131497 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:40.131462    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:40.511538 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:40.511515    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"]
Nov 01 05:23:40.513056 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:23:40.513026    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod82302c53_5b2f_4d7c_b86d_5dfd3d9a8060.slice/crio-94dba05d11ec96c5fcda139a7d2fc1e4ecbec7b6649306b9328def2ebe899332 WatchSource:0}: Error finding container 94dba05d11ec96c5fcda139a7d2fc1e4ecbec7b6649306b9328def2ebe899332: Status 404 returned error can't find the container with id 94dba05d11ec96c5fcda139a7d2fc1e4ecbec7b6649306b9328def2ebe899332
Nov 01 05:23:41.344212 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:41.344175    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb" event={"ID":"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060","Type":"ContainerStarted","Data":"9560e9b915aaa46efb922639e7d7a892c183d2094273aab33e7471545437e68b"}
Nov 01 05:23:41.344407 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:41.344237    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb" event={"ID":"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060","Type":"ContainerStarted","Data":"94dba05d11ec96c5fcda139a7d2fc1e4ecbec7b6649306b9328def2ebe899332"}
Nov 01 05:23:41.465121 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:41.465057    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb" podStartSLOduration=2.465036189 podStartE2EDuration="2.465036189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:23:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:23:41.460950828 +0000 UTC m=+1238.351326804" watchObservedRunningTime="2026-11-01 05:23:41.465036189 +0000 UTC m=+1238.355412165"
Nov 01 05:23:42.132360 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:42.132278    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:50.132698 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:50.132664    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:50.146062 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:50.146039    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:50.429962 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:23:50.429886    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:23:52.525168 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:23:52.525125    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:24:03.444909 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:03.444872    4864 scope.go:122] "RemoveContainer" containerID="44e4fabb17f10f29dd54aef82c3cda3ecf64eee6a6dbdf8ca87a372e88e697c3"
Nov 01 05:24:14.908810 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:14.907912    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8" podUID="071c5c98-548c-439b-b8f6-3b07c70df932" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 05:24:15.022810 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:15.021923    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk" podUID="b28a4792-6e4e-4f9b-80e0-737dc18ba934" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 05:24:15.186696 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:15.186581    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx" podUID="7b847150-a3e8-4210-af71-4f2ec0010c03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 05:24:22.523758 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:24:22.523720    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:24:25.905549 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:25.904377    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8" podUID="071c5c98-548c-439b-b8f6-3b07c70df932" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 05:24:25.995929 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:25.995889    4864 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk" podUID="b28a4792-6e4e-4f9b-80e0-737dc18ba934" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/health\": context deadline exceeded"
Nov 01 05:24:52.524191 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:24:52.524112    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:24:56.527343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:56.527308    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"]
Nov 01 05:24:56.531220 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:56.531205    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"
Nov 01 05:24:56.583965 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:56.583937    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-918b8d-serving-cert\""
Nov 01 05:24:56.616581 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:56.616546    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82qnm\" (UniqueName: \"kubernetes.io/projected/b94fa739-43bf-4e30-a4d2-c96f7d18d498-kube-api-access-82qnm\") pod \"maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r\" (UID: \"b94fa739-43bf-4e30-a4d2-c96f7d18d498\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"
Nov 01 05:24:56.616742 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:56.616681    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b94fa739-43bf-4e30-a4d2-c96f7d18d498-metrics-certs\") pod \"maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r\" (UID: \"b94fa739-43bf-4e30-a4d2-c96f7d18d498\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"
Nov 01 05:24:56.616837 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:56.616819    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b94fa739-43bf-4e30-a4d2-c96f7d18d498-maas-api-tls\") pod \"maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r\" (UID: \"b94fa739-43bf-4e30-a4d2-c96f7d18d498\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"
Nov 01 05:24:56.616902 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:56.616847    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b94fa739-43bf-4e30-a4d2-c96f7d18d498-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r\" (UID: \"b94fa739-43bf-4e30-a4d2-c96f7d18d498\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"
Nov 01 05:24:56.636830 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:56.636807    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"]
Nov 01 05:24:56.717702 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:56.717675    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b94fa739-43bf-4e30-a4d2-c96f7d18d498-maas-api-tls\") pod \"maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r\" (UID: \"b94fa739-43bf-4e30-a4d2-c96f7d18d498\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"
Nov 01 05:24:56.717898 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:56.717708    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b94fa739-43bf-4e30-a4d2-c96f7d18d498-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r\" (UID: \"b94fa739-43bf-4e30-a4d2-c96f7d18d498\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"
Nov 01 05:24:56.717898 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:56.717771    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82qnm\" (UniqueName: \"kubernetes.io/projected/b94fa739-43bf-4e30-a4d2-c96f7d18d498-kube-api-access-82qnm\") pod \"maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r\" (UID: \"b94fa739-43bf-4e30-a4d2-c96f7d18d498\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"
Nov 01 05:24:56.717898 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:24:56.717847    4864 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-serving-cert: secret "maas-api-e2e-praxis-a-918b8d-serving-cert" not found
Nov 01 05:24:56.717898 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:56.717867    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b94fa739-43bf-4e30-a4d2-c96f7d18d498-metrics-certs\") pod \"maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r\" (UID: \"b94fa739-43bf-4e30-a4d2-c96f7d18d498\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"
Nov 01 05:24:56.718108 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:24:56.717905    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b94fa739-43bf-4e30-a4d2-c96f7d18d498-maas-api-tls podName:b94fa739-43bf-4e30-a4d2-c96f7d18d498 nodeName:}" failed. No retries permitted until 2026-11-01 05:24:57.217890255 +0000 UTC m=+1314.108266228 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b94fa739-43bf-4e30-a4d2-c96f7d18d498-maas-api-tls") pod "maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r" (UID: "b94fa739-43bf-4e30-a4d2-c96f7d18d498") : secret "maas-api-e2e-praxis-a-918b8d-serving-cert" not found
Nov 01 05:24:56.718556 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:56.718538    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b94fa739-43bf-4e30-a4d2-c96f7d18d498-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r\" (UID: \"b94fa739-43bf-4e30-a4d2-c96f7d18d498\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"
Nov 01 05:24:56.720359 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:56.720343    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b94fa739-43bf-4e30-a4d2-c96f7d18d498-metrics-certs\") pod \"maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r\" (UID: \"b94fa739-43bf-4e30-a4d2-c96f7d18d498\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"
Nov 01 05:24:56.783895 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:56.783807    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82qnm\" (UniqueName: \"kubernetes.io/projected/b94fa739-43bf-4e30-a4d2-c96f7d18d498-kube-api-access-82qnm\") pod \"maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r\" (UID: \"b94fa739-43bf-4e30-a4d2-c96f7d18d498\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"
Nov 01 05:24:57.224458 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:57.224412    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b94fa739-43bf-4e30-a4d2-c96f7d18d498-maas-api-tls\") pod \"maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r\" (UID: \"b94fa739-43bf-4e30-a4d2-c96f7d18d498\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"
Nov 01 05:24:57.227607 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:57.227517    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b94fa739-43bf-4e30-a4d2-c96f7d18d498-maas-api-tls\") pod \"maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r\" (UID: \"b94fa739-43bf-4e30-a4d2-c96f7d18d498\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"
Nov 01 05:24:57.452880 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:57.452847    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"
Nov 01 05:24:57.874326 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:57.874293    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"]
Nov 01 05:24:58.494693 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:58.494654    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r" event={"ID":"b94fa739-43bf-4e30-a4d2-c96f7d18d498","Type":"ContainerStarted","Data":"15594a4b1a67e40ccfb9ac76542bdcb3ed48d44c6a3c7ffba70b2ac56eaef7e0"}
Nov 01 05:24:59.503194 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:59.503161    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r" event={"ID":"b94fa739-43bf-4e30-a4d2-c96f7d18d498","Type":"ContainerStarted","Data":"4f58d5c8facf50bbdfacb0e33b18d34ac05652659c0067c0326f54df626ffaa7"}
Nov 01 05:24:59.503688 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:59.503346    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"
Nov 01 05:24:59.621995 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:24:59.621944    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r" podStartSLOduration=2.493942227 podStartE2EDuration="3.621929453s" totalImagesPullingTime="1.127987226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:24:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:24:59.620470946 +0000 UTC m=+1316.510846923" watchObservedRunningTime="2026-11-01 05:24:59.621929453 +0000 UTC m=+1316.512305447"
Nov 01 05:25:00.933808 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:00.932850    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"]
Nov 01 05:25:00.939810 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:00.937904    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:00.992325 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:00.992292    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:00.992565 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:00.992542    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:00.992717 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:00.992698    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:00.992910 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:00.992890    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:00.993049 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:00.993031    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:00.993183 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:00.993164    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3695950-3ae2-48f4-a299-8a398f7640d9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:00.993328 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:00.993312    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xzv8\" (UniqueName: \"kubernetes.io/projected/e3695950-3ae2-48f4-a299-8a398f7640d9-kube-api-access-5xzv8\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:00.993502 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:00.993484    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:00.993659 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:00.993642    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:01.077823 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.076591    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"]
Nov 01 05:25:01.119896 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.117196    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:01.119896 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.117741    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:01.119896 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.117810    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:01.119896 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.117872    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:01.119896 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.117899    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:01.119896 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.117944    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3695950-3ae2-48f4-a299-8a398f7640d9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:01.119896 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.117996    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xzv8\" (UniqueName: \"kubernetes.io/projected/e3695950-3ae2-48f4-a299-8a398f7640d9-kube-api-access-5xzv8\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:01.119896 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.118076    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:01.119896 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.118138    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:01.124864 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.117618    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:01.143893 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.143851    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:01.144538 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.144498    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:01.145072 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.145054    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:01.146101 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.146082    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3695950-3ae2-48f4-a299-8a398f7640d9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:01.146521 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.146495    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:01.153341 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.153319    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:01.224889 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.224851    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:01.230088 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.230058    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xzv8\" (UniqueName: \"kubernetes.io/projected/e3695950-3ae2-48f4-a299-8a398f7640d9-kube-api-access-5xzv8\") pod \"maas-default-gateway-openshift-default-58d898bd55-px79s\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:01.254395 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.254367    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:01.696973 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.696943    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"]
Nov 01 05:25:01.698221 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:25:01.698192    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3695950_3ae2_48f4_a299_8a398f7640d9.slice/crio-01a1786acd6c8a41f64fb361375d0a5f127d22539a50ed4d47ce63e00d555414 WatchSource:0}: Error finding container 01a1786acd6c8a41f64fb361375d0a5f127d22539a50ed4d47ce63e00d555414: Status 404 returned error can't find the container with id 01a1786acd6c8a41f64fb361375d0a5f127d22539a50ed4d47ce63e00d555414
Nov 01 05:25:01.700681 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.700652    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:25:01.700768 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.700715    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:25:01.700768 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:01.700743    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:25:02.556245 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:02.556201    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s" event={"ID":"e3695950-3ae2-48f4-a299-8a398f7640d9","Type":"ContainerStarted","Data":"37ec1eff988ce87db85bd6fea734e1fbbd173bb4b966aa5f45acfbc2345040e1"}
Nov 01 05:25:02.556665 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:02.556265    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s" event={"ID":"e3695950-3ae2-48f4-a299-8a398f7640d9","Type":"ContainerStarted","Data":"01a1786acd6c8a41f64fb361375d0a5f127d22539a50ed4d47ce63e00d555414"}
Nov 01 05:25:02.695666 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:02.695604    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s" podStartSLOduration=2.695574223 podStartE2EDuration="2.695574223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:25:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:25:02.694161993 +0000 UTC m=+1319.584537970" watchObservedRunningTime="2026-11-01 05:25:02.695574223 +0000 UTC m=+1319.585950198"
Nov 01 05:25:03.255191 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:03.255165    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:03.256441 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:03.256417    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-px79s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 05:25:03.256562 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:03.256475    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s" podUID="e3695950-3ae2-48f4-a299-8a398f7640d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 05:25:04.255276 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:04.255240    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-px79s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 05:25:04.255683 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:04.255294    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s" podUID="e3695950-3ae2-48f4-a299-8a398f7640d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 05:25:05.254823 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:05.254774    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-px79s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 05:25:05.255026 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:05.254849    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s" podUID="e3695950-3ae2-48f4-a299-8a398f7640d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 05:25:05.548778 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:05.548699    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"
Nov 01 05:25:06.255453 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:06.255419    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-px79s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 05:25:06.255632 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:06.255472    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s" podUID="e3695950-3ae2-48f4-a299-8a398f7640d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 05:25:07.254920 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:07.254886    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-px79s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 05:25:07.255349 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:07.254950    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s" podUID="e3695950-3ae2-48f4-a299-8a398f7640d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 05:25:08.255697 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:08.255663    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-px79s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 05:25:08.256188 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:08.255713    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s" podUID="e3695950-3ae2-48f4-a299-8a398f7640d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 05:25:09.255372 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:09.255261    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-px79s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 05:25:09.255372 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:09.255337    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s" podUID="e3695950-3ae2-48f4-a299-8a398f7640d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 05:25:09.627216 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:09.627181    4864 generic.go:356] "Generic (PLEG): container finished" podID="899c1e60-3610-4825-8803-854638f8f9fd" containerID="33da4e010b7c363acc2fe6453972d71c7b5b8723f7bc4c97952ebf0956e22444" exitCode=0
Nov 01 05:25:09.627692 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:09.627264    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-rv56j-mw2kw" event={"ID":"899c1e60-3610-4825-8803-854638f8f9fd","Type":"ContainerDied","Data":"33da4e010b7c363acc2fe6453972d71c7b5b8723f7bc4c97952ebf0956e22444"}
Nov 01 05:25:10.260441 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:10.260402    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:10.260905 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:10.260878    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:10.261627 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:10.261606    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:25:10.824355 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:10.824328    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-rv56j-mw2kw"
Nov 01 05:25:10.866904 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:10.866865    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/899c1e60-3610-4825-8803-854638f8f9fd-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/899c1e60-3610-4825-8803-854638f8f9fd-archives-path\") pod \"899c1e60-3610-4825-8803-854638f8f9fd\" (UID: \"899c1e60-3610-4825-8803-854638f8f9fd\") "
Nov 01 05:25:10.867075 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:10.866928    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/899c1e60-3610-4825-8803-854638f8f9fd-kube-api-access-9c5vr\" (UniqueName: \"kubernetes.io/projected/899c1e60-3610-4825-8803-854638f8f9fd-kube-api-access-9c5vr\") pod \"899c1e60-3610-4825-8803-854638f8f9fd\" (UID: \"899c1e60-3610-4825-8803-854638f8f9fd\") "
Nov 01 05:25:10.867075 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:10.866990    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/899c1e60-3610-4825-8803-854638f8f9fd-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/899c1e60-3610-4825-8803-854638f8f9fd-service-ca-bundle\") pod \"899c1e60-3610-4825-8803-854638f8f9fd\" (UID: \"899c1e60-3610-4825-8803-854638f8f9fd\") "
Nov 01 05:25:10.867165 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:10.867079    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/899c1e60-3610-4825-8803-854638f8f9fd-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/899c1e60-3610-4825-8803-854638f8f9fd-trusted-ca-bundle\") pod \"899c1e60-3610-4825-8803-854638f8f9fd\" (UID: \"899c1e60-3610-4825-8803-854638f8f9fd\") "
Nov 01 05:25:10.867270 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:10.867200    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/899c1e60-3610-4825-8803-854638f8f9fd-archives-path" pod "899c1e60-3610-4825-8803-854638f8f9fd" (UID: "899c1e60-3610-4825-8803-854638f8f9fd"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:25:10.867386 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:10.867364    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/899c1e60-3610-4825-8803-854638f8f9fd-service-ca-bundle" pod "899c1e60-3610-4825-8803-854638f8f9fd" (UID: "899c1e60-3610-4825-8803-854638f8f9fd"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:25:10.867459 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:10.867442    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/899c1e60-3610-4825-8803-854638f8f9fd-trusted-ca-bundle" pod "899c1e60-3610-4825-8803-854638f8f9fd" (UID: "899c1e60-3610-4825-8803-854638f8f9fd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:25:10.867599 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:10.867585    4864 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/899c1e60-3610-4825-8803-854638f8f9fd-service-ca-bundle\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:25:10.867635 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:10.867602    4864 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/899c1e60-3610-4825-8803-854638f8f9fd-trusted-ca-bundle\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:25:10.867635 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:10.867612    4864 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/899c1e60-3610-4825-8803-854638f8f9fd-archives-path\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:25:10.869321 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:10.869302    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/899c1e60-3610-4825-8803-854638f8f9fd-kube-api-access-9c5vr" pod "899c1e60-3610-4825-8803-854638f8f9fd" (UID: "899c1e60-3610-4825-8803-854638f8f9fd"). InnerVolumeSpecName "kube-api-access-9c5vr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:25:10.968864 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:10.968757    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9c5vr\" (UniqueName: \"kubernetes.io/projected/899c1e60-3610-4825-8803-854638f8f9fd-kube-api-access-9c5vr\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:25:11.641119 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:11.641087    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-rv56j-mw2kw"
Nov 01 05:25:11.641119 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:11.641094    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-rv56j-mw2kw" event={"ID":"899c1e60-3610-4825-8803-854638f8f9fd","Type":"ContainerDied","Data":"0442fb16b3b72ada69e9ef16b11d273171cea16fd54e1454720d4186a066264c"}
Nov 01 05:25:11.641343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:11.641146    4864 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0442fb16b3b72ada69e9ef16b11d273171cea16fd54e1454720d4186a066264c"
Nov 01 05:25:22.525857 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:25:22.525819    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:25:32.759002 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:32.758967    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"]
Nov 01 05:25:47.780657 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:47.780600    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb" podUID="82302c53-5b2f-4d7c-b86d-5dfd3d9a8060" containerName="main" containerID="cri-o://9560e9b915aaa46efb922639e7d7a892c183d2094273aab33e7471545437e68b" gracePeriod=45
Nov 01 05:25:47.975243 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:47.975214    4864 generic.go:356] "Generic (PLEG): container finished" podID="82302c53-5b2f-4d7c-b86d-5dfd3d9a8060" containerID="9560e9b915aaa46efb922639e7d7a892c183d2094273aab33e7471545437e68b" exitCode=0
Nov 01 05:25:47.975416 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:47.975251    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb" event={"ID":"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060","Type":"ContainerDied","Data":"9560e9b915aaa46efb922639e7d7a892c183d2094273aab33e7471545437e68b"}
Nov 01 05:25:48.057280 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.057252    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:25:48.146683 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.146653    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-home\" (UniqueName: \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-home\") pod \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") "
Nov 01 05:25:48.146904 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.146726    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-tmp-dir\") pod \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") "
Nov 01 05:25:48.146904 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.146752    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-kube-api-access-ztxll\" (UniqueName: \"kubernetes.io/projected/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-kube-api-access-ztxll\") pod \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") "
Nov 01 05:25:48.146904 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.146848    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-tls-certs\" (UniqueName: \"kubernetes.io/secret/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-tls-certs\") pod \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") "
Nov 01 05:25:48.146904 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.146892    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-model-cache\") pod \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") "
Nov 01 05:25:48.147118 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.146961    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-dshm\" (UniqueName: \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-dshm\") pod \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\" (UID: \"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060\") "
Nov 01 05:25:48.147118 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.146958    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-tmp-dir" pod "82302c53-5b2f-4d7c-b86d-5dfd3d9a8060" (UID: "82302c53-5b2f-4d7c-b86d-5dfd3d9a8060"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:25:48.147118 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.146997    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-home" pod "82302c53-5b2f-4d7c-b86d-5dfd3d9a8060" (UID: "82302c53-5b2f-4d7c-b86d-5dfd3d9a8060"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:25:48.147262 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.147216    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-model-cache" pod "82302c53-5b2f-4d7c-b86d-5dfd3d9a8060" (UID: "82302c53-5b2f-4d7c-b86d-5dfd3d9a8060"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:25:48.147436 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.147415    4864 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-home\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:25:48.147436 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.147434    4864 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-tmp-dir\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:25:48.147436 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.147442    4864 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-model-cache\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:25:48.149283 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.149259    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-dshm" pod "82302c53-5b2f-4d7c-b86d-5dfd3d9a8060" (UID: "82302c53-5b2f-4d7c-b86d-5dfd3d9a8060"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:25:48.149283 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.149269    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-tls-certs" pod "82302c53-5b2f-4d7c-b86d-5dfd3d9a8060" (UID: "82302c53-5b2f-4d7c-b86d-5dfd3d9a8060"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:25:48.149398 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.149259    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-kube-api-access-ztxll" pod "82302c53-5b2f-4d7c-b86d-5dfd3d9a8060" (UID: "82302c53-5b2f-4d7c-b86d-5dfd3d9a8060"). InnerVolumeSpecName "kube-api-access-ztxll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:25:48.248625 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.248597    4864 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-tls-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:25:48.248625 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.248621    4864 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-dshm\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:25:48.248625 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.248631    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztxll\" (UniqueName: \"kubernetes.io/projected/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060-kube-api-access-ztxll\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:25:48.990497 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.990452    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb" event={"ID":"82302c53-5b2f-4d7c-b86d-5dfd3d9a8060","Type":"ContainerDied","Data":"94dba05d11ec96c5fcda139a7d2fc1e4ecbec7b6649306b9328def2ebe899332"}
Nov 01 05:25:48.990497 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.990500    4864 scope.go:122] "RemoveContainer" containerID="9560e9b915aaa46efb922639e7d7a892c183d2094273aab33e7471545437e68b"
Nov 01 05:25:48.991040 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:48.990506    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"
Nov 01 05:25:49.158641 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.158598    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"]
Nov 01 05:25:49.206333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.206304    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-2864b6/ipp-route-2864b6-kserve-74f7784cd9-7dlpb"]
Nov 01 05:25:49.247889 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.247826    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="82302c53-5b2f-4d7c-b86d-5dfd3d9a8060" path="/var/lib/kubelet/pods/82302c53-5b2f-4d7c-b86d-5dfd3d9a8060/volumes"
Nov 01 05:25:49.512441 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.512370    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"]
Nov 01 05:25:49.512933 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.512918    4864 cpu_manager.go:414] "containerMap: removing container" podUID="82302c53-5b2f-4d7c-b86d-5dfd3d9a8060" containerName="main"
Nov 01 05:25:49.512980 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.512935    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82302c53-5b2f-4d7c-b86d-5dfd3d9a8060" containerName="main"
Nov 01 05:25:49.512980 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.512957    4864 cpu_manager.go:414] "containerMap: removing container" podUID="899c1e60-3610-4825-8803-854638f8f9fd" containerName="insights-gathering"
Nov 01 05:25:49.512980 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.512962    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="899c1e60-3610-4825-8803-854638f8f9fd" containerName="insights-gathering"
Nov 01 05:25:49.513071 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.513040    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="82302c53-5b2f-4d7c-b86d-5dfd3d9a8060" containerName="main"
Nov 01 05:25:49.513071 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.513049    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="899c1e60-3610-4825-8803-854638f8f9fd" containerName="insights-gathering"
Nov 01 05:25:49.516268 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.516249    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"
Nov 01 05:25:49.568499 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.568472    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-dee6aa-serving-cert\""
Nov 01 05:25:49.576622 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.576577    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"]
Nov 01 05:25:49.676562 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.676524    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-maas-api-tls\") pod \"maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4\" (UID: \"48c3060a-f1c5-4ca4-8955-3b80c61eacbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"
Nov 01 05:25:49.676722 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.676632    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4\" (UID: \"48c3060a-f1c5-4ca4-8955-3b80c61eacbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"
Nov 01 05:25:49.676722 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.676660    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxq5t\" (UniqueName: \"kubernetes.io/projected/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-kube-api-access-zxq5t\") pod \"maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4\" (UID: \"48c3060a-f1c5-4ca4-8955-3b80c61eacbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"
Nov 01 05:25:49.676845 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.676764    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-metrics-certs\") pod \"maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4\" (UID: \"48c3060a-f1c5-4ca4-8955-3b80c61eacbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"
Nov 01 05:25:49.777950 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.777860    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-maas-api-tls\") pod \"maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4\" (UID: \"48c3060a-f1c5-4ca4-8955-3b80c61eacbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"
Nov 01 05:25:49.777950 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.777939    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4\" (UID: \"48c3060a-f1c5-4ca4-8955-3b80c61eacbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"
Nov 01 05:25:49.778193 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.777969    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxq5t\" (UniqueName: \"kubernetes.io/projected/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-kube-api-access-zxq5t\") pod \"maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4\" (UID: \"48c3060a-f1c5-4ca4-8955-3b80c61eacbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"
Nov 01 05:25:49.778193 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:25:49.778043    4864 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-serving-cert: secret "maas-api-e2e-extmod-a-dee6aa-serving-cert" not found
Nov 01 05:25:49.778193 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:25:49.778134    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-maas-api-tls podName:48c3060a-f1c5-4ca4-8955-3b80c61eacbd nodeName:}" failed. No retries permitted until 2026-11-01 05:25:50.278112506 +0000 UTC m=+1367.168488473 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-maas-api-tls") pod "maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4" (UID: "48c3060a-f1c5-4ca4-8955-3b80c61eacbd") : secret "maas-api-e2e-extmod-a-dee6aa-serving-cert" not found
Nov 01 05:25:49.778193 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.778054    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-metrics-certs\") pod \"maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4\" (UID: \"48c3060a-f1c5-4ca4-8955-3b80c61eacbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"
Nov 01 05:25:49.778718 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.778698    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4\" (UID: \"48c3060a-f1c5-4ca4-8955-3b80c61eacbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"
Nov 01 05:25:49.780650 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.780629    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-metrics-certs\") pod \"maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4\" (UID: \"48c3060a-f1c5-4ca4-8955-3b80c61eacbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"
Nov 01 05:25:49.841554 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:49.841522    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxq5t\" (UniqueName: \"kubernetes.io/projected/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-kube-api-access-zxq5t\") pod \"maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4\" (UID: \"48c3060a-f1c5-4ca4-8955-3b80c61eacbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"
Nov 01 05:25:50.284309 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:50.284271    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-maas-api-tls\") pod \"maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4\" (UID: \"48c3060a-f1c5-4ca4-8955-3b80c61eacbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"
Nov 01 05:25:50.287278 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:50.287254    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-maas-api-tls\") pod \"maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4\" (UID: \"48c3060a-f1c5-4ca4-8955-3b80c61eacbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"
Nov 01 05:25:50.429755 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:50.429727    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"
Nov 01 05:25:50.808995 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:50.808965    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"]
Nov 01 05:25:50.809886 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:25:50.809854    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48c3060a_f1c5_4ca4_8955_3b80c61eacbd.slice/crio-2b4d13f87c265fa78c7b04f5098c48e1408e8b88012c28485d9563ff4cabbafb WatchSource:0}: Error finding container 2b4d13f87c265fa78c7b04f5098c48e1408e8b88012c28485d9563ff4cabbafb: Status 404 returned error can't find the container with id 2b4d13f87c265fa78c7b04f5098c48e1408e8b88012c28485d9563ff4cabbafb
Nov 01 05:25:51.011861 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:51.011544    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4" event={"ID":"48c3060a-f1c5-4ca4-8955-3b80c61eacbd","Type":"ContainerStarted","Data":"2b4d13f87c265fa78c7b04f5098c48e1408e8b88012c28485d9563ff4cabbafb"}
Nov 01 05:25:52.020117 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:52.020040    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4" event={"ID":"48c3060a-f1c5-4ca4-8955-3b80c61eacbd","Type":"ContainerStarted","Data":"9fca6249a2e9333d6bbf022d51330919346d8124b7bf5621133dd734a1344d89"}
Nov 01 05:25:52.020117 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:52.020108    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"
Nov 01 05:25:52.139561 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:52.139504    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4" podStartSLOduration=2.251340531 podStartE2EDuration="3.139485291s" totalImagesPullingTime="888.14476ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:25:52.137284528 +0000 UTC m=+1369.027660505" watchObservedRunningTime="2026-11-01 05:25:52.139485291 +0000 UTC m=+1369.029861268"
Nov 01 05:25:52.525007 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:25:52.524971    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:25:58.032384 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:25:58.032356    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"
Nov 01 05:26:22.526479 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:26:22.526401    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:26:45.330743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.330706    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"]
Nov 01 05:26:45.334992 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.334969    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.392344 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.392283    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-c6e99c-openshift-default-dockercfg-qgjc6\""
Nov 01 05:26:45.427318 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.427273    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-credential-socket\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.427665 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.427635    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-istio-data\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.428365 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.427956    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-istio-envoy\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.428365 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.428008    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/702e5c95-5de0-4329-bae0-404d4d65a941-istio-token\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.428365 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.428041    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/702e5c95-5de0-4329-bae0-404d4d65a941-istiod-ca-cert\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.428365 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.428093    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hmq5\" (UniqueName: \"kubernetes.io/projected/702e5c95-5de0-4329-bae0-404d4d65a941-kube-api-access-8hmq5\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.428365 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.428268    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-workload-socket\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.428365 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.428306    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/702e5c95-5de0-4329-bae0-404d4d65a941-istio-podinfo\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.429231 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.429178    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-workload-certs\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.453837 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.450490    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"]
Nov 01 05:26:45.530409 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.530372    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-istio-envoy\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.530409 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.530413    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/702e5c95-5de0-4329-bae0-404d4d65a941-istio-token\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.530742 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.530429    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/702e5c95-5de0-4329-bae0-404d4d65a941-istiod-ca-cert\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.530742 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.530468    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hmq5\" (UniqueName: \"kubernetes.io/projected/702e5c95-5de0-4329-bae0-404d4d65a941-kube-api-access-8hmq5\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.530742 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.530593    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-workload-socket\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.530742 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.530615    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/702e5c95-5de0-4329-bae0-404d4d65a941-istio-podinfo\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.530742 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.530682    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-workload-certs\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.531079 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.531010    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-workload-socket\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.531143 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.531068    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-credential-socket\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.531200 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.531137    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-istio-data\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.531200 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.531075    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-workload-certs\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.531200 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.531295    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-credential-socket\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.531656 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.531633    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-istio-data\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.532315 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.532273    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/702e5c95-5de0-4329-bae0-404d4d65a941-istiod-ca-cert\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.533989 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.533966    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/702e5c95-5de0-4329-bae0-404d4d65a941-istio-podinfo\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.533989 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.533979    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-istio-envoy\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.595236 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.595157    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/702e5c95-5de0-4329-bae0-404d4d65a941-istio-token\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.597646 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.597616    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hmq5\" (UniqueName: \"kubernetes.io/projected/702e5c95-5de0-4329-bae0-404d4d65a941-kube-api-access-8hmq5\") pod \"e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") " pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:45.655354 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:45.655326    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:46.096006 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.095974    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"]
Nov 01 05:26:46.097026 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:26:46.096997    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod702e5c95_5de0_4329_bae0_404d4d65a941.slice/crio-4597466e518e0419e476aaf48d770cd2ce0f1613c82a536d24071401ba427a0e WatchSource:0}: Error finding container 4597466e518e0419e476aaf48d770cd2ce0f1613c82a536d24071401ba427a0e: Status 404 returned error can't find the container with id 4597466e518e0419e476aaf48d770cd2ce0f1613c82a536d24071401ba427a0e
Nov 01 05:26:46.099877 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.099840    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:26:46.099967 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.099907    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:26:46.099967 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.099936    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:26:46.267397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.267359    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"]
Nov 01 05:26:46.271416 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.271400    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"
Nov 01 05:26:46.323270 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.323238    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-ca7a55-serving-cert\""
Nov 01 05:26:46.332159 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.332133    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"]
Nov 01 05:26:46.342331 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.342303    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89p2d\" (UniqueName: \"kubernetes.io/projected/cc0858b2-dff5-49b3-b702-514824d14d14-kube-api-access-89p2d\") pod \"maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9\" (UID: \"cc0858b2-dff5-49b3-b702-514824d14d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"
Nov 01 05:26:46.342530 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.342419    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc0858b2-dff5-49b3-b702-514824d14d14-metrics-certs\") pod \"maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9\" (UID: \"cc0858b2-dff5-49b3-b702-514824d14d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"
Nov 01 05:26:46.342618 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.342598    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc0858b2-dff5-49b3-b702-514824d14d14-maas-api-tls\") pod \"maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9\" (UID: \"cc0858b2-dff5-49b3-b702-514824d14d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"
Nov 01 05:26:46.342692 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.342649    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc0858b2-dff5-49b3-b702-514824d14d14-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9\" (UID: \"cc0858b2-dff5-49b3-b702-514824d14d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"
Nov 01 05:26:46.443689 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.443594    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89p2d\" (UniqueName: \"kubernetes.io/projected/cc0858b2-dff5-49b3-b702-514824d14d14-kube-api-access-89p2d\") pod \"maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9\" (UID: \"cc0858b2-dff5-49b3-b702-514824d14d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"
Nov 01 05:26:46.443689 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.443645    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc0858b2-dff5-49b3-b702-514824d14d14-metrics-certs\") pod \"maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9\" (UID: \"cc0858b2-dff5-49b3-b702-514824d14d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"
Nov 01 05:26:46.443983 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.443733    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc0858b2-dff5-49b3-b702-514824d14d14-maas-api-tls\") pod \"maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9\" (UID: \"cc0858b2-dff5-49b3-b702-514824d14d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"
Nov 01 05:26:46.443983 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.443752    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc0858b2-dff5-49b3-b702-514824d14d14-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9\" (UID: \"cc0858b2-dff5-49b3-b702-514824d14d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"
Nov 01 05:26:46.444240 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:26:46.444217    4864 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-serving-cert: secret "maas-api-e2e-praxis-b-ca7a55-serving-cert" not found
Nov 01 05:26:46.444427 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:26:46.444406    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc0858b2-dff5-49b3-b702-514824d14d14-maas-api-tls podName:cc0858b2-dff5-49b3-b702-514824d14d14 nodeName:}" failed. No retries permitted until 2026-11-01 05:26:46.944387249 +0000 UTC m=+1423.834763220 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cc0858b2-dff5-49b3-b702-514824d14d14-maas-api-tls") pod "maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9" (UID: "cc0858b2-dff5-49b3-b702-514824d14d14") : secret "maas-api-e2e-praxis-b-ca7a55-serving-cert" not found
Nov 01 05:26:46.444590 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.444567    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc0858b2-dff5-49b3-b702-514824d14d14-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9\" (UID: \"cc0858b2-dff5-49b3-b702-514824d14d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"
Nov 01 05:26:46.445704 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.445679    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft" event={"ID":"702e5c95-5de0-4329-bae0-404d4d65a941","Type":"ContainerStarted","Data":"8d9fe1177dbd35d166b45319dbf191e197230b13b42d2e12cc3ed1f57341acd7"}
Nov 01 05:26:46.445837 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.445725    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft" event={"ID":"702e5c95-5de0-4329-bae0-404d4d65a941","Type":"ContainerStarted","Data":"4597466e518e0419e476aaf48d770cd2ce0f1613c82a536d24071401ba427a0e"}
Nov 01 05:26:46.446817 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.446771    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc0858b2-dff5-49b3-b702-514824d14d14-metrics-certs\") pod \"maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9\" (UID: \"cc0858b2-dff5-49b3-b702-514824d14d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"
Nov 01 05:26:46.505210 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.505180    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89p2d\" (UniqueName: \"kubernetes.io/projected/cc0858b2-dff5-49b3-b702-514824d14d14-kube-api-access-89p2d\") pod \"maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9\" (UID: \"cc0858b2-dff5-49b3-b702-514824d14d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"
Nov 01 05:26:46.568882 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.568822    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft" podStartSLOduration=1.568779218 podStartE2EDuration="1.568779218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:26:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:26:46.567631935 +0000 UTC m=+1423.458007912" watchObservedRunningTime="2026-11-01 05:26:46.568779218 +0000 UTC m=+1423.459155194"
Nov 01 05:26:46.656798 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.656747    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:46.950354 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.950320    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc0858b2-dff5-49b3-b702-514824d14d14-maas-api-tls\") pod \"maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9\" (UID: \"cc0858b2-dff5-49b3-b702-514824d14d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"
Nov 01 05:26:46.952971 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:46.952950    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc0858b2-dff5-49b3-b702-514824d14d14-maas-api-tls\") pod \"maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9\" (UID: \"cc0858b2-dff5-49b3-b702-514824d14d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"
Nov 01 05:26:47.184708 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:47.184676    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"
Nov 01 05:26:47.568155 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:47.568133    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"]
Nov 01 05:26:47.569544 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:26:47.569518    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc0858b2_dff5_49b3_b702_514824d14d14.slice/crio-98acae046bf4014da0a4c2ced5ca20b0aaa807f1670a91f55d67d1795c01b93e WatchSource:0}: Error finding container 98acae046bf4014da0a4c2ced5ca20b0aaa807f1670a91f55d67d1795c01b93e: Status 404 returned error can't find the container with id 98acae046bf4014da0a4c2ced5ca20b0aaa807f1670a91f55d67d1795c01b93e
Nov 01 05:26:47.661772 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:47.661744    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:48.463810 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:48.463247    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9" event={"ID":"cc0858b2-dff5-49b3-b702-514824d14d14","Type":"ContainerStarted","Data":"98acae046bf4014da0a4c2ced5ca20b0aaa807f1670a91f55d67d1795c01b93e"}
Nov 01 05:26:48.463810 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:48.463670    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:48.464756 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:48.464736    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:26:49.471299 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:49.471256    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9" event={"ID":"cc0858b2-dff5-49b3-b702-514824d14d14","Type":"ContainerStarted","Data":"77dbbeea86e55947624b2eceaaa17c22757fabf08ef19805a9ebc68b8fd66a93"}
Nov 01 05:26:49.471755 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:49.471439    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"
Nov 01 05:26:49.594473 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:49.594417    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9" podStartSLOduration=2.760989688 podStartE2EDuration="3.59435251s" totalImagesPullingTime="833.362822ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:26:49.591819154 +0000 UTC m=+1426.482195124" watchObservedRunningTime="2026-11-01 05:26:49.59435251 +0000 UTC m=+1426.484728492"
Nov 01 05:26:52.524225 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:26:52.524177    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:26:55.483825 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:26:55.483779    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"
Nov 01 05:27:22.525215 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:27:22.525177    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:27:52.523934 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:27:52.523900    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:27:58.787811 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:58.787097    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"]
Nov 01 05:27:58.794102 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:58.794078    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:27:58.834553 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:58.834524    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"]
Nov 01 05:27:58.888116 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:58.888079    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-praxis-a-918b8d\"/\"openshift-service-ca.crt\""
Nov 01 05:27:58.896046 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:58.896021    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-praxis-a-918b8d\"/\"kube-root-ca.crt\""
Nov 01 05:27:58.896180 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:58.896156    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-praxis-a-918b8d\"/\"default-dockercfg-nmqq7\""
Nov 01 05:27:58.896235 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:58.896213    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-praxis-a-918b8d\"/\"praxis-route-918b8d-kserve-self-signed-certs\""
Nov 01 05:27:58.921528 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:58.921490    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5d934f5-3410-4c7e-9f93-ede52dde324a-tls-certs\") pod \"praxis-route-918b8d-kserve-6f9b465557-h8n27\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") " pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:27:58.921729 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:58.921596    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-model-cache\") pod \"praxis-route-918b8d-kserve-6f9b465557-h8n27\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") " pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:27:58.921729 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:58.921701    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd669\" (UniqueName: \"kubernetes.io/projected/a5d934f5-3410-4c7e-9f93-ede52dde324a-kube-api-access-hd669\") pod \"praxis-route-918b8d-kserve-6f9b465557-h8n27\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") " pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:27:58.921913 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:58.921730    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-tmp-dir\") pod \"praxis-route-918b8d-kserve-6f9b465557-h8n27\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") " pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:27:58.921913 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:58.921764    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-dshm\") pod \"praxis-route-918b8d-kserve-6f9b465557-h8n27\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") " pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:27:58.921913 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:58.921863    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-home\") pod \"praxis-route-918b8d-kserve-6f9b465557-h8n27\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") " pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:27:59.022736 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:59.022697    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-model-cache\") pod \"praxis-route-918b8d-kserve-6f9b465557-h8n27\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") " pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:27:59.022966 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:59.022835    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd669\" (UniqueName: \"kubernetes.io/projected/a5d934f5-3410-4c7e-9f93-ede52dde324a-kube-api-access-hd669\") pod \"praxis-route-918b8d-kserve-6f9b465557-h8n27\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") " pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:27:59.022966 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:59.022867    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-tmp-dir\") pod \"praxis-route-918b8d-kserve-6f9b465557-h8n27\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") " pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:27:59.022966 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:59.022952    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-dshm\") pod \"praxis-route-918b8d-kserve-6f9b465557-h8n27\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") " pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:27:59.023334 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:59.023312    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-tmp-dir\") pod \"praxis-route-918b8d-kserve-6f9b465557-h8n27\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") " pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:27:59.023403 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:59.023361    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-home\") pod \"praxis-route-918b8d-kserve-6f9b465557-h8n27\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") " pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:27:59.023516 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:59.023497    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5d934f5-3410-4c7e-9f93-ede52dde324a-tls-certs\") pod \"praxis-route-918b8d-kserve-6f9b465557-h8n27\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") " pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:27:59.023680 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:59.023658    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-home\") pod \"praxis-route-918b8d-kserve-6f9b465557-h8n27\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") " pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:27:59.023830 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:59.023808    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-model-cache\") pod \"praxis-route-918b8d-kserve-6f9b465557-h8n27\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") " pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:27:59.025978 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:59.025956    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-dshm\") pod \"praxis-route-918b8d-kserve-6f9b465557-h8n27\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") " pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:27:59.026691 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:59.026667    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5d934f5-3410-4c7e-9f93-ede52dde324a-tls-certs\") pod \"praxis-route-918b8d-kserve-6f9b465557-h8n27\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") " pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:27:59.085898 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:59.085871    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd669\" (UniqueName: \"kubernetes.io/projected/a5d934f5-3410-4c7e-9f93-ede52dde324a-kube-api-access-hd669\") pod \"praxis-route-918b8d-kserve-6f9b465557-h8n27\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") " pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:27:59.111661 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:59.111449    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:27:59.551383 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:59.551354    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"]
Nov 01 05:27:59.977761 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:59.977681    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27" event={"ID":"a5d934f5-3410-4c7e-9f93-ede52dde324a","Type":"ContainerStarted","Data":"b816434d3077eff602da9257be27a895d2b02b48171ec6b228729964e79f5c0e"}
Nov 01 05:27:59.977761 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:27:59.977731    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27" event={"ID":"a5d934f5-3410-4c7e-9f93-ede52dde324a","Type":"ContainerStarted","Data":"eef65c013db78b619c9f5d56dd5a4abf9ee880d86054d1f558e38be6d90109b0"}
Nov 01 05:28:00.098857 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:00.098796    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27" podStartSLOduration=2.098762164 podStartE2EDuration="2.098762164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:27:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:28:00.097248488 +0000 UTC m=+1496.987624490" watchObservedRunningTime="2026-11-01 05:28:00.098762164 +0000 UTC m=+1496.989138144"
Nov 01 05:28:00.112416 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:00.112386    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:28:03.670148 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:03.670120    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:28:03.675756 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:03.675732    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:28:03.679842 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:03.679815    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:28:03.684801 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:03.684771    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:28:04.132299 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:04.130759    4864 patch_prober.go:29] interesting pod/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": read tcp 10.132.0.2:51270->10.132.0.79:15021: read: connection reset by peer" start-of-body=
Nov 01 05:28:04.132299 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:04.130853    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft" podUID="702e5c95-5de0-4329-bae0-404d4d65a941" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": read tcp 10.132.0.2:51270->10.132.0.79:15021: read: connection reset by peer"
Nov 01 05:28:05.020595 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:05.020553    4864 generic.go:356] "Generic (PLEG): container finished" podID="702e5c95-5de0-4329-bae0-404d4d65a941" containerID="8d9fe1177dbd35d166b45319dbf191e197230b13b42d2e12cc3ed1f57341acd7" exitCode=0
Nov 01 05:28:05.021013 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:05.020605    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft" event={"ID":"702e5c95-5de0-4329-bae0-404d4d65a941","Type":"ContainerDied","Data":"8d9fe1177dbd35d166b45319dbf191e197230b13b42d2e12cc3ed1f57341acd7"}
Nov 01 05:28:05.021092 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:05.021073    4864 scope.go:122] "RemoveContainer" containerID="8d9fe1177dbd35d166b45319dbf191e197230b13b42d2e12cc3ed1f57341acd7"
Nov 01 05:28:05.022691 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:05.022665    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:28:05.022755 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:05.022732    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:28:05.022833 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:05.022764    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:28:06.030588 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:06.030558    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft" event={"ID":"702e5c95-5de0-4329-bae0-404d4d65a941","Type":"ContainerStarted","Data":"48c337a1fd2312924241a6ec34c583defe054e0246ee34e71ec2c8a4975e3d1d"}
Nov 01 05:28:06.656105 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:06.656070    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:28:06.656293 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:06.656249    4864 patch_prober.go:29] interesting pod/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 01 05:28:06.656344 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:06.656293    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft" podUID="702e5c95-5de0-4329-bae0-404d4d65a941" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 01 05:28:07.657826 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:07.656941    4864 patch_prober.go:29] interesting pod/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 01 05:28:07.657826 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:07.657031    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft" podUID="702e5c95-5de0-4329-bae0-404d4d65a941" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 01 05:28:08.656054 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:08.656016    4864 patch_prober.go:29] interesting pod/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 01 05:28:08.656249 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:08.656079    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft" podUID="702e5c95-5de0-4329-bae0-404d4d65a941" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 01 05:28:09.113000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:09.112955    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:28:09.129267 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:09.129239    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:28:09.656030 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:09.655995    4864 patch_prober.go:29] interesting pod/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 01 05:28:09.656190 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:09.656051    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft" podUID="702e5c95-5de0-4329-bae0-404d4d65a941" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 01 05:28:10.095601 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:10.095569    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:28:10.656428 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:10.656392    4864 patch_prober.go:29] interesting pod/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 01 05:28:10.656875 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:10.656456    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft" podUID="702e5c95-5de0-4329-bae0-404d4d65a941" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 01 05:28:11.660048 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:11.660019    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:28:11.660447 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:11.660424    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:28:11.661183 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:11.661161    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 05:28:13.072983 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:13.072941    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"]
Nov 01 05:28:13.073490 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:13.073138    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z" podUID="d74d58db-55bb-40f4-8aa9-588fab2d1a49" containerName="maas-api" containerID="cri-o://89b101106cd61429d126ee5acf9fcb19f8ec2805e27308d0b34f89665ddaabfd" gracePeriod=30
Nov 01 05:28:13.435167 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:13.435140    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"
Nov 01 05:28:13.522896 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:13.522850    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d74d58db-55bb-40f4-8aa9-588fab2d1a49-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d74d58db-55bb-40f4-8aa9-588fab2d1a49-openshift-service-ca\") pod \"d74d58db-55bb-40f4-8aa9-588fab2d1a49\" (UID: \"d74d58db-55bb-40f4-8aa9-588fab2d1a49\") "
Nov 01 05:28:13.523098 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:13.523051    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d74d58db-55bb-40f4-8aa9-588fab2d1a49-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d74d58db-55bb-40f4-8aa9-588fab2d1a49-metrics-certs\") pod \"d74d58db-55bb-40f4-8aa9-588fab2d1a49\" (UID: \"d74d58db-55bb-40f4-8aa9-588fab2d1a49\") "
Nov 01 05:28:13.523163 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:13.523105    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d74d58db-55bb-40f4-8aa9-588fab2d1a49-kube-api-access-5rchx\" (UniqueName: \"kubernetes.io/projected/d74d58db-55bb-40f4-8aa9-588fab2d1a49-kube-api-access-5rchx\") pod \"d74d58db-55bb-40f4-8aa9-588fab2d1a49\" (UID: \"d74d58db-55bb-40f4-8aa9-588fab2d1a49\") "
Nov 01 05:28:13.523278 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:13.523245    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d74d58db-55bb-40f4-8aa9-588fab2d1a49-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d74d58db-55bb-40f4-8aa9-588fab2d1a49-maas-api-tls\") pod \"d74d58db-55bb-40f4-8aa9-588fab2d1a49\" (UID: \"d74d58db-55bb-40f4-8aa9-588fab2d1a49\") "
Nov 01 05:28:13.523360 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:13.523278    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d74d58db-55bb-40f4-8aa9-588fab2d1a49-openshift-service-ca" pod "d74d58db-55bb-40f4-8aa9-588fab2d1a49" (UID: "d74d58db-55bb-40f4-8aa9-588fab2d1a49"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:28:13.524624 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:13.523774    4864 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d74d58db-55bb-40f4-8aa9-588fab2d1a49-openshift-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:28:13.528600 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:13.526930    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d74d58db-55bb-40f4-8aa9-588fab2d1a49-maas-api-tls" pod "d74d58db-55bb-40f4-8aa9-588fab2d1a49" (UID: "d74d58db-55bb-40f4-8aa9-588fab2d1a49"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:28:13.529918 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:13.529887    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d74d58db-55bb-40f4-8aa9-588fab2d1a49-metrics-certs" pod "d74d58db-55bb-40f4-8aa9-588fab2d1a49" (UID: "d74d58db-55bb-40f4-8aa9-588fab2d1a49"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:28:13.530333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:13.530307    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d74d58db-55bb-40f4-8aa9-588fab2d1a49-kube-api-access-5rchx" pod "d74d58db-55bb-40f4-8aa9-588fab2d1a49" (UID: "d74d58db-55bb-40f4-8aa9-588fab2d1a49"). InnerVolumeSpecName "kube-api-access-5rchx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:28:13.625243 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:13.625209    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d74d58db-55bb-40f4-8aa9-588fab2d1a49-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:28:13.625243 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:13.625240    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rchx\" (UniqueName: \"kubernetes.io/projected/d74d58db-55bb-40f4-8aa9-588fab2d1a49-kube-api-access-5rchx\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:28:13.625243 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:13.625251    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d74d58db-55bb-40f4-8aa9-588fab2d1a49-maas-api-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:28:14.108694 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:14.108654    4864 generic.go:356] "Generic (PLEG): container finished" podID="d74d58db-55bb-40f4-8aa9-588fab2d1a49" containerID="89b101106cd61429d126ee5acf9fcb19f8ec2805e27308d0b34f89665ddaabfd" exitCode=0
Nov 01 05:28:14.112821 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:14.112746    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z" event={"ID":"d74d58db-55bb-40f4-8aa9-588fab2d1a49","Type":"ContainerDied","Data":"89b101106cd61429d126ee5acf9fcb19f8ec2805e27308d0b34f89665ddaabfd"}
Nov 01 05:28:14.112976 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:14.112871    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z" event={"ID":"d74d58db-55bb-40f4-8aa9-588fab2d1a49","Type":"ContainerDied","Data":"45181d63831ae0e7a33a3b04f437007ec21adf7e5f8af24f32ffffad7e97f08d"}
Nov 01 05:28:14.112976 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:14.112907    4864 scope.go:122] "RemoveContainer" containerID="89b101106cd61429d126ee5acf9fcb19f8ec2805e27308d0b34f89665ddaabfd"
Nov 01 05:28:14.113119 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:14.113101    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"
Nov 01 05:28:14.141330 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:14.141306    4864 scope.go:122] "RemoveContainer" containerID="89b101106cd61429d126ee5acf9fcb19f8ec2805e27308d0b34f89665ddaabfd"
Nov 01 05:28:14.141942 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:28:14.141913    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89b101106cd61429d126ee5acf9fcb19f8ec2805e27308d0b34f89665ddaabfd\": container with ID starting with 89b101106cd61429d126ee5acf9fcb19f8ec2805e27308d0b34f89665ddaabfd not found: ID does not exist" containerID="89b101106cd61429d126ee5acf9fcb19f8ec2805e27308d0b34f89665ddaabfd"
Nov 01 05:28:14.142060 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:14.141949    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89b101106cd61429d126ee5acf9fcb19f8ec2805e27308d0b34f89665ddaabfd"} err="failed to get container status \"89b101106cd61429d126ee5acf9fcb19f8ec2805e27308d0b34f89665ddaabfd\": rpc error: code = NotFound desc = could not find container \"89b101106cd61429d126ee5acf9fcb19f8ec2805e27308d0b34f89665ddaabfd\": container with ID starting with 89b101106cd61429d126ee5acf9fcb19f8ec2805e27308d0b34f89665ddaabfd not found: ID does not exist"
Nov 01 05:28:14.282245 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:14.282209    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"]
Nov 01 05:28:14.329835 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:14.329796    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5451cb-6597bc89bf-x6b8z"]
Nov 01 05:28:15.248100 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:28:15.248064    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d74d58db-55bb-40f4-8aa9-588fab2d1a49" path="/var/lib/kubelet/pods/d74d58db-55bb-40f4-8aa9-588fab2d1a49/volumes"
Nov 01 05:28:22.588810 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:28:22.587771    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:28:52.523855 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:28:52.523813    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:29:19.698268 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.698188    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"]
Nov 01 05:29:19.698816 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.698779    4864 cpu_manager.go:414] "containerMap: removing container" podUID="d74d58db-55bb-40f4-8aa9-588fab2d1a49" containerName="maas-api"
Nov 01 05:29:19.698872 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.698818    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d74d58db-55bb-40f4-8aa9-588fab2d1a49" containerName="maas-api"
Nov 01 05:29:19.698941 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.698932    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="d74d58db-55bb-40f4-8aa9-588fab2d1a49" containerName="maas-api"
Nov 01 05:29:19.703548 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.703525    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:19.807140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.807106    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-7d1ec3\"/\"kube-root-ca.crt\""
Nov 01 05:29:19.807140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.807136    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-7d1ec3\"/\"openshift-service-ca.crt\""
Nov 01 05:29:19.807362 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.807189    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-7d1ec3\"/\"default-dockercfg-4l4n6\""
Nov 01 05:29:19.807362 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.807264    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-7d1ec3\"/\"distinct-2-w0-7d1ec3-kserve-self-signed-certs\""
Nov 01 05:29:19.810896 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.810874    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"]
Nov 01 05:29:19.844337 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.844300    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61ef19b8-157e-4f82-b624-49971d51b8c7-tls-certs\") pod \"distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") " pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:19.844519 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.844416    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-dshm\") pod \"distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") " pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:19.844519 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.844472    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-home\") pod \"distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") " pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:19.844700 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.844674    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-model-cache\") pod \"distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") " pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:19.844771 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.844752    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-tmp-dir\") pod \"distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") " pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:19.844972 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.844955    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmrw7\" (UniqueName: \"kubernetes.io/projected/61ef19b8-157e-4f82-b624-49971d51b8c7-kube-api-access-xmrw7\") pod \"distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") " pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:19.946618 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.946587    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-tmp-dir\") pod \"distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") " pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:19.946832 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.946697    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmrw7\" (UniqueName: \"kubernetes.io/projected/61ef19b8-157e-4f82-b624-49971d51b8c7-kube-api-access-xmrw7\") pod \"distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") " pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:19.946832 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.946747    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61ef19b8-157e-4f82-b624-49971d51b8c7-tls-certs\") pod \"distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") " pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:19.946832 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.946823    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-dshm\") pod \"distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") " pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:19.947104 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.947079    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-tmp-dir\") pod \"distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") " pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:19.947189 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.947084    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-home\") pod \"distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") " pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:19.947189 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.947165    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-model-cache\") pod \"distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") " pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:19.947384 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.947362    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-home\") pod \"distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") " pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:19.947520 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.947496    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-model-cache\") pod \"distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") " pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:19.949459 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.949400    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-dshm\") pod \"distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") " pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:19.949562 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:19.949544    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61ef19b8-157e-4f82-b624-49971d51b8c7-tls-certs\") pod \"distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") " pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:20.007771 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:20.007733    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmrw7\" (UniqueName: \"kubernetes.io/projected/61ef19b8-157e-4f82-b624-49971d51b8c7-kube-api-access-xmrw7\") pod \"distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") " pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:20.017027 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:20.016838    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:20.459341 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:20.459312    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"]
Nov 01 05:29:20.460270 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:29:20.460245    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61ef19b8_157e_4f82_b624_49971d51b8c7.slice/crio-551d1d82b57563fd9bc6e9b2263f0230bec2c025ab2f8c860d00b334da3ad0d3 WatchSource:0}: Error finding container 551d1d82b57563fd9bc6e9b2263f0230bec2c025ab2f8c860d00b334da3ad0d3: Status 404 returned error can't find the container with id 551d1d82b57563fd9bc6e9b2263f0230bec2c025ab2f8c860d00b334da3ad0d3
Nov 01 05:29:20.648107 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:20.648073    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c" event={"ID":"61ef19b8-157e-4f82-b624-49971d51b8c7","Type":"ContainerStarted","Data":"eeba4677a8a601eecfed89397c4fa35153ed431d402a96869d0a756fb05e82bd"}
Nov 01 05:29:20.648273 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:20.648122    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c" event={"ID":"61ef19b8-157e-4f82-b624-49971d51b8c7","Type":"ContainerStarted","Data":"551d1d82b57563fd9bc6e9b2263f0230bec2c025ab2f8c860d00b334da3ad0d3"}
Nov 01 05:29:20.766520 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:20.766424    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c" podStartSLOduration=1.766409111 podStartE2EDuration="1.766409111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:29:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:29:20.765139231 +0000 UTC m=+1577.655515277" watchObservedRunningTime="2026-11-01 05:29:20.766409111 +0000 UTC m=+1577.656785094"
Nov 01 05:29:21.017934 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:21.017864    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:22.526059 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:29:22.526017    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:29:30.034119 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:30.034076    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:30.059569 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:30.059540    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:30.733304 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:30.733273    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 05:29:43.843254 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:43.843225    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"]
Nov 01 05:29:43.843672 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:43.843451    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj" podUID="5d57abe1-4de8-49e9-a864-2c48cc5ce6be" containerName="istio-proxy" containerID="cri-o://de24428edcff99a22ad6149894ac3c62ca327d27fd3295c05918f13ce7a37c28" gracePeriod=30
Nov 01 05:29:48.827945 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:48.827909    4864 patch_prober.go:29] interesting pod/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 05:29:48.828315 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:48.827966    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj" podUID="5d57abe1-4de8-49e9-a864-2c48cc5ce6be" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 05:29:49.125536 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.125512    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:29:49.296241 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.296208    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-kube-api-access-ffz58\" (UniqueName: \"kubernetes.io/projected/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-kube-api-access-ffz58\") pod \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") "
Nov 01 05:29:49.296458 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.296329    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-workload-certs\") pod \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") "
Nov 01 05:29:49.296669 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.296640    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-workload-certs" pod "5d57abe1-4de8-49e9-a864-2c48cc5ce6be" (UID: "5d57abe1-4de8-49e9-a864-2c48cc5ce6be"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:29:49.297300 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.297172    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istiod-ca-cert" pod "5d57abe1-4de8-49e9-a864-2c48cc5ce6be" (UID: "5d57abe1-4de8-49e9-a864-2c48cc5ce6be"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:29:49.297300 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.296372    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istiod-ca-cert\") pod \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") "
Nov 01 05:29:49.297468 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.297379    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-envoy\") pod \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") "
Nov 01 05:29:49.297468 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.297447    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-workload-socket\") pod \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") "
Nov 01 05:29:49.297566 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.297528    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-credential-socket\") pod \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") "
Nov 01 05:29:49.297613 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.297565    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-data\") pod \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") "
Nov 01 05:29:49.297660 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.297647    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-token\" (UniqueName: \"kubernetes.io/projected/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-token\") pod \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") "
Nov 01 05:29:49.297709 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.297674    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-podinfo\") pod \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\" (UID: \"5d57abe1-4de8-49e9-a864-2c48cc5ce6be\") "
Nov 01 05:29:49.298437 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.297992    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-credential-socket" pod "5d57abe1-4de8-49e9-a864-2c48cc5ce6be" (UID: "5d57abe1-4de8-49e9-a864-2c48cc5ce6be"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:29:49.298437 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.298208    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:29:49.298437 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.298228    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:29:49.298437 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.298243    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:29:49.300460 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.298594    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-workload-socket" pod "5d57abe1-4de8-49e9-a864-2c48cc5ce6be" (UID: "5d57abe1-4de8-49e9-a864-2c48cc5ce6be"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:29:49.300460 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.298992    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-data" pod "5d57abe1-4de8-49e9-a864-2c48cc5ce6be" (UID: "5d57abe1-4de8-49e9-a864-2c48cc5ce6be"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:29:49.302440 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.302413    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-podinfo" pod "5d57abe1-4de8-49e9-a864-2c48cc5ce6be" (UID: "5d57abe1-4de8-49e9-a864-2c48cc5ce6be"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 05:29:49.302692 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.302656    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-token" pod "5d57abe1-4de8-49e9-a864-2c48cc5ce6be" (UID: "5d57abe1-4de8-49e9-a864-2c48cc5ce6be"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:29:49.303157 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.303129    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-kube-api-access-ffz58" pod "5d57abe1-4de8-49e9-a864-2c48cc5ce6be" (UID: "5d57abe1-4de8-49e9-a864-2c48cc5ce6be"). InnerVolumeSpecName "kube-api-access-ffz58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:29:49.304069 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.304043    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-envoy" pod "5d57abe1-4de8-49e9-a864-2c48cc5ce6be" (UID: "5d57abe1-4de8-49e9-a864-2c48cc5ce6be"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:29:49.399318 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.399241    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:29:49.399318 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.399271    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:29:49.399318 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.399285    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:29:49.399318 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.399299    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:29:49.399318 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.399310    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:29:49.399582 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.399323    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffz58\" (UniqueName: \"kubernetes.io/projected/5d57abe1-4de8-49e9-a864-2c48cc5ce6be-kube-api-access-ffz58\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:29:49.866854 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.866824    4864 generic.go:356] "Generic (PLEG): container finished" podID="5d57abe1-4de8-49e9-a864-2c48cc5ce6be" containerID="de24428edcff99a22ad6149894ac3c62ca327d27fd3295c05918f13ce7a37c28" exitCode=0
Nov 01 05:29:49.867300 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.866865    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj" event={"ID":"5d57abe1-4de8-49e9-a864-2c48cc5ce6be","Type":"ContainerDied","Data":"de24428edcff99a22ad6149894ac3c62ca327d27fd3295c05918f13ce7a37c28"}
Nov 01 05:29:49.867300 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.866906    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj" event={"ID":"5d57abe1-4de8-49e9-a864-2c48cc5ce6be","Type":"ContainerDied","Data":"954fb1595bab469d13e4dafb914203e4198cfdc231e7fc2bdc396d069d3bb208"}
Nov 01 05:29:49.867300 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.866926    4864 scope.go:122] "RemoveContainer" containerID="de24428edcff99a22ad6149894ac3c62ca327d27fd3295c05918f13ce7a37c28"
Nov 01 05:29:49.867300 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.866925    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"
Nov 01 05:29:49.878138 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.877835    4864 scope.go:122] "RemoveContainer" containerID="de24428edcff99a22ad6149894ac3c62ca327d27fd3295c05918f13ce7a37c28"
Nov 01 05:29:49.878253 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:29:49.878173    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de24428edcff99a22ad6149894ac3c62ca327d27fd3295c05918f13ce7a37c28\": container with ID starting with de24428edcff99a22ad6149894ac3c62ca327d27fd3295c05918f13ce7a37c28 not found: ID does not exist" containerID="de24428edcff99a22ad6149894ac3c62ca327d27fd3295c05918f13ce7a37c28"
Nov 01 05:29:49.878253 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:49.878203    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de24428edcff99a22ad6149894ac3c62ca327d27fd3295c05918f13ce7a37c28"} err="failed to get container status \"de24428edcff99a22ad6149894ac3c62ca327d27fd3295c05918f13ce7a37c28\": rpc error: code = NotFound desc = could not find container \"de24428edcff99a22ad6149894ac3c62ca327d27fd3295c05918f13ce7a37c28\": container with ID starting with de24428edcff99a22ad6149894ac3c62ca327d27fd3295c05918f13ce7a37c28 not found: ID does not exist"
Nov 01 05:29:50.034622 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:50.034594    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"]
Nov 01 05:29:50.086267 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:50.086235    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj"]
Nov 01 05:29:51.247961 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:29:51.247928    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5d57abe1-4de8-49e9-a864-2c48cc5ce6be" path="/var/lib/kubelet/pods/5d57abe1-4de8-49e9-a864-2c48cc5ce6be/volumes"
Nov 01 05:29:52.525100 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:29:52.525065    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:30:00.205541 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:00.205500    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv"]
Nov 01 05:30:00.206352 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:00.206330    4864 cpu_manager.go:414] "containerMap: removing container" podUID="5d57abe1-4de8-49e9-a864-2c48cc5ce6be" containerName="istio-proxy"
Nov 01 05:30:00.206461 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:00.206354    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d57abe1-4de8-49e9-a864-2c48cc5ce6be" containerName="istio-proxy"
Nov 01 05:30:00.206548 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:00.206533    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d57abe1-4de8-49e9-a864-2c48cc5ce6be" containerName="istio-proxy"
Nov 01 05:30:00.210544 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:00.210523    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv"
Nov 01 05:30:00.218344 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:00.218304    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f33bc96a-5937-4527-a1a7-da46df5143ff-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv\" (UID: \"f33bc96a-5937-4527-a1a7-da46df5143ff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv"
Nov 01 05:30:00.218526 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:00.218506    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l8rp\" (UniqueName: \"kubernetes.io/projected/f33bc96a-5937-4527-a1a7-da46df5143ff-kube-api-access-8l8rp\") pod \"maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv\" (UID: \"f33bc96a-5937-4527-a1a7-da46df5143ff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv"
Nov 01 05:30:00.320562 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:00.320521    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f33bc96a-5937-4527-a1a7-da46df5143ff-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv\" (UID: \"f33bc96a-5937-4527-a1a7-da46df5143ff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv"
Nov 01 05:30:00.320758 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:00.320695    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l8rp\" (UniqueName: \"kubernetes.io/projected/f33bc96a-5937-4527-a1a7-da46df5143ff-kube-api-access-8l8rp\") pod \"maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv\" (UID: \"f33bc96a-5937-4527-a1a7-da46df5143ff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv"
Nov 01 05:30:00.322084 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:00.322057    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f33bc96a-5937-4527-a1a7-da46df5143ff-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv\" (UID: \"f33bc96a-5937-4527-a1a7-da46df5143ff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv"
Nov 01 05:30:00.322554 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:00.322529    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv"]
Nov 01 05:30:00.324188 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:00.324163    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sp8g2\""
Nov 01 05:30:00.382575 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:00.382540    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l8rp\" (UniqueName: \"kubernetes.io/projected/f33bc96a-5937-4527-a1a7-da46df5143ff-kube-api-access-8l8rp\") pod \"maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv\" (UID: \"f33bc96a-5937-4527-a1a7-da46df5143ff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv"
Nov 01 05:30:00.525600 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:00.525524    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv"
Nov 01 05:30:00.923069 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:00.920391    4864 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 05:30:00.931574 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:00.931536    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv"]
Nov 01 05:30:00.953949 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:00.953919    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv" event={"ID":"f33bc96a-5937-4527-a1a7-da46df5143ff","Type":"ContainerStarted","Data":"de7c5bb70eca78499b40068b2c3052c295906b334b1309d9e93af23b655960fa"}
Nov 01 05:30:07.012577 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:07.012537    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv" event={"ID":"f33bc96a-5937-4527-a1a7-da46df5143ff","Type":"ContainerStarted","Data":"77c86d4e4943efbc88e1d23ce7c7f3d9ba3ed89240bb120c02e6a691eff3efc1"}
Nov 01 05:30:07.130082 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:07.129979    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv" podStartSLOduration=1.5149304890000002 podStartE2EDuration="7.129964856s" totalImagesPullingTime="5.615034367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:30:07.126995944 +0000 UTC m=+1624.017371911" watchObservedRunningTime="2026-11-01 05:30:07.129964856 +0000 UTC m=+1624.020340832"
Nov 01 05:30:08.023023 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:08.022989    4864 generic.go:356] "Generic (PLEG): container finished" podID="f33bc96a-5937-4527-a1a7-da46df5143ff" containerID="77c86d4e4943efbc88e1d23ce7c7f3d9ba3ed89240bb120c02e6a691eff3efc1" exitCode=60
Nov 01 05:30:08.023537 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:08.023045    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv" event={"ID":"f33bc96a-5937-4527-a1a7-da46df5143ff","Type":"ContainerDied","Data":"77c86d4e4943efbc88e1d23ce7c7f3d9ba3ed89240bb120c02e6a691eff3efc1"}
Nov 01 05:30:08.023537 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:08.023395    4864 scope.go:122] "RemoveContainer" containerID="77c86d4e4943efbc88e1d23ce7c7f3d9ba3ed89240bb120c02e6a691eff3efc1"
Nov 01 05:30:09.031915 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:09.031881    4864 generic.go:356] "Generic (PLEG): container finished" podID="f33bc96a-5937-4527-a1a7-da46df5143ff" containerID="e86e8e50599a906233371a111edee745e52942d316d94964bc005e37b4afb77c" exitCode=60
Nov 01 05:30:09.032381 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:09.031968    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv" event={"ID":"f33bc96a-5937-4527-a1a7-da46df5143ff","Type":"ContainerDied","Data":"e86e8e50599a906233371a111edee745e52942d316d94964bc005e37b4afb77c"}
Nov 01 05:30:09.032381 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:09.032027    4864 scope.go:122] "RemoveContainer" containerID="77c86d4e4943efbc88e1d23ce7c7f3d9ba3ed89240bb120c02e6a691eff3efc1"
Nov 01 05:30:09.032381 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:09.032330    4864 scope.go:122] "RemoveContainer" containerID="e86e8e50599a906233371a111edee745e52942d316d94964bc005e37b4afb77c"
Nov 01 05:30:09.032612 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:30:09.032592    4864 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv_odh-ai-gateway-infra(f33bc96a-5937-4527-a1a7-da46df5143ff)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv" podUID="f33bc96a-5937-4527-a1a7-da46df5143ff"
Nov 01 05:30:10.041149 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:10.041124    4864 scope.go:122] "RemoveContainer" containerID="e86e8e50599a906233371a111edee745e52942d316d94964bc005e37b4afb77c"
Nov 01 05:30:10.041550 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:30:10.041400    4864 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv_odh-ai-gateway-infra(f33bc96a-5937-4527-a1a7-da46df5143ff)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv" podUID="f33bc96a-5937-4527-a1a7-da46df5143ff"
Nov 01 05:30:20.241312 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:20.241282    4864 scope.go:122] "RemoveContainer" containerID="e86e8e50599a906233371a111edee745e52942d316d94964bc005e37b4afb77c"
Nov 01 05:30:21.125231 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:21.125194    4864 generic.go:356] "Generic (PLEG): container finished" podID="f33bc96a-5937-4527-a1a7-da46df5143ff" containerID="055ba09d3af77f36dac021a108d4a919cdc11555a839960f8825fc3f2c28a2ac" exitCode=60
Nov 01 05:30:21.125406 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:21.125271    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv" event={"ID":"f33bc96a-5937-4527-a1a7-da46df5143ff","Type":"ContainerDied","Data":"055ba09d3af77f36dac021a108d4a919cdc11555a839960f8825fc3f2c28a2ac"}
Nov 01 05:30:21.125406 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:21.125358    4864 scope.go:122] "RemoveContainer" containerID="e86e8e50599a906233371a111edee745e52942d316d94964bc005e37b4afb77c"
Nov 01 05:30:21.125698 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:21.125681    4864 scope.go:122] "RemoveContainer" containerID="055ba09d3af77f36dac021a108d4a919cdc11555a839960f8825fc3f2c28a2ac"
Nov 01 05:30:21.125997 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:30:21.125969    4864 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv_odh-ai-gateway-infra(f33bc96a-5937-4527-a1a7-da46df5143ff)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv" podUID="f33bc96a-5937-4527-a1a7-da46df5143ff"
Nov 01 05:30:22.253448 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:22.253414    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv"]
Nov 01 05:30:22.401976 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:22.401947    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv"
Nov 01 05:30:22.499732 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:22.499702    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f33bc96a-5937-4527-a1a7-da46df5143ff-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f33bc96a-5937-4527-a1a7-da46df5143ff-maas-api-service-ca\") pod \"f33bc96a-5937-4527-a1a7-da46df5143ff\" (UID: \"f33bc96a-5937-4527-a1a7-da46df5143ff\") "
Nov 01 05:30:22.499940 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:22.499769    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f33bc96a-5937-4527-a1a7-da46df5143ff-kube-api-access-8l8rp\" (UniqueName: \"kubernetes.io/projected/f33bc96a-5937-4527-a1a7-da46df5143ff-kube-api-access-8l8rp\") pod \"f33bc96a-5937-4527-a1a7-da46df5143ff\" (UID: \"f33bc96a-5937-4527-a1a7-da46df5143ff\") "
Nov 01 05:30:22.500217 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:22.500183    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f33bc96a-5937-4527-a1a7-da46df5143ff-maas-api-service-ca" pod "f33bc96a-5937-4527-a1a7-da46df5143ff" (UID: "f33bc96a-5937-4527-a1a7-da46df5143ff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:30:22.502075 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:22.502053    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f33bc96a-5937-4527-a1a7-da46df5143ff-kube-api-access-8l8rp" pod "f33bc96a-5937-4527-a1a7-da46df5143ff" (UID: "f33bc96a-5937-4527-a1a7-da46df5143ff"). InnerVolumeSpecName "kube-api-access-8l8rp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:30:22.537461 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:30:22.537423    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:30:22.602183 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:22.602147    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f33bc96a-5937-4527-a1a7-da46df5143ff-maas-api-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:22.602183 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:22.602183    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8l8rp\" (UniqueName: \"kubernetes.io/projected/f33bc96a-5937-4527-a1a7-da46df5143ff-kube-api-access-8l8rp\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:23.141766 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:23.141736    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv"
Nov 01 05:30:23.141977 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:23.141764    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv" event={"ID":"f33bc96a-5937-4527-a1a7-da46df5143ff","Type":"ContainerDied","Data":"de7c5bb70eca78499b40068b2c3052c295906b334b1309d9e93af23b655960fa"}
Nov 01 05:30:23.141977 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:23.141843    4864 scope.go:122] "RemoveContainer" containerID="055ba09d3af77f36dac021a108d4a919cdc11555a839960f8825fc3f2c28a2ac"
Nov 01 05:30:23.307943 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:23.307908    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv"]
Nov 01 05:30:23.357393 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:23.357360    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-ca7a55-29891850-pm8vv"]
Nov 01 05:30:25.247423 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:25.247386    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f33bc96a-5937-4527-a1a7-da46df5143ff" path="/var/lib/kubelet/pods/f33bc96a-5937-4527-a1a7-da46df5143ff/volumes"
Nov 01 05:30:28.163384 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:28.163344    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"]
Nov 01 05:30:28.163915 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:28.163597    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7" podUID="59915eb9-7731-4c07-925d-d9d1a2789d9a" containerName="istio-proxy" containerID="cri-o://b4922312f39c4e7d1b2cf2b8635eb66b27793a86ceea482a36cd858f5451902a" gracePeriod=30
Nov 01 05:30:28.212912 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:28.212877    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"]
Nov 01 05:30:28.213118 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:28.213095    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j" podUID="03518f25-1bdb-4021-aa8c-6d1e9083c8fe" containerName="istio-proxy" containerID="cri-o://aa8a28ca172c0d6bec7371311e815d50df0a551dde4aa2ccbc6f458ba9faba72" gracePeriod=30
Nov 01 05:30:32.904516 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:32.904477    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 05:30:32.905105 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:32.904552    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7" podUID="59915eb9-7731-4c07-925d-d9d1a2789d9a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 05:30:33.501317 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.501289    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:30:33.518169 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.518148    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:30:33.527681 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.527652    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-workload-socket\") pod \"59915eb9-7731-4c07-925d-d9d1a2789d9a\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") "
Nov 01 05:30:33.529554 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.527715    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-credential-socket\") pod \"59915eb9-7731-4c07-925d-d9d1a2789d9a\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") "
Nov 01 05:30:33.529554 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.527805    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-envoy\") pod \"59915eb9-7731-4c07-925d-d9d1a2789d9a\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") "
Nov 01 05:30:33.529554 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.527834    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-podinfo\") pod \"59915eb9-7731-4c07-925d-d9d1a2789d9a\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") "
Nov 01 05:30:33.529554 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.527885    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/59915eb9-7731-4c07-925d-d9d1a2789d9a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59915eb9-7731-4c07-925d-d9d1a2789d9a-istiod-ca-cert\") pod \"59915eb9-7731-4c07-925d-d9d1a2789d9a\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") "
Nov 01 05:30:33.529554 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.527932    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-workload-certs\") pod \"59915eb9-7731-4c07-925d-d9d1a2789d9a\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") "
Nov 01 05:30:33.529554 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.527978    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-data\") pod \"59915eb9-7731-4c07-925d-d9d1a2789d9a\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") "
Nov 01 05:30:33.529554 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.528019    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-token\" (UniqueName: \"kubernetes.io/projected/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-token\") pod \"59915eb9-7731-4c07-925d-d9d1a2789d9a\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") "
Nov 01 05:30:33.529554 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.528028    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-workload-socket" pod "59915eb9-7731-4c07-925d-d9d1a2789d9a" (UID: "59915eb9-7731-4c07-925d-d9d1a2789d9a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:30:33.529554 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.528053    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59915eb9-7731-4c07-925d-d9d1a2789d9a-kube-api-access-q8lhk\" (UniqueName: \"kubernetes.io/projected/59915eb9-7731-4c07-925d-d9d1a2789d9a-kube-api-access-q8lhk\") pod \"59915eb9-7731-4c07-925d-d9d1a2789d9a\" (UID: \"59915eb9-7731-4c07-925d-d9d1a2789d9a\") "
Nov 01 05:30:33.529554 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.528645    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59915eb9-7731-4c07-925d-d9d1a2789d9a-istiod-ca-cert" pod "59915eb9-7731-4c07-925d-d9d1a2789d9a" (UID: "59915eb9-7731-4c07-925d-d9d1a2789d9a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:30:33.529554 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.528983    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59915eb9-7731-4c07-925d-d9d1a2789d9a-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:33.529554 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.529004    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:33.529554 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.529064    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-credential-socket" pod "59915eb9-7731-4c07-925d-d9d1a2789d9a" (UID: "59915eb9-7731-4c07-925d-d9d1a2789d9a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:30:33.529554 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.529179    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-workload-certs" pod "59915eb9-7731-4c07-925d-d9d1a2789d9a" (UID: "59915eb9-7731-4c07-925d-d9d1a2789d9a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:30:33.530278 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.529576    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-data" pod "59915eb9-7731-4c07-925d-d9d1a2789d9a" (UID: "59915eb9-7731-4c07-925d-d9d1a2789d9a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:30:33.532623 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.532597    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-podinfo" pod "59915eb9-7731-4c07-925d-d9d1a2789d9a" (UID: "59915eb9-7731-4c07-925d-d9d1a2789d9a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 05:30:33.532838 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.532809    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-token" pod "59915eb9-7731-4c07-925d-d9d1a2789d9a" (UID: "59915eb9-7731-4c07-925d-d9d1a2789d9a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:30:33.533342 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.533315    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59915eb9-7731-4c07-925d-d9d1a2789d9a-kube-api-access-q8lhk" pod "59915eb9-7731-4c07-925d-d9d1a2789d9a" (UID: "59915eb9-7731-4c07-925d-d9d1a2789d9a"). InnerVolumeSpecName "kube-api-access-q8lhk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:30:33.534159 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.534138    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-envoy" pod "59915eb9-7731-4c07-925d-d9d1a2789d9a" (UID: "59915eb9-7731-4c07-925d-d9d1a2789d9a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:30:33.630297 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.630265    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-token\" (UniqueName: \"kubernetes.io/projected/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-token\") pod \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") "
Nov 01 05:30:33.630500 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.630339    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-workload-socket\") pod \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") "
Nov 01 05:30:33.630500 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.630380    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-workload-certs\") pod \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") "
Nov 01 05:30:33.630500 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.630428    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-kube-api-access-rch6c\" (UniqueName: \"kubernetes.io/projected/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-kube-api-access-rch6c\") pod \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") "
Nov 01 05:30:33.630695 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.630509    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-credential-socket\") pod \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") "
Nov 01 05:30:33.630695 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.630563    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-data\") pod \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") "
Nov 01 05:30:33.630695 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.630610    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istiod-ca-cert\") pod \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") "
Nov 01 05:30:33.630695 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.630665    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-envoy\") pod \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") "
Nov 01 05:30:33.630695 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.630680    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-workload-socket" pod "03518f25-1bdb-4021-aa8c-6d1e9083c8fe" (UID: "03518f25-1bdb-4021-aa8c-6d1e9083c8fe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:30:33.631007 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.630702    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-workload-certs" pod "03518f25-1bdb-4021-aa8c-6d1e9083c8fe" (UID: "03518f25-1bdb-4021-aa8c-6d1e9083c8fe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:30:33.631007 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.630708    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-podinfo\") pod \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\" (UID: \"03518f25-1bdb-4021-aa8c-6d1e9083c8fe\") "
Nov 01 05:30:33.631108 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.631001    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-credential-socket" pod "03518f25-1bdb-4021-aa8c-6d1e9083c8fe" (UID: "03518f25-1bdb-4021-aa8c-6d1e9083c8fe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:30:33.631158 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.631135    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istiod-ca-cert" pod "03518f25-1bdb-4021-aa8c-6d1e9083c8fe" (UID: "03518f25-1bdb-4021-aa8c-6d1e9083c8fe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:30:33.631340 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.631312    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-data" pod "03518f25-1bdb-4021-aa8c-6d1e9083c8fe" (UID: "03518f25-1bdb-4021-aa8c-6d1e9083c8fe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:30:33.632082 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.631421    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:33.632082 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.631447    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8lhk\" (UniqueName: \"kubernetes.io/projected/59915eb9-7731-4c07-925d-d9d1a2789d9a-kube-api-access-q8lhk\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:33.632082 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.631465    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:33.632082 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.631486    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:33.632082 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.631503    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:33.632082 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.631517    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:33.632082 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.631585    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:33.632082 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.631603    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:33.632082 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.631617    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:33.632082 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.631630    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:33.632082 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.631645    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59915eb9-7731-4c07-925d-d9d1a2789d9a-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:33.633298 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.633266    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-token" pod "03518f25-1bdb-4021-aa8c-6d1e9083c8fe" (UID: "03518f25-1bdb-4021-aa8c-6d1e9083c8fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:30:33.633395 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.633293    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-envoy" pod "03518f25-1bdb-4021-aa8c-6d1e9083c8fe" (UID: "03518f25-1bdb-4021-aa8c-6d1e9083c8fe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:30:33.633581 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.633555    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-kube-api-access-rch6c" pod "03518f25-1bdb-4021-aa8c-6d1e9083c8fe" (UID: "03518f25-1bdb-4021-aa8c-6d1e9083c8fe"). InnerVolumeSpecName "kube-api-access-rch6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:30:33.633693 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.633585    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-podinfo" pod "03518f25-1bdb-4021-aa8c-6d1e9083c8fe" (UID: "03518f25-1bdb-4021-aa8c-6d1e9083c8fe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 05:30:33.732755 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.732672    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:33.732755 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.732700    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:33.732755 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.732710    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:33.732755 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.732720    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:33.732755 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:33.732729    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rch6c\" (UniqueName: \"kubernetes.io/projected/03518f25-1bdb-4021-aa8c-6d1e9083c8fe-kube-api-access-rch6c\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:34.226158 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:34.226122    4864 generic.go:356] "Generic (PLEG): container finished" podID="59915eb9-7731-4c07-925d-d9d1a2789d9a" containerID="b4922312f39c4e7d1b2cf2b8635eb66b27793a86ceea482a36cd858f5451902a" exitCode=0
Nov 01 05:30:34.226569 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:34.226181    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"
Nov 01 05:30:34.226569 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:34.226202    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7" event={"ID":"59915eb9-7731-4c07-925d-d9d1a2789d9a","Type":"ContainerDied","Data":"b4922312f39c4e7d1b2cf2b8635eb66b27793a86ceea482a36cd858f5451902a"}
Nov 01 05:30:34.226569 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:34.226264    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7" event={"ID":"59915eb9-7731-4c07-925d-d9d1a2789d9a","Type":"ContainerDied","Data":"1fff466b47bea8e4aa7090b45e979d13175f9df0f4c603dd9ee169aab99b9080"}
Nov 01 05:30:34.226569 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:34.226285    4864 scope.go:122] "RemoveContainer" containerID="b4922312f39c4e7d1b2cf2b8635eb66b27793a86ceea482a36cd858f5451902a"
Nov 01 05:30:34.228386 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:34.228362    4864 generic.go:356] "Generic (PLEG): container finished" podID="03518f25-1bdb-4021-aa8c-6d1e9083c8fe" containerID="aa8a28ca172c0d6bec7371311e815d50df0a551dde4aa2ccbc6f458ba9faba72" exitCode=0
Nov 01 05:30:34.228514 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:34.228407    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j" event={"ID":"03518f25-1bdb-4021-aa8c-6d1e9083c8fe","Type":"ContainerDied","Data":"aa8a28ca172c0d6bec7371311e815d50df0a551dde4aa2ccbc6f458ba9faba72"}
Nov 01 05:30:34.228514 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:34.228481    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j" event={"ID":"03518f25-1bdb-4021-aa8c-6d1e9083c8fe","Type":"ContainerDied","Data":"007b8741a739c1901e84458c446cb7f1b5e65096e862142d5995303c308e750f"}
Nov 01 05:30:34.228514 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:34.228417    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"
Nov 01 05:30:34.240830 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:34.240797    4864 scope.go:122] "RemoveContainer" containerID="b4922312f39c4e7d1b2cf2b8635eb66b27793a86ceea482a36cd858f5451902a"
Nov 01 05:30:34.241128 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:30:34.241107    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4922312f39c4e7d1b2cf2b8635eb66b27793a86ceea482a36cd858f5451902a\": container with ID starting with b4922312f39c4e7d1b2cf2b8635eb66b27793a86ceea482a36cd858f5451902a not found: ID does not exist" containerID="b4922312f39c4e7d1b2cf2b8635eb66b27793a86ceea482a36cd858f5451902a"
Nov 01 05:30:34.241197 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:34.241134    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4922312f39c4e7d1b2cf2b8635eb66b27793a86ceea482a36cd858f5451902a"} err="failed to get container status \"b4922312f39c4e7d1b2cf2b8635eb66b27793a86ceea482a36cd858f5451902a\": rpc error: code = NotFound desc = could not find container \"b4922312f39c4e7d1b2cf2b8635eb66b27793a86ceea482a36cd858f5451902a\": container with ID starting with b4922312f39c4e7d1b2cf2b8635eb66b27793a86ceea482a36cd858f5451902a not found: ID does not exist"
Nov 01 05:30:34.241197 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:34.241151    4864 scope.go:122] "RemoveContainer" containerID="aa8a28ca172c0d6bec7371311e815d50df0a551dde4aa2ccbc6f458ba9faba72"
Nov 01 05:30:34.250894 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:34.250873    4864 scope.go:122] "RemoveContainer" containerID="aa8a28ca172c0d6bec7371311e815d50df0a551dde4aa2ccbc6f458ba9faba72"
Nov 01 05:30:34.251202 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:30:34.251183    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa8a28ca172c0d6bec7371311e815d50df0a551dde4aa2ccbc6f458ba9faba72\": container with ID starting with aa8a28ca172c0d6bec7371311e815d50df0a551dde4aa2ccbc6f458ba9faba72 not found: ID does not exist" containerID="aa8a28ca172c0d6bec7371311e815d50df0a551dde4aa2ccbc6f458ba9faba72"
Nov 01 05:30:34.251251 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:34.251207    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa8a28ca172c0d6bec7371311e815d50df0a551dde4aa2ccbc6f458ba9faba72"} err="failed to get container status \"aa8a28ca172c0d6bec7371311e815d50df0a551dde4aa2ccbc6f458ba9faba72\": rpc error: code = NotFound desc = could not find container \"aa8a28ca172c0d6bec7371311e815d50df0a551dde4aa2ccbc6f458ba9faba72\": container with ID starting with aa8a28ca172c0d6bec7371311e815d50df0a551dde4aa2ccbc6f458ba9faba72 not found: ID does not exist"
Nov 01 05:30:34.392885 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:34.392855    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"]
Nov 01 05:30:34.442742 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:34.442702    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j"]
Nov 01 05:30:34.507931 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:34.507841    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"]
Nov 01 05:30:34.511985 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:34.511957    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7"]
Nov 01 05:30:35.255402 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:35.255365    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="03518f25-1bdb-4021-aa8c-6d1e9083c8fe" path="/var/lib/kubelet/pods/03518f25-1bdb-4021-aa8c-6d1e9083c8fe/volumes"
Nov 01 05:30:35.256129 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:35.256105    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="59915eb9-7731-4c07-925d-d9d1a2789d9a" path="/var/lib/kubelet/pods/59915eb9-7731-4c07-925d-d9d1a2789d9a/volumes"
Nov 01 05:30:39.712962 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:39.712925    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"]
Nov 01 05:30:51.918960 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:51.918874    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"]
Nov 01 05:30:51.919454 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:51.919438    4864 cpu_manager.go:414] "containerMap: removing container" podUID="f33bc96a-5937-4527-a1a7-da46df5143ff" containerName="cleanup"
Nov 01 05:30:51.919498 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:51.919456    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f33bc96a-5937-4527-a1a7-da46df5143ff" containerName="cleanup"
Nov 01 05:30:51.919498 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:51.919473    4864 cpu_manager.go:414] "containerMap: removing container" podUID="59915eb9-7731-4c07-925d-d9d1a2789d9a" containerName="istio-proxy"
Nov 01 05:30:51.919498 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:51.919478    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="59915eb9-7731-4c07-925d-d9d1a2789d9a" containerName="istio-proxy"
Nov 01 05:30:51.919498 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:51.919498    4864 cpu_manager.go:414] "containerMap: removing container" podUID="03518f25-1bdb-4021-aa8c-6d1e9083c8fe" containerName="istio-proxy"
Nov 01 05:30:51.919620 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:51.919503    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="03518f25-1bdb-4021-aa8c-6d1e9083c8fe" containerName="istio-proxy"
Nov 01 05:30:51.919620 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:51.919509    4864 cpu_manager.go:414] "containerMap: removing container" podUID="f33bc96a-5937-4527-a1a7-da46df5143ff" containerName="cleanup"
Nov 01 05:30:51.919620 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:51.919513    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f33bc96a-5937-4527-a1a7-da46df5143ff" containerName="cleanup"
Nov 01 05:30:51.919620 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:51.919595    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="f33bc96a-5937-4527-a1a7-da46df5143ff" containerName="cleanup"
Nov 01 05:30:51.919620 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:51.919603    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="59915eb9-7731-4c07-925d-d9d1a2789d9a" containerName="istio-proxy"
Nov 01 05:30:51.919620 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:51.919610    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="03518f25-1bdb-4021-aa8c-6d1e9083c8fe" containerName="istio-proxy"
Nov 01 05:30:51.919620 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:51.919616    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="f33bc96a-5937-4527-a1a7-da46df5143ff" containerName="cleanup"
Nov 01 05:30:51.919620 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:51.919620    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="f33bc96a-5937-4527-a1a7-da46df5143ff" containerName="cleanup"
Nov 01 05:30:51.923016 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:51.922995    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.043077 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.043036    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-envoy\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.043254 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.043104    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-podinfo\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.043254 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.043229    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-workload-socket\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.043333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.043264    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-data\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.043403 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.043380    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-token\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.043462 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.043444    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-credential-socket\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.043526 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.043511    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istiod-ca-cert\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.043577 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.043552    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-workload-certs\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.043632 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.043576    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv4fx\" (UniqueName: \"kubernetes.io/projected/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-kube-api-access-bv4fx\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.131354 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.131309    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"]
Nov 01 05:30:52.144710 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.144678    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-token\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.144910 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.144728    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-credential-socket\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.144910 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.144763    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istiod-ca-cert\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.144910 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.144813    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-workload-certs\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.144910 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.144836    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv4fx\" (UniqueName: \"kubernetes.io/projected/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-kube-api-access-bv4fx\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.144910 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.144898    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-envoy\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.145187 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.144935    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-podinfo\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.145187 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.145013    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-workload-socket\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.145187 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.145046    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-data\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.145337 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.145247    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-credential-socket\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.145337 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.145309    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-workload-certs\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.145622 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.145597    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istiod-ca-cert\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.145776 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.145751    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-workload-socket\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.145882 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.145772    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-data\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.147423 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.147393    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-envoy\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.147653 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.147633    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-podinfo\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.205935 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.205856    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-token\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.206076 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.206008    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv4fx\" (UniqueName: \"kubernetes.io/projected/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-kube-api-access-bv4fx\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.236543 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.236511    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:52.525479 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:30:52.525397    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:30:52.666735 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.666702    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"]
Nov 01 05:30:52.670365 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:30:52.670340    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4002ccda_eb43_49ff_90e6_5ce4e72a70c1.slice/crio-3c3626f071d50abc1e2a27d664b89f0d39845a5d671a510eae2266077eb627b4 WatchSource:0}: Error finding container 3c3626f071d50abc1e2a27d664b89f0d39845a5d671a510eae2266077eb627b4: Status 404 returned error can't find the container with id 3c3626f071d50abc1e2a27d664b89f0d39845a5d671a510eae2266077eb627b4
Nov 01 05:30:52.672956 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.672924    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:30:52.673037 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.672994    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:30:52.673037 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:52.673025    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:30:53.389812 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:53.389631    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t" event={"ID":"4002ccda-eb43-49ff-90e6-5ce4e72a70c1","Type":"ContainerStarted","Data":"e73a32d3862b9699f1ad9edc69179c51ae484cdb8653112949dc2cf6c1c1298b"}
Nov 01 05:30:53.389812 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:53.389701    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t" event={"ID":"4002ccda-eb43-49ff-90e6-5ce4e72a70c1","Type":"ContainerStarted","Data":"3c3626f071d50abc1e2a27d664b89f0d39845a5d671a510eae2266077eb627b4"}
Nov 01 05:30:54.236734 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:54.236696    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:30:54.238514 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:54.238487    4864 patch_prober.go:29] interesting pod/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 05:30:54.238593 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:54.238546    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t" podUID="4002ccda-eb43-49ff-90e6-5ce4e72a70c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 05:30:54.734400 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:54.734332    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27" podUID="a5d934f5-3410-4c7e-9f93-ede52dde324a" containerName="main" containerID="cri-o://b816434d3077eff602da9257be27a895d2b02b48171ec6b228729964e79f5c0e" gracePeriod=45
Nov 01 05:30:55.047533 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.047510    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:30:55.100491 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.100457    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5d934f5-3410-4c7e-9f93-ede52dde324a-kube-api-access-hd669\" (UniqueName: \"kubernetes.io/projected/a5d934f5-3410-4c7e-9f93-ede52dde324a-kube-api-access-hd669\") pod \"a5d934f5-3410-4c7e-9f93-ede52dde324a\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") "
Nov 01 05:30:55.100667 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.100504    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-dshm\") pod \"a5d934f5-3410-4c7e-9f93-ede52dde324a\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") "
Nov 01 05:30:55.100667 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.100549    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-tmp-dir\") pod \"a5d934f5-3410-4c7e-9f93-ede52dde324a\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") "
Nov 01 05:30:55.100667 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.100599    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a5d934f5-3410-4c7e-9f93-ede52dde324a-tls-certs\" (UniqueName: \"kubernetes.io/secret/a5d934f5-3410-4c7e-9f93-ede52dde324a-tls-certs\") pod \"a5d934f5-3410-4c7e-9f93-ede52dde324a\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") "
Nov 01 05:30:55.100667 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.100651    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-model-cache\") pod \"a5d934f5-3410-4c7e-9f93-ede52dde324a\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") "
Nov 01 05:30:55.100881 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.100750    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-home\" (UniqueName: \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-home\") pod \"a5d934f5-3410-4c7e-9f93-ede52dde324a\" (UID: \"a5d934f5-3410-4c7e-9f93-ede52dde324a\") "
Nov 01 05:30:55.100983 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.100860    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-tmp-dir" pod "a5d934f5-3410-4c7e-9f93-ede52dde324a" (UID: "a5d934f5-3410-4c7e-9f93-ede52dde324a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:30:55.101052 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.100993    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-model-cache" pod "a5d934f5-3410-4c7e-9f93-ede52dde324a" (UID: "a5d934f5-3410-4c7e-9f93-ede52dde324a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:30:55.101113 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.101052    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-home" pod "a5d934f5-3410-4c7e-9f93-ede52dde324a" (UID: "a5d934f5-3410-4c7e-9f93-ede52dde324a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:30:55.101269 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.101253    4864 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-model-cache\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:55.101333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.101272    4864 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-home\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:55.101333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.101283    4864 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-tmp-dir\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:55.102910 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.102893    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-dshm" pod "a5d934f5-3410-4c7e-9f93-ede52dde324a" (UID: "a5d934f5-3410-4c7e-9f93-ede52dde324a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:30:55.103412 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.103394    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5d934f5-3410-4c7e-9f93-ede52dde324a-kube-api-access-hd669" pod "a5d934f5-3410-4c7e-9f93-ede52dde324a" (UID: "a5d934f5-3410-4c7e-9f93-ede52dde324a"). InnerVolumeSpecName "kube-api-access-hd669". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:30:55.103412 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.103403    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5d934f5-3410-4c7e-9f93-ede52dde324a-tls-certs" pod "a5d934f5-3410-4c7e-9f93-ede52dde324a" (UID: "a5d934f5-3410-4c7e-9f93-ede52dde324a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:30:55.202317 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.202283    4864 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5d934f5-3410-4c7e-9f93-ede52dde324a-tls-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:55.202317 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.202314    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hd669\" (UniqueName: \"kubernetes.io/projected/a5d934f5-3410-4c7e-9f93-ede52dde324a-kube-api-access-hd669\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:55.202317 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.202326    4864 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5d934f5-3410-4c7e-9f93-ede52dde324a-dshm\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:30:55.223144 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.223091    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t" podStartSLOduration=4.223076501 podStartE2EDuration="4.223076501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:30:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:30:53.581643897 +0000 UTC m=+1670.472019888" watchObservedRunningTime="2026-11-01 05:30:55.223076501 +0000 UTC m=+1672.113452469"
Nov 01 05:30:55.237848 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.237776    4864 patch_prober.go:29] interesting pod/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 05:30:55.237999 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.237875    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t" podUID="4002ccda-eb43-49ff-90e6-5ce4e72a70c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 05:30:55.418014 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.417973    4864 generic.go:356] "Generic (PLEG): container finished" podID="a5d934f5-3410-4c7e-9f93-ede52dde324a" containerID="b816434d3077eff602da9257be27a895d2b02b48171ec6b228729964e79f5c0e" exitCode=0
Nov 01 05:30:55.418215 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.418060    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"
Nov 01 05:30:55.418215 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.418081    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27" event={"ID":"a5d934f5-3410-4c7e-9f93-ede52dde324a","Type":"ContainerDied","Data":"b816434d3077eff602da9257be27a895d2b02b48171ec6b228729964e79f5c0e"}
Nov 01 05:30:55.418215 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.418138    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27" event={"ID":"a5d934f5-3410-4c7e-9f93-ede52dde324a","Type":"ContainerDied","Data":"eef65c013db78b619c9f5d56dd5a4abf9ee880d86054d1f558e38be6d90109b0"}
Nov 01 05:30:55.418215 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.418160    4864 scope.go:122] "RemoveContainer" containerID="b816434d3077eff602da9257be27a895d2b02b48171ec6b228729964e79f5c0e"
Nov 01 05:30:55.429438 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.429406    4864 scope.go:122] "RemoveContainer" containerID="b816434d3077eff602da9257be27a895d2b02b48171ec6b228729964e79f5c0e"
Nov 01 05:30:55.429773 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:30:55.429750    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b816434d3077eff602da9257be27a895d2b02b48171ec6b228729964e79f5c0e\": container with ID starting with b816434d3077eff602da9257be27a895d2b02b48171ec6b228729964e79f5c0e not found: ID does not exist" containerID="b816434d3077eff602da9257be27a895d2b02b48171ec6b228729964e79f5c0e"
Nov 01 05:30:55.429878 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.429799    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b816434d3077eff602da9257be27a895d2b02b48171ec6b228729964e79f5c0e"} err="failed to get container status \"b816434d3077eff602da9257be27a895d2b02b48171ec6b228729964e79f5c0e\": rpc error: code = NotFound desc = could not find container \"b816434d3077eff602da9257be27a895d2b02b48171ec6b228729964e79f5c0e\": container with ID starting with b816434d3077eff602da9257be27a895d2b02b48171ec6b228729964e79f5c0e not found: ID does not exist"
Nov 01 05:30:55.594799 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.594755    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"]
Nov 01 05:30:55.599054 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:55.599017    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-praxis-a-918b8d/praxis-route-918b8d-kserve-6f9b465557-h8n27"]
Nov 01 05:30:56.237366 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:56.237336    4864 patch_prober.go:29] interesting pod/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 05:30:56.237773 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:56.237382    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t" podUID="4002ccda-eb43-49ff-90e6-5ce4e72a70c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 05:30:57.237928 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:57.237890    4864 patch_prober.go:29] interesting pod/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 05:30:57.238333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:57.237947    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t" podUID="4002ccda-eb43-49ff-90e6-5ce4e72a70c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 05:30:57.247928 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:57.247900    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5d934f5-3410-4c7e-9f93-ede52dde324a" path="/var/lib/kubelet/pods/a5d934f5-3410-4c7e-9f93-ede52dde324a/volumes"
Nov 01 05:30:58.237593 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:58.237557    4864 patch_prober.go:29] interesting pod/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 05:30:58.237896 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:58.237633    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t" podUID="4002ccda-eb43-49ff-90e6-5ce4e72a70c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 05:30:59.237927 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:59.237872    4864 patch_prober.go:29] interesting pod/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 05:30:59.238334 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:30:59.237963    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t" podUID="4002ccda-eb43-49ff-90e6-5ce4e72a70c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 05:31:00.241097 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:00.241062    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:31:00.241535 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:00.241311    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:31:00.241930 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:00.241913    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:31:21.923826 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:21.923779    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"]
Nov 01 05:31:21.924364 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:21.924348    4864 cpu_manager.go:414] "containerMap: removing container" podUID="f33bc96a-5937-4527-a1a7-da46df5143ff" containerName="cleanup"
Nov 01 05:31:21.924403 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:21.924365    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f33bc96a-5937-4527-a1a7-da46df5143ff" containerName="cleanup"
Nov 01 05:31:21.924403 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:21.924396    4864 cpu_manager.go:414] "containerMap: removing container" podUID="a5d934f5-3410-4c7e-9f93-ede52dde324a" containerName="main"
Nov 01 05:31:21.924403 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:21.924401    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5d934f5-3410-4c7e-9f93-ede52dde324a" containerName="main"
Nov 01 05:31:21.924514 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:21.924489    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5d934f5-3410-4c7e-9f93-ede52dde324a" containerName="main"
Nov 01 05:31:21.927914 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:21.927893    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.028588 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.028554    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"]
Nov 01 05:31:22.062399 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.062359    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-istio-envoy\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.062572 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.062418    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28515788-ceb7-439c-829f-9de5067031f9-istio-podinfo\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.062572 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.062526    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28515788-ceb7-439c-829f-9de5067031f9-istiod-ca-cert\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.062684 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.062595    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-workload-certs\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.062723 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.062705    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28515788-ceb7-439c-829f-9de5067031f9-istio-token\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.062770 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.062740    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-workload-socket\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.062860 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.062843    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqrt9\" (UniqueName: \"kubernetes.io/projected/28515788-ceb7-439c-829f-9de5067031f9-kube-api-access-tqrt9\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.063021 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.062994    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-credential-socket\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.063100 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.063083    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-istio-data\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.164558 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.164523    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28515788-ceb7-439c-829f-9de5067031f9-istio-token\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.164558 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.164565    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-workload-socket\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.164869 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.164596    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqrt9\" (UniqueName: \"kubernetes.io/projected/28515788-ceb7-439c-829f-9de5067031f9-kube-api-access-tqrt9\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.164869 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.164640    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-credential-socket\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.164869 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.164694    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-istio-data\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.164869 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.164778    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-istio-envoy\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.164869 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.164861    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28515788-ceb7-439c-829f-9de5067031f9-istio-podinfo\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.165133 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.164897    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28515788-ceb7-439c-829f-9de5067031f9-istiod-ca-cert\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.165133 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.164948    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-workload-certs\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.165133 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.165093    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-credential-socket\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.165292 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.165223    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-istio-data\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.165415 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.165394    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-workload-socket\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.165483 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.165426    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-workload-certs\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.165699 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.165682    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28515788-ceb7-439c-829f-9de5067031f9-istiod-ca-cert\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.167637 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.167615    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-istio-envoy\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.168070 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.168050    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28515788-ceb7-439c-829f-9de5067031f9-istio-podinfo\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.225923 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.225849    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28515788-ceb7-439c-829f-9de5067031f9-istio-token\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.226059 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.225948    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqrt9\" (UniqueName: \"kubernetes.io/projected/28515788-ceb7-439c-829f-9de5067031f9-kube-api-access-tqrt9\") pod \"e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") " pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.241570 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.241544    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:22.526189 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:31:22.526101    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:31:22.686165 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.686133    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"]
Nov 01 05:31:22.687842 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:31:22.687769    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28515788_ceb7_439c_829f_9de5067031f9.slice/crio-e8592e78e9f6a301751edc1cf78abf45bd7197e51a032064fdea85874b41b243 WatchSource:0}: Error finding container e8592e78e9f6a301751edc1cf78abf45bd7197e51a032064fdea85874b41b243: Status 404 returned error can't find the container with id e8592e78e9f6a301751edc1cf78abf45bd7197e51a032064fdea85874b41b243
Nov 01 05:31:22.690699 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.690660    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:31:22.690801 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.690746    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:31:22.690849 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:22.690826    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:31:23.613644 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:23.613605    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx" event={"ID":"28515788-ceb7-439c-829f-9de5067031f9","Type":"ContainerStarted","Data":"055dcbb3d8d97810e1ab7fc7589f33bea5b1c859957492d7fba72eb8847ccfb4"}
Nov 01 05:31:23.614190 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:23.613666    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx" event={"ID":"28515788-ceb7-439c-829f-9de5067031f9","Type":"ContainerStarted","Data":"e8592e78e9f6a301751edc1cf78abf45bd7197e51a032064fdea85874b41b243"}
Nov 01 05:31:23.785811 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:23.785500    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx" podStartSLOduration=2.785481931 podStartE2EDuration="2.785481931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:31:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:31:23.782994173 +0000 UTC m=+1700.673370151" watchObservedRunningTime="2026-11-01 05:31:23.785481931 +0000 UTC m=+1700.675857908"
Nov 01 05:31:24.242452 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:24.242421    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:24.243738 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:24.243716    4864 patch_prober.go:29] interesting pod/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 01 05:31:24.243865 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:24.243769    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx" podUID="28515788-ceb7-439c-829f-9de5067031f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 01 05:31:25.242982 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:25.242944    4864 patch_prober.go:29] interesting pod/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 01 05:31:25.243429 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:25.242996    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx" podUID="28515788-ceb7-439c-829f-9de5067031f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 01 05:31:26.242229 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:26.242187    4864 patch_prober.go:29] interesting pod/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 01 05:31:26.242413 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:26.242258    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx" podUID="28515788-ceb7-439c-829f-9de5067031f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 01 05:31:27.242168 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:27.242126    4864 patch_prober.go:29] interesting pod/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 01 05:31:27.242596 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:27.242197    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx" podUID="28515788-ceb7-439c-829f-9de5067031f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 01 05:31:28.208923 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:28.208888    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"]
Nov 01 05:31:28.209160 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:28.209121    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r" podUID="b94fa739-43bf-4e30-a4d2-c96f7d18d498" containerName="maas-api" containerID="cri-o://4f58d5c8facf50bbdfacb0e33b18d34ac05652659c0067c0326f54df626ffaa7" gracePeriod=30
Nov 01 05:31:28.241985 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:28.241953    4864 patch_prober.go:29] interesting pod/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 01 05:31:28.242177 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:28.242015    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx" podUID="28515788-ceb7-439c-829f-9de5067031f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 01 05:31:29.253810 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.243174    4864 patch_prober.go:29] interesting pod/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 01 05:31:29.253810 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.243250    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx" podUID="28515788-ceb7-439c-829f-9de5067031f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 01 05:31:29.495262 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.495227    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"
Nov 01 05:31:29.643158 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.643120    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b94fa739-43bf-4e30-a4d2-c96f7d18d498-kube-api-access-82qnm\" (UniqueName: \"kubernetes.io/projected/b94fa739-43bf-4e30-a4d2-c96f7d18d498-kube-api-access-82qnm\") pod \"b94fa739-43bf-4e30-a4d2-c96f7d18d498\" (UID: \"b94fa739-43bf-4e30-a4d2-c96f7d18d498\") "
Nov 01 05:31:29.643332 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.643239    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b94fa739-43bf-4e30-a4d2-c96f7d18d498-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b94fa739-43bf-4e30-a4d2-c96f7d18d498-openshift-service-ca\") pod \"b94fa739-43bf-4e30-a4d2-c96f7d18d498\" (UID: \"b94fa739-43bf-4e30-a4d2-c96f7d18d498\") "
Nov 01 05:31:29.643332 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.643312    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b94fa739-43bf-4e30-a4d2-c96f7d18d498-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b94fa739-43bf-4e30-a4d2-c96f7d18d498-maas-api-tls\") pod \"b94fa739-43bf-4e30-a4d2-c96f7d18d498\" (UID: \"b94fa739-43bf-4e30-a4d2-c96f7d18d498\") "
Nov 01 05:31:29.643466 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.643371    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b94fa739-43bf-4e30-a4d2-c96f7d18d498-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b94fa739-43bf-4e30-a4d2-c96f7d18d498-metrics-certs\") pod \"b94fa739-43bf-4e30-a4d2-c96f7d18d498\" (UID: \"b94fa739-43bf-4e30-a4d2-c96f7d18d498\") "
Nov 01 05:31:29.643642 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.643614    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b94fa739-43bf-4e30-a4d2-c96f7d18d498-openshift-service-ca" pod "b94fa739-43bf-4e30-a4d2-c96f7d18d498" (UID: "b94fa739-43bf-4e30-a4d2-c96f7d18d498"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:31:29.643908 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.643887    4864 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b94fa739-43bf-4e30-a4d2-c96f7d18d498-openshift-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:31:29.645589 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.645561    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b94fa739-43bf-4e30-a4d2-c96f7d18d498-kube-api-access-82qnm" pod "b94fa739-43bf-4e30-a4d2-c96f7d18d498" (UID: "b94fa739-43bf-4e30-a4d2-c96f7d18d498"). InnerVolumeSpecName "kube-api-access-82qnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:31:29.646379 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.646348    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b94fa739-43bf-4e30-a4d2-c96f7d18d498-metrics-certs" pod "b94fa739-43bf-4e30-a4d2-c96f7d18d498" (UID: "b94fa739-43bf-4e30-a4d2-c96f7d18d498"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:31:29.646594 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.646570    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b94fa739-43bf-4e30-a4d2-c96f7d18d498-maas-api-tls" pod "b94fa739-43bf-4e30-a4d2-c96f7d18d498" (UID: "b94fa739-43bf-4e30-a4d2-c96f7d18d498"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:31:29.675581 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.675548    4864 generic.go:356] "Generic (PLEG): container finished" podID="b94fa739-43bf-4e30-a4d2-c96f7d18d498" containerID="4f58d5c8facf50bbdfacb0e33b18d34ac05652659c0067c0326f54df626ffaa7" exitCode=0
Nov 01 05:31:29.675724 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.675632    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r" event={"ID":"b94fa739-43bf-4e30-a4d2-c96f7d18d498","Type":"ContainerDied","Data":"4f58d5c8facf50bbdfacb0e33b18d34ac05652659c0067c0326f54df626ffaa7"}
Nov 01 05:31:29.675724 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.675669    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r" event={"ID":"b94fa739-43bf-4e30-a4d2-c96f7d18d498","Type":"ContainerDied","Data":"15594a4b1a67e40ccfb9ac76542bdcb3ed48d44c6a3c7ffba70b2ac56eaef7e0"}
Nov 01 05:31:29.675724 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.675690    4864 scope.go:122] "RemoveContainer" containerID="4f58d5c8facf50bbdfacb0e33b18d34ac05652659c0067c0326f54df626ffaa7"
Nov 01 05:31:29.675988 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.675835    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"
Nov 01 05:31:29.695481 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.695461    4864 scope.go:122] "RemoveContainer" containerID="4f58d5c8facf50bbdfacb0e33b18d34ac05652659c0067c0326f54df626ffaa7"
Nov 01 05:31:29.695895 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:31:29.695872    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f58d5c8facf50bbdfacb0e33b18d34ac05652659c0067c0326f54df626ffaa7\": container with ID starting with 4f58d5c8facf50bbdfacb0e33b18d34ac05652659c0067c0326f54df626ffaa7 not found: ID does not exist" containerID="4f58d5c8facf50bbdfacb0e33b18d34ac05652659c0067c0326f54df626ffaa7"
Nov 01 05:31:29.695972 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.695907    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f58d5c8facf50bbdfacb0e33b18d34ac05652659c0067c0326f54df626ffaa7"} err="failed to get container status \"4f58d5c8facf50bbdfacb0e33b18d34ac05652659c0067c0326f54df626ffaa7\": rpc error: code = NotFound desc = could not find container \"4f58d5c8facf50bbdfacb0e33b18d34ac05652659c0067c0326f54df626ffaa7\": container with ID starting with 4f58d5c8facf50bbdfacb0e33b18d34ac05652659c0067c0326f54df626ffaa7 not found: ID does not exist"
Nov 01 05:31:29.745071 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.745036    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b94fa739-43bf-4e30-a4d2-c96f7d18d498-maas-api-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:31:29.745071 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.745070    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b94fa739-43bf-4e30-a4d2-c96f7d18d498-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:31:29.745285 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.745083    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82qnm\" (UniqueName: \"kubernetes.io/projected/b94fa739-43bf-4e30-a4d2-c96f7d18d498-kube-api-access-82qnm\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:31:29.841776 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.841726    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"]
Nov 01 05:31:29.845704 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:29.845673    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-918b8d-85b69786cd-fgg4r"]
Nov 01 05:31:30.315718 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:30.315673    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:30.316191 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:30.316072    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:30.317297 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:30.317277    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:31:31.251599 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:31:31.251558    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b94fa739-43bf-4e30-a4d2-c96f7d18d498" path="/var/lib/kubelet/pods/b94fa739-43bf-4e30-a4d2-c96f7d18d498/volumes"
Nov 01 05:31:52.549834 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:31:52.549775    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:32:22.524976 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:32:22.524935    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:32:23.670979 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:23.670948    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"]
Nov 01 05:32:23.671427 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:23.671176    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9" podUID="cc0858b2-dff5-49b3-b702-514824d14d14" containerName="maas-api" containerID="cri-o://77dbbeea86e55947624b2eceaaa17c22757fabf08ef19805a9ebc68b8fd66a93" gracePeriod=30
Nov 01 05:32:24.041605 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.041578    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"
Nov 01 05:32:24.075831 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.075765    4864 generic.go:356] "Generic (PLEG): container finished" podID="cc0858b2-dff5-49b3-b702-514824d14d14" containerID="77dbbeea86e55947624b2eceaaa17c22757fabf08ef19805a9ebc68b8fd66a93" exitCode=0
Nov 01 05:32:24.076032 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.075837    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9" event={"ID":"cc0858b2-dff5-49b3-b702-514824d14d14","Type":"ContainerDied","Data":"77dbbeea86e55947624b2eceaaa17c22757fabf08ef19805a9ebc68b8fd66a93"}
Nov 01 05:32:24.076032 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.075887    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9" event={"ID":"cc0858b2-dff5-49b3-b702-514824d14d14","Type":"ContainerDied","Data":"98acae046bf4014da0a4c2ced5ca20b0aaa807f1670a91f55d67d1795c01b93e"}
Nov 01 05:32:24.076032 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.075917    4864 scope.go:122] "RemoveContainer" containerID="77dbbeea86e55947624b2eceaaa17c22757fabf08ef19805a9ebc68b8fd66a93"
Nov 01 05:32:24.076202 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.076071    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"
Nov 01 05:32:24.102324 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.102297    4864 scope.go:122] "RemoveContainer" containerID="77dbbeea86e55947624b2eceaaa17c22757fabf08ef19805a9ebc68b8fd66a93"
Nov 01 05:32:24.103154 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:32:24.103121    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77dbbeea86e55947624b2eceaaa17c22757fabf08ef19805a9ebc68b8fd66a93\": container with ID starting with 77dbbeea86e55947624b2eceaaa17c22757fabf08ef19805a9ebc68b8fd66a93 not found: ID does not exist" containerID="77dbbeea86e55947624b2eceaaa17c22757fabf08ef19805a9ebc68b8fd66a93"
Nov 01 05:32:24.103391 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.103170    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77dbbeea86e55947624b2eceaaa17c22757fabf08ef19805a9ebc68b8fd66a93"} err="failed to get container status \"77dbbeea86e55947624b2eceaaa17c22757fabf08ef19805a9ebc68b8fd66a93\": rpc error: code = NotFound desc = could not find container \"77dbbeea86e55947624b2eceaaa17c22757fabf08ef19805a9ebc68b8fd66a93\": container with ID starting with 77dbbeea86e55947624b2eceaaa17c22757fabf08ef19805a9ebc68b8fd66a93 not found: ID does not exist"
Nov 01 05:32:24.146581 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.146534    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cc0858b2-dff5-49b3-b702-514824d14d14-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc0858b2-dff5-49b3-b702-514824d14d14-maas-api-tls\") pod \"cc0858b2-dff5-49b3-b702-514824d14d14\" (UID: \"cc0858b2-dff5-49b3-b702-514824d14d14\") "
Nov 01 05:32:24.146814 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.146618    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc0858b2-dff5-49b3-b702-514824d14d14-kube-api-access-89p2d\" (UniqueName: \"kubernetes.io/projected/cc0858b2-dff5-49b3-b702-514824d14d14-kube-api-access-89p2d\") pod \"cc0858b2-dff5-49b3-b702-514824d14d14\" (UID: \"cc0858b2-dff5-49b3-b702-514824d14d14\") "
Nov 01 05:32:24.146814 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.146739    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cc0858b2-dff5-49b3-b702-514824d14d14-metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc0858b2-dff5-49b3-b702-514824d14d14-metrics-certs\") pod \"cc0858b2-dff5-49b3-b702-514824d14d14\" (UID: \"cc0858b2-dff5-49b3-b702-514824d14d14\") "
Nov 01 05:32:24.146961 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.146836    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cc0858b2-dff5-49b3-b702-514824d14d14-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc0858b2-dff5-49b3-b702-514824d14d14-openshift-service-ca\") pod \"cc0858b2-dff5-49b3-b702-514824d14d14\" (UID: \"cc0858b2-dff5-49b3-b702-514824d14d14\") "
Nov 01 05:32:24.147487 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.147461    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc0858b2-dff5-49b3-b702-514824d14d14-openshift-service-ca" pod "cc0858b2-dff5-49b3-b702-514824d14d14" (UID: "cc0858b2-dff5-49b3-b702-514824d14d14"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:32:24.149664 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.149627    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc0858b2-dff5-49b3-b702-514824d14d14-maas-api-tls" pod "cc0858b2-dff5-49b3-b702-514824d14d14" (UID: "cc0858b2-dff5-49b3-b702-514824d14d14"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:32:24.149664 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.149648    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc0858b2-dff5-49b3-b702-514824d14d14-metrics-certs" pod "cc0858b2-dff5-49b3-b702-514824d14d14" (UID: "cc0858b2-dff5-49b3-b702-514824d14d14"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:32:24.149887 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.149762    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc0858b2-dff5-49b3-b702-514824d14d14-kube-api-access-89p2d" pod "cc0858b2-dff5-49b3-b702-514824d14d14" (UID: "cc0858b2-dff5-49b3-b702-514824d14d14"). InnerVolumeSpecName "kube-api-access-89p2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:32:24.248743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.248659    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc0858b2-dff5-49b3-b702-514824d14d14-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:32:24.248743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.248693    4864 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc0858b2-dff5-49b3-b702-514824d14d14-openshift-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:32:24.248743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.248707    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc0858b2-dff5-49b3-b702-514824d14d14-maas-api-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:32:24.248743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.248719    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89p2d\" (UniqueName: \"kubernetes.io/projected/cc0858b2-dff5-49b3-b702-514824d14d14-kube-api-access-89p2d\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:32:24.495898 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.495868    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"]
Nov 01 05:32:24.500294 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:24.500222    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-ca7a55-57c5dbc879-jmzk9"]
Nov 01 05:32:25.247971 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:32:25.247935    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cc0858b2-dff5-49b3-b702-514824d14d14" path="/var/lib/kubelet/pods/cc0858b2-dff5-49b3-b702-514824d14d14/volumes"
Nov 01 05:32:52.524636 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:32:52.524597    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:33:03.730400 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:03.730365    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:33:03.738374 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:03.738349    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:33:03.739367 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:03.739344    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:33:03.746696 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:03.746676    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:33:22.525761 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:33:22.525719    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:33:29.684096 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.684062    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"]
Nov 01 05:33:29.684641 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.684626    4864 cpu_manager.go:414] "containerMap: removing container" podUID="b94fa739-43bf-4e30-a4d2-c96f7d18d498" containerName="maas-api"
Nov 01 05:33:29.684689 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.684642    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b94fa739-43bf-4e30-a4d2-c96f7d18d498" containerName="maas-api"
Nov 01 05:33:29.684689 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.684657    4864 cpu_manager.go:414] "containerMap: removing container" podUID="cc0858b2-dff5-49b3-b702-514824d14d14" containerName="maas-api"
Nov 01 05:33:29.684689 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.684662    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc0858b2-dff5-49b3-b702-514824d14d14" containerName="maas-api"
Nov 01 05:33:29.684806 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.684747    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="b94fa739-43bf-4e30-a4d2-c96f7d18d498" containerName="maas-api"
Nov 01 05:33:29.684806 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.684756    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc0858b2-dff5-49b3-b702-514824d14d14" containerName="maas-api"
Nov 01 05:33:29.688251 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.688225    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"
Nov 01 05:33:29.740256 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.740227    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ec52ff90-serving-cert\""
Nov 01 05:33:29.795262 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.795230    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"]
Nov 01 05:33:29.832721 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.832687    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11336c88-aa11-40b2-8770-1a7253bc8dd0-metrics-certs\") pod \"maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp\" (UID: \"11336c88-aa11-40b2-8770-1a7253bc8dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"
Nov 01 05:33:29.832960 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.832877    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11336c88-aa11-40b2-8770-1a7253bc8dd0-maas-api-tls\") pod \"maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp\" (UID: \"11336c88-aa11-40b2-8770-1a7253bc8dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"
Nov 01 05:33:29.832960 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.832912    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd2ln\" (UniqueName: \"kubernetes.io/projected/11336c88-aa11-40b2-8770-1a7253bc8dd0-kube-api-access-kd2ln\") pod \"maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp\" (UID: \"11336c88-aa11-40b2-8770-1a7253bc8dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"
Nov 01 05:33:29.832960 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.832940    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11336c88-aa11-40b2-8770-1a7253bc8dd0-openshift-service-ca\") pod \"maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp\" (UID: \"11336c88-aa11-40b2-8770-1a7253bc8dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"
Nov 01 05:33:29.933929 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.933889    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11336c88-aa11-40b2-8770-1a7253bc8dd0-metrics-certs\") pod \"maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp\" (UID: \"11336c88-aa11-40b2-8770-1a7253bc8dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"
Nov 01 05:33:29.934164 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.934020    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11336c88-aa11-40b2-8770-1a7253bc8dd0-maas-api-tls\") pod \"maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp\" (UID: \"11336c88-aa11-40b2-8770-1a7253bc8dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"
Nov 01 05:33:29.934164 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.934058    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd2ln\" (UniqueName: \"kubernetes.io/projected/11336c88-aa11-40b2-8770-1a7253bc8dd0-kube-api-access-kd2ln\") pod \"maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp\" (UID: \"11336c88-aa11-40b2-8770-1a7253bc8dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"
Nov 01 05:33:29.934164 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.934089    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11336c88-aa11-40b2-8770-1a7253bc8dd0-openshift-service-ca\") pod \"maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp\" (UID: \"11336c88-aa11-40b2-8770-1a7253bc8dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"
Nov 01 05:33:29.934851 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.934814    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11336c88-aa11-40b2-8770-1a7253bc8dd0-openshift-service-ca\") pod \"maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp\" (UID: \"11336c88-aa11-40b2-8770-1a7253bc8dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"
Nov 01 05:33:29.936526 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.936498    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11336c88-aa11-40b2-8770-1a7253bc8dd0-maas-api-tls\") pod \"maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp\" (UID: \"11336c88-aa11-40b2-8770-1a7253bc8dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"
Nov 01 05:33:29.936665 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.936644    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11336c88-aa11-40b2-8770-1a7253bc8dd0-metrics-certs\") pod \"maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp\" (UID: \"11336c88-aa11-40b2-8770-1a7253bc8dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"
Nov 01 05:33:29.997929 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:29.997901    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd2ln\" (UniqueName: \"kubernetes.io/projected/11336c88-aa11-40b2-8770-1a7253bc8dd0-kube-api-access-kd2ln\") pod \"maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp\" (UID: \"11336c88-aa11-40b2-8770-1a7253bc8dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"
Nov 01 05:33:30.001847 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:30.001816    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"
Nov 01 05:33:30.426459 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:30.413341    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"]
Nov 01 05:33:30.542420 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:30.542377    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp" event={"ID":"11336c88-aa11-40b2-8770-1a7253bc8dd0","Type":"ContainerStarted","Data":"7d7553b29928738971a13a96a17ebe697ae865407aa94d6411053ac584f49050"}
Nov 01 05:33:31.550478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:31.550433    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp" event={"ID":"11336c88-aa11-40b2-8770-1a7253bc8dd0","Type":"ContainerStarted","Data":"5f1c58beff0e3526c37385e3d5648ed5e2fba38e0d242c84717d75ae8036cbe1"}
Nov 01 05:33:31.551036 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:31.550567    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"
Nov 01 05:33:31.668519 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:31.668420    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp" podStartSLOduration=1.753401889 podStartE2EDuration="2.668406008s" totalImagesPullingTime="915.004119ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:33:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:33:31.666589041 +0000 UTC m=+1828.556965027" watchObservedRunningTime="2026-11-01 05:33:31.668406008 +0000 UTC m=+1828.558781983"
Nov 01 05:33:37.561829 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:33:37.561776    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"
Nov 01 05:33:52.524251 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:33:52.524165    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:34:15.917376 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:15.917342    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"]
Nov 01 05:34:15.917982 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:15.917635    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8" podUID="0acea0d8-ee39-4724-8c86-4396234a5654" containerName="istio-proxy" containerID="cri-o://f9f9c0e58476f13f6c249b5f2a3f1cf0ef955149b4d887d31a93e7eafb507960" gracePeriod=30
Nov 01 05:34:17.077432 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:17.077387    4864 patch_prober.go:29] interesting pod/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 05:34:17.078000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:17.077465    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8" podUID="0acea0d8-ee39-4724-8c86-4396234a5654" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 05:34:20.174114 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:20.174078    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"]
Nov 01 05:34:20.174614 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:20.174277    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5" podUID="59d81eeb-a67f-4bc9-a951-3145d53aef60" containerName="istio-proxy" containerID="cri-o://5b72186da76c62f65d774b6eda5d07981f48d09426895b19db9a1ab15b2efdef" gracePeriod=30
Nov 01 05:34:20.223496 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:20.223465    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"]
Nov 01 05:34:20.223697 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:20.223653    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s" podUID="e3695950-3ae2-48f4-a299-8a398f7640d9" containerName="istio-proxy" containerID="cri-o://37ec1eff988ce87db85bd6fea734e1fbbd173bb4b966aa5f45acfbc2345040e1" gracePeriod=30
Nov 01 05:34:21.188885 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.188815    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:34:21.275048 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.275010    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0acea0d8-ee39-4724-8c86-4396234a5654-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0acea0d8-ee39-4724-8c86-4396234a5654-istio-podinfo\") pod \"0acea0d8-ee39-4724-8c86-4396234a5654\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") "
Nov 01 05:34:21.275234 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.275068    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-credential-socket\") pod \"0acea0d8-ee39-4724-8c86-4396234a5654\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") "
Nov 01 05:34:21.275234 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.275099    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0acea0d8-ee39-4724-8c86-4396234a5654-kube-api-access-xrzbz\" (UniqueName: \"kubernetes.io/projected/0acea0d8-ee39-4724-8c86-4396234a5654-kube-api-access-xrzbz\") pod \"0acea0d8-ee39-4724-8c86-4396234a5654\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") "
Nov 01 05:34:21.275234 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.275150    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-workload-certs\") pod \"0acea0d8-ee39-4724-8c86-4396234a5654\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") "
Nov 01 05:34:21.275234 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.275212    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-istio-data\") pod \"0acea0d8-ee39-4724-8c86-4396234a5654\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") "
Nov 01 05:34:21.275530 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.275246    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-workload-socket\") pod \"0acea0d8-ee39-4724-8c86-4396234a5654\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") "
Nov 01 05:34:21.275530 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.275355    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0acea0d8-ee39-4724-8c86-4396234a5654-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0acea0d8-ee39-4724-8c86-4396234a5654-istiod-ca-cert\") pod \"0acea0d8-ee39-4724-8c86-4396234a5654\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") "
Nov 01 05:34:21.275530 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.275424    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0acea0d8-ee39-4724-8c86-4396234a5654-istio-token\" (UniqueName: \"kubernetes.io/projected/0acea0d8-ee39-4724-8c86-4396234a5654-istio-token\") pod \"0acea0d8-ee39-4724-8c86-4396234a5654\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") "
Nov 01 05:34:21.275530 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.275471    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-istio-envoy\") pod \"0acea0d8-ee39-4724-8c86-4396234a5654\" (UID: \"0acea0d8-ee39-4724-8c86-4396234a5654\") "
Nov 01 05:34:21.275729 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.275583    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-istio-data" pod "0acea0d8-ee39-4724-8c86-4396234a5654" (UID: "0acea0d8-ee39-4724-8c86-4396234a5654"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:34:21.275729 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.275596    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-workload-socket" pod "0acea0d8-ee39-4724-8c86-4396234a5654" (UID: "0acea0d8-ee39-4724-8c86-4396234a5654"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:34:21.275729 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.275616    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-workload-certs" pod "0acea0d8-ee39-4724-8c86-4396234a5654" (UID: "0acea0d8-ee39-4724-8c86-4396234a5654"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:34:21.275729 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.275712    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-credential-socket" pod "0acea0d8-ee39-4724-8c86-4396234a5654" (UID: "0acea0d8-ee39-4724-8c86-4396234a5654"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:34:21.276022 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.275827    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0acea0d8-ee39-4724-8c86-4396234a5654-istiod-ca-cert" pod "0acea0d8-ee39-4724-8c86-4396234a5654" (UID: "0acea0d8-ee39-4724-8c86-4396234a5654"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:34:21.276104 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.276048    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:21.276104 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.276071    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:21.276210 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.276125    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:21.277645 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.277624    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0acea0d8-ee39-4724-8c86-4396234a5654-istio-podinfo" pod "0acea0d8-ee39-4724-8c86-4396234a5654" (UID: "0acea0d8-ee39-4724-8c86-4396234a5654"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 05:34:21.278322 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.278300    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0acea0d8-ee39-4724-8c86-4396234a5654-kube-api-access-xrzbz" pod "0acea0d8-ee39-4724-8c86-4396234a5654" (UID: "0acea0d8-ee39-4724-8c86-4396234a5654"). InnerVolumeSpecName "kube-api-access-xrzbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:34:21.278434 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.278324    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0acea0d8-ee39-4724-8c86-4396234a5654-istio-token" pod "0acea0d8-ee39-4724-8c86-4396234a5654" (UID: "0acea0d8-ee39-4724-8c86-4396234a5654"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:34:21.278434 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.278402    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-istio-envoy" pod "0acea0d8-ee39-4724-8c86-4396234a5654" (UID: "0acea0d8-ee39-4724-8c86-4396234a5654"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:34:21.377499 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.377461    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0acea0d8-ee39-4724-8c86-4396234a5654-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:21.377499 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.377494    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:21.377499 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.377508    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrzbz\" (UniqueName: \"kubernetes.io/projected/0acea0d8-ee39-4724-8c86-4396234a5654-kube-api-access-xrzbz\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:21.377763 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.377521    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0acea0d8-ee39-4724-8c86-4396234a5654-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:21.377763 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.377533    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0acea0d8-ee39-4724-8c86-4396234a5654-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:21.377763 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.377545    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0acea0d8-ee39-4724-8c86-4396234a5654-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:21.681844 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.681808    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-8lzd5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 05:34:21.682009 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.681876    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5" podUID="59d81eeb-a67f-4bc9-a951-3145d53aef60" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 05:34:21.888585 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.888549    4864 generic.go:356] "Generic (PLEG): container finished" podID="0acea0d8-ee39-4724-8c86-4396234a5654" containerID="f9f9c0e58476f13f6c249b5f2a3f1cf0ef955149b4d887d31a93e7eafb507960" exitCode=0
Nov 01 05:34:21.888752 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.888630    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"
Nov 01 05:34:21.888752 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.888630    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8" event={"ID":"0acea0d8-ee39-4724-8c86-4396234a5654","Type":"ContainerDied","Data":"f9f9c0e58476f13f6c249b5f2a3f1cf0ef955149b4d887d31a93e7eafb507960"}
Nov 01 05:34:21.888868 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.888780    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8" event={"ID":"0acea0d8-ee39-4724-8c86-4396234a5654","Type":"ContainerDied","Data":"d9175050e1b7b27a90f0e559f21a690900d545c857207cb23c1b23ca03254224"}
Nov 01 05:34:21.888868 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.888820    4864 scope.go:122] "RemoveContainer" containerID="f9f9c0e58476f13f6c249b5f2a3f1cf0ef955149b4d887d31a93e7eafb507960"
Nov 01 05:34:21.900828 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.900804    4864 scope.go:122] "RemoveContainer" containerID="f9f9c0e58476f13f6c249b5f2a3f1cf0ef955149b4d887d31a93e7eafb507960"
Nov 01 05:34:21.901178 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:34:21.901152    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9f9c0e58476f13f6c249b5f2a3f1cf0ef955149b4d887d31a93e7eafb507960\": container with ID starting with f9f9c0e58476f13f6c249b5f2a3f1cf0ef955149b4d887d31a93e7eafb507960 not found: ID does not exist" containerID="f9f9c0e58476f13f6c249b5f2a3f1cf0ef955149b4d887d31a93e7eafb507960"
Nov 01 05:34:21.901301 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:21.901190    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9f9c0e58476f13f6c249b5f2a3f1cf0ef955149b4d887d31a93e7eafb507960"} err="failed to get container status \"f9f9c0e58476f13f6c249b5f2a3f1cf0ef955149b4d887d31a93e7eafb507960\": rpc error: code = NotFound desc = could not find container \"f9f9c0e58476f13f6c249b5f2a3f1cf0ef955149b4d887d31a93e7eafb507960\": container with ID starting with f9f9c0e58476f13f6c249b5f2a3f1cf0ef955149b4d887d31a93e7eafb507960 not found: ID does not exist"
Nov 01 05:34:22.053156 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:22.053130    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"]
Nov 01 05:34:22.102349 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:22.102310    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8"]
Nov 01 05:34:22.526100 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:34:22.526020    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:34:23.247971 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:23.247937    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0acea0d8-ee39-4724-8c86-4396234a5654" path="/var/lib/kubelet/pods/0acea0d8-ee39-4724-8c86-4396234a5654/volumes"
Nov 01 05:34:25.262187 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.262159    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-px79s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 05:34:25.262544 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.262232    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s" podUID="e3695950-3ae2-48f4-a299-8a398f7640d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 05:34:25.520139 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.520117    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:34:25.534599 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.534567    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:34:25.625817 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.625768    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-data\") pod \"59d81eeb-a67f-4bc9-a951-3145d53aef60\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") "
Nov 01 05:34:25.626016 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.625832    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3695950-3ae2-48f4-a299-8a398f7640d9-kube-api-access-5xzv8\" (UniqueName: \"kubernetes.io/projected/e3695950-3ae2-48f4-a299-8a398f7640d9-kube-api-access-5xzv8\") pod \"e3695950-3ae2-48f4-a299-8a398f7640d9\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") "
Nov 01 05:34:25.626016 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.625889    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-envoy\") pod \"e3695950-3ae2-48f4-a299-8a398f7640d9\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") "
Nov 01 05:34:25.626016 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.625941    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-data\") pod \"e3695950-3ae2-48f4-a299-8a398f7640d9\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") "
Nov 01 05:34:25.626016 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.626008    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59d81eeb-a67f-4bc9-a951-3145d53aef60-kube-api-access-gxrct\" (UniqueName: \"kubernetes.io/projected/59d81eeb-a67f-4bc9-a951-3145d53aef60-kube-api-access-gxrct\") pod \"59d81eeb-a67f-4bc9-a951-3145d53aef60\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") "
Nov 01 05:34:25.626274 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.626037    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/59d81eeb-a67f-4bc9-a951-3145d53aef60-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59d81eeb-a67f-4bc9-a951-3145d53aef60-istiod-ca-cert\") pod \"59d81eeb-a67f-4bc9-a951-3145d53aef60\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") "
Nov 01 05:34:25.626274 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.626039    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-data" pod "59d81eeb-a67f-4bc9-a951-3145d53aef60" (UID: "59d81eeb-a67f-4bc9-a951-3145d53aef60"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:34:25.626274 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.626097    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-workload-certs\") pod \"e3695950-3ae2-48f4-a299-8a398f7640d9\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") "
Nov 01 05:34:25.626274 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.626145    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-workload-certs\") pod \"59d81eeb-a67f-4bc9-a951-3145d53aef60\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") "
Nov 01 05:34:25.626274 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.626180    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-podinfo\") pod \"59d81eeb-a67f-4bc9-a951-3145d53aef60\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") "
Nov 01 05:34:25.626274 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.626209    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3695950-3ae2-48f4-a299-8a398f7640d9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3695950-3ae2-48f4-a299-8a398f7640d9-istiod-ca-cert\") pod \"e3695950-3ae2-48f4-a299-8a398f7640d9\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") "
Nov 01 05:34:25.626274 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.626246    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-workload-socket\") pod \"59d81eeb-a67f-4bc9-a951-3145d53aef60\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") "
Nov 01 05:34:25.626599 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.626488    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-workload-socket" pod "59d81eeb-a67f-4bc9-a951-3145d53aef60" (UID: "59d81eeb-a67f-4bc9-a951-3145d53aef60"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:34:25.626665 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.626614    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-token\" (UniqueName: \"kubernetes.io/projected/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-token\") pod \"e3695950-3ae2-48f4-a299-8a398f7640d9\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") "
Nov 01 05:34:25.626721 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.626690    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-token\" (UniqueName: \"kubernetes.io/projected/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-token\") pod \"59d81eeb-a67f-4bc9-a951-3145d53aef60\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") "
Nov 01 05:34:25.628050 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.626815    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-credential-socket\") pod \"e3695950-3ae2-48f4-a299-8a398f7640d9\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") "
Nov 01 05:34:25.628050 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.626942    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59d81eeb-a67f-4bc9-a951-3145d53aef60-istiod-ca-cert" pod "59d81eeb-a67f-4bc9-a951-3145d53aef60" (UID: "59d81eeb-a67f-4bc9-a951-3145d53aef60"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:34:25.628050 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.627024    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-envoy\") pod \"59d81eeb-a67f-4bc9-a951-3145d53aef60\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") "
Nov 01 05:34:25.628050 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.627098    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-workload-socket\") pod \"e3695950-3ae2-48f4-a299-8a398f7640d9\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") "
Nov 01 05:34:25.628050 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.627140    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-podinfo\") pod \"e3695950-3ae2-48f4-a299-8a398f7640d9\" (UID: \"e3695950-3ae2-48f4-a299-8a398f7640d9\") "
Nov 01 05:34:25.628050 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.627190    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-credential-socket\") pod \"59d81eeb-a67f-4bc9-a951-3145d53aef60\" (UID: \"59d81eeb-a67f-4bc9-a951-3145d53aef60\") "
Nov 01 05:34:25.628050 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.627521    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-workload-certs" pod "e3695950-3ae2-48f4-a299-8a398f7640d9" (UID: "e3695950-3ae2-48f4-a299-8a398f7640d9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:34:25.628050 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.627727    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-credential-socket" pod "59d81eeb-a67f-4bc9-a951-3145d53aef60" (UID: "59d81eeb-a67f-4bc9-a951-3145d53aef60"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:34:25.628050 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.627899    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-workload-certs" pod "59d81eeb-a67f-4bc9-a951-3145d53aef60" (UID: "59d81eeb-a67f-4bc9-a951-3145d53aef60"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:34:25.628601 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.628076    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-credential-socket" pod "e3695950-3ae2-48f4-a299-8a398f7640d9" (UID: "e3695950-3ae2-48f4-a299-8a398f7640d9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:34:25.628601 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.628333    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:25.628601 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.628355    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:25.628601 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.628371    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:25.628601 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.628389    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:25.628601 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.628400    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:25.628601 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.628411    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:25.628601 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.628492    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-workload-socket" pod "e3695950-3ae2-48f4-a299-8a398f7640d9" (UID: "e3695950-3ae2-48f4-a299-8a398f7640d9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:34:25.629067 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.628675    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59d81eeb-a67f-4bc9-a951-3145d53aef60-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:25.629067 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.628940    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59d81eeb-a67f-4bc9-a951-3145d53aef60-kube-api-access-gxrct" pod "59d81eeb-a67f-4bc9-a951-3145d53aef60" (UID: "59d81eeb-a67f-4bc9-a951-3145d53aef60"). InnerVolumeSpecName "kube-api-access-gxrct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:34:25.629172 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.629118    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3695950-3ae2-48f4-a299-8a398f7640d9-istiod-ca-cert" pod "e3695950-3ae2-48f4-a299-8a398f7640d9" (UID: "e3695950-3ae2-48f4-a299-8a398f7640d9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:34:25.629530 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.629453    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-data" pod "e3695950-3ae2-48f4-a299-8a398f7640d9" (UID: "e3695950-3ae2-48f4-a299-8a398f7640d9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:34:25.629772 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.629749    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-envoy" pod "e3695950-3ae2-48f4-a299-8a398f7640d9" (UID: "e3695950-3ae2-48f4-a299-8a398f7640d9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:34:25.630062 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.630036    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-podinfo" pod "59d81eeb-a67f-4bc9-a951-3145d53aef60" (UID: "59d81eeb-a67f-4bc9-a951-3145d53aef60"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 05:34:25.630246 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.630226    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3695950-3ae2-48f4-a299-8a398f7640d9-kube-api-access-5xzv8" pod "e3695950-3ae2-48f4-a299-8a398f7640d9" (UID: "e3695950-3ae2-48f4-a299-8a398f7640d9"). InnerVolumeSpecName "kube-api-access-5xzv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:34:25.631042 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.630959    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-podinfo" pod "e3695950-3ae2-48f4-a299-8a398f7640d9" (UID: "e3695950-3ae2-48f4-a299-8a398f7640d9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 05:34:25.631299 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.631277    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-token" pod "59d81eeb-a67f-4bc9-a951-3145d53aef60" (UID: "59d81eeb-a67f-4bc9-a951-3145d53aef60"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:34:25.631536 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.631514    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-token" pod "e3695950-3ae2-48f4-a299-8a398f7640d9" (UID: "e3695950-3ae2-48f4-a299-8a398f7640d9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:34:25.631825 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.631766    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-envoy" pod "59d81eeb-a67f-4bc9-a951-3145d53aef60" (UID: "59d81eeb-a67f-4bc9-a951-3145d53aef60"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:34:25.730030 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.729996    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:25.730176 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.730034    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxrct\" (UniqueName: \"kubernetes.io/projected/59d81eeb-a67f-4bc9-a951-3145d53aef60-kube-api-access-gxrct\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:25.730176 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.730050    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:25.730176 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.730065    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3695950-3ae2-48f4-a299-8a398f7640d9-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:25.730176 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.730078    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:25.730176 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.730092    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:25.730176 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.730106    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59d81eeb-a67f-4bc9-a951-3145d53aef60-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:25.730176 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.730120    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:25.730176 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.730150    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:25.730176 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.730165    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xzv8\" (UniqueName: \"kubernetes.io/projected/e3695950-3ae2-48f4-a299-8a398f7640d9-kube-api-access-5xzv8\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:25.730446 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.730178    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3695950-3ae2-48f4-a299-8a398f7640d9-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:34:25.920977 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.920946    4864 generic.go:356] "Generic (PLEG): container finished" podID="59d81eeb-a67f-4bc9-a951-3145d53aef60" containerID="5b72186da76c62f65d774b6eda5d07981f48d09426895b19db9a1ab15b2efdef" exitCode=0
Nov 01 05:34:25.921196 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.920995    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"
Nov 01 05:34:25.921196 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.921038    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5" event={"ID":"59d81eeb-a67f-4bc9-a951-3145d53aef60","Type":"ContainerDied","Data":"5b72186da76c62f65d774b6eda5d07981f48d09426895b19db9a1ab15b2efdef"}
Nov 01 05:34:25.921196 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.921105    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5" event={"ID":"59d81eeb-a67f-4bc9-a951-3145d53aef60","Type":"ContainerDied","Data":"5a993a0ee2e3b0bc8a0091d69389dad6ddb5423eead6f9b8a02852738c6e5586"}
Nov 01 05:34:25.921196 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.921136    4864 scope.go:122] "RemoveContainer" containerID="5b72186da76c62f65d774b6eda5d07981f48d09426895b19db9a1ab15b2efdef"
Nov 01 05:34:25.922527 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.922505    4864 generic.go:356] "Generic (PLEG): container finished" podID="e3695950-3ae2-48f4-a299-8a398f7640d9" containerID="37ec1eff988ce87db85bd6fea734e1fbbd173bb4b966aa5f45acfbc2345040e1" exitCode=0
Nov 01 05:34:25.922649 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.922540    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s" event={"ID":"e3695950-3ae2-48f4-a299-8a398f7640d9","Type":"ContainerDied","Data":"37ec1eff988ce87db85bd6fea734e1fbbd173bb4b966aa5f45acfbc2345040e1"}
Nov 01 05:34:25.922649 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.922565    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"
Nov 01 05:34:25.922649 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.922592    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s" event={"ID":"e3695950-3ae2-48f4-a299-8a398f7640d9","Type":"ContainerDied","Data":"01a1786acd6c8a41f64fb361375d0a5f127d22539a50ed4d47ce63e00d555414"}
Nov 01 05:34:25.932380 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.932362    4864 scope.go:122] "RemoveContainer" containerID="5b72186da76c62f65d774b6eda5d07981f48d09426895b19db9a1ab15b2efdef"
Nov 01 05:34:25.932715 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:34:25.932693    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b72186da76c62f65d774b6eda5d07981f48d09426895b19db9a1ab15b2efdef\": container with ID starting with 5b72186da76c62f65d774b6eda5d07981f48d09426895b19db9a1ab15b2efdef not found: ID does not exist" containerID="5b72186da76c62f65d774b6eda5d07981f48d09426895b19db9a1ab15b2efdef"
Nov 01 05:34:25.932834 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.932727    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b72186da76c62f65d774b6eda5d07981f48d09426895b19db9a1ab15b2efdef"} err="failed to get container status \"5b72186da76c62f65d774b6eda5d07981f48d09426895b19db9a1ab15b2efdef\": rpc error: code = NotFound desc = could not find container \"5b72186da76c62f65d774b6eda5d07981f48d09426895b19db9a1ab15b2efdef\": container with ID starting with 5b72186da76c62f65d774b6eda5d07981f48d09426895b19db9a1ab15b2efdef not found: ID does not exist"
Nov 01 05:34:25.932834 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.932752    4864 scope.go:122] "RemoveContainer" containerID="37ec1eff988ce87db85bd6fea734e1fbbd173bb4b966aa5f45acfbc2345040e1"
Nov 01 05:34:25.943689 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.943672    4864 scope.go:122] "RemoveContainer" containerID="37ec1eff988ce87db85bd6fea734e1fbbd173bb4b966aa5f45acfbc2345040e1"
Nov 01 05:34:25.943976 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:34:25.943958    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37ec1eff988ce87db85bd6fea734e1fbbd173bb4b966aa5f45acfbc2345040e1\": container with ID starting with 37ec1eff988ce87db85bd6fea734e1fbbd173bb4b966aa5f45acfbc2345040e1 not found: ID does not exist" containerID="37ec1eff988ce87db85bd6fea734e1fbbd173bb4b966aa5f45acfbc2345040e1"
Nov 01 05:34:25.944042 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:25.943981    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37ec1eff988ce87db85bd6fea734e1fbbd173bb4b966aa5f45acfbc2345040e1"} err="failed to get container status \"37ec1eff988ce87db85bd6fea734e1fbbd173bb4b966aa5f45acfbc2345040e1\": rpc error: code = NotFound desc = could not find container \"37ec1eff988ce87db85bd6fea734e1fbbd173bb4b966aa5f45acfbc2345040e1\": container with ID starting with 37ec1eff988ce87db85bd6fea734e1fbbd173bb4b966aa5f45acfbc2345040e1 not found: ID does not exist"
Nov 01 05:34:26.085595 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:26.085559    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"]
Nov 01 05:34:26.091368 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:26.091341    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s"]
Nov 01 05:34:26.200912 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:26.200838    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"]
Nov 01 05:34:26.213352 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:26.213324    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5"]
Nov 01 05:34:27.248059 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:27.248029    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="59d81eeb-a67f-4bc9-a951-3145d53aef60" path="/var/lib/kubelet/pods/59d81eeb-a67f-4bc9-a951-3145d53aef60/volumes"
Nov 01 05:34:27.248510 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:34:27.248494    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3695950-3ae2-48f4-a299-8a398f7640d9" path="/var/lib/kubelet/pods/e3695950-3ae2-48f4-a299-8a398f7640d9/volumes"
Nov 01 05:34:52.524219 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:34:52.524177    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:35:16.905719 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:16.905630    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"]
Nov 01 05:35:16.906252 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:16.905861    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp" podUID="11336c88-aa11-40b2-8770-1a7253bc8dd0" containerName="maas-api" containerID="cri-o://5f1c58beff0e3526c37385e3d5648ed5e2fba38e0d242c84717d75ae8036cbe1" gracePeriod=30
Nov 01 05:35:17.244298 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.244268    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"
Nov 01 05:35:17.275013 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.274981    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/11336c88-aa11-40b2-8770-1a7253bc8dd0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11336c88-aa11-40b2-8770-1a7253bc8dd0-maas-api-tls\") pod \"11336c88-aa11-40b2-8770-1a7253bc8dd0\" (UID: \"11336c88-aa11-40b2-8770-1a7253bc8dd0\") "
Nov 01 05:35:17.275190 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.275088    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11336c88-aa11-40b2-8770-1a7253bc8dd0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11336c88-aa11-40b2-8770-1a7253bc8dd0-openshift-service-ca\") pod \"11336c88-aa11-40b2-8770-1a7253bc8dd0\" (UID: \"11336c88-aa11-40b2-8770-1a7253bc8dd0\") "
Nov 01 05:35:17.275190 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.275157    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/11336c88-aa11-40b2-8770-1a7253bc8dd0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/11336c88-aa11-40b2-8770-1a7253bc8dd0-metrics-certs\") pod \"11336c88-aa11-40b2-8770-1a7253bc8dd0\" (UID: \"11336c88-aa11-40b2-8770-1a7253bc8dd0\") "
Nov 01 05:35:17.275190 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.275186    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11336c88-aa11-40b2-8770-1a7253bc8dd0-kube-api-access-kd2ln\" (UniqueName: \"kubernetes.io/projected/11336c88-aa11-40b2-8770-1a7253bc8dd0-kube-api-access-kd2ln\") pod \"11336c88-aa11-40b2-8770-1a7253bc8dd0\" (UID: \"11336c88-aa11-40b2-8770-1a7253bc8dd0\") "
Nov 01 05:35:17.275450 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.275411    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11336c88-aa11-40b2-8770-1a7253bc8dd0-openshift-service-ca" pod "11336c88-aa11-40b2-8770-1a7253bc8dd0" (UID: "11336c88-aa11-40b2-8770-1a7253bc8dd0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:35:17.275603 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.275576    4864 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11336c88-aa11-40b2-8770-1a7253bc8dd0-openshift-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:35:17.277371 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.277346    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11336c88-aa11-40b2-8770-1a7253bc8dd0-metrics-certs" pod "11336c88-aa11-40b2-8770-1a7253bc8dd0" (UID: "11336c88-aa11-40b2-8770-1a7253bc8dd0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:35:17.277466 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.277381    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11336c88-aa11-40b2-8770-1a7253bc8dd0-maas-api-tls" pod "11336c88-aa11-40b2-8770-1a7253bc8dd0" (UID: "11336c88-aa11-40b2-8770-1a7253bc8dd0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:35:17.277466 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.277446    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11336c88-aa11-40b2-8770-1a7253bc8dd0-kube-api-access-kd2ln" pod "11336c88-aa11-40b2-8770-1a7253bc8dd0" (UID: "11336c88-aa11-40b2-8770-1a7253bc8dd0"). InnerVolumeSpecName "kube-api-access-kd2ln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:35:17.289259 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.289225    4864 generic.go:356] "Generic (PLEG): container finished" podID="11336c88-aa11-40b2-8770-1a7253bc8dd0" containerID="5f1c58beff0e3526c37385e3d5648ed5e2fba38e0d242c84717d75ae8036cbe1" exitCode=0
Nov 01 05:35:17.289391 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.289328    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"
Nov 01 05:35:17.289391 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.289357    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp" event={"ID":"11336c88-aa11-40b2-8770-1a7253bc8dd0","Type":"ContainerDied","Data":"5f1c58beff0e3526c37385e3d5648ed5e2fba38e0d242c84717d75ae8036cbe1"}
Nov 01 05:35:17.289463 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.289418    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp" event={"ID":"11336c88-aa11-40b2-8770-1a7253bc8dd0","Type":"ContainerDied","Data":"7d7553b29928738971a13a96a17ebe697ae865407aa94d6411053ac584f49050"}
Nov 01 05:35:17.289463 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.289450    4864 scope.go:122] "RemoveContainer" containerID="5f1c58beff0e3526c37385e3d5648ed5e2fba38e0d242c84717d75ae8036cbe1"
Nov 01 05:35:17.302745 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.302722    4864 scope.go:122] "RemoveContainer" containerID="5f1c58beff0e3526c37385e3d5648ed5e2fba38e0d242c84717d75ae8036cbe1"
Nov 01 05:35:17.303116 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:35:17.303097    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f1c58beff0e3526c37385e3d5648ed5e2fba38e0d242c84717d75ae8036cbe1\": container with ID starting with 5f1c58beff0e3526c37385e3d5648ed5e2fba38e0d242c84717d75ae8036cbe1 not found: ID does not exist" containerID="5f1c58beff0e3526c37385e3d5648ed5e2fba38e0d242c84717d75ae8036cbe1"
Nov 01 05:35:17.303176 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.303127    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f1c58beff0e3526c37385e3d5648ed5e2fba38e0d242c84717d75ae8036cbe1"} err="failed to get container status \"5f1c58beff0e3526c37385e3d5648ed5e2fba38e0d242c84717d75ae8036cbe1\": rpc error: code = NotFound desc = could not find container \"5f1c58beff0e3526c37385e3d5648ed5e2fba38e0d242c84717d75ae8036cbe1\": container with ID starting with 5f1c58beff0e3526c37385e3d5648ed5e2fba38e0d242c84717d75ae8036cbe1 not found: ID does not exist"
Nov 01 05:35:17.376665 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.376628    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11336c88-aa11-40b2-8770-1a7253bc8dd0-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:35:17.376873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.376671    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kd2ln\" (UniqueName: \"kubernetes.io/projected/11336c88-aa11-40b2-8770-1a7253bc8dd0-kube-api-access-kd2ln\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:35:17.376873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.376688    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11336c88-aa11-40b2-8770-1a7253bc8dd0-maas-api-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:35:17.479282 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.479200    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"]
Nov 01 05:35:17.482572 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:17.482548    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ec52ff90-858d867fc9-76pnp"]
Nov 01 05:35:19.270354 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:35:19.270313    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11336c88-aa11-40b2-8770-1a7253bc8dd0" path="/var/lib/kubelet/pods/11336c88-aa11-40b2-8770-1a7253bc8dd0/volumes"
Nov 01 05:35:22.526153 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:35:22.526116    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:35:52.525910 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:35:52.525873    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:36:01.561920 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.561885    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt"]
Nov 01 05:36:01.562423 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.562408    4864 cpu_manager.go:414] "containerMap: removing container" podUID="0acea0d8-ee39-4724-8c86-4396234a5654" containerName="istio-proxy"
Nov 01 05:36:01.562483 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.562425    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0acea0d8-ee39-4724-8c86-4396234a5654" containerName="istio-proxy"
Nov 01 05:36:01.562483 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.562444    4864 cpu_manager.go:414] "containerMap: removing container" podUID="11336c88-aa11-40b2-8770-1a7253bc8dd0" containerName="maas-api"
Nov 01 05:36:01.562483 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.562449    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11336c88-aa11-40b2-8770-1a7253bc8dd0" containerName="maas-api"
Nov 01 05:36:01.562483 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.562462    4864 cpu_manager.go:414] "containerMap: removing container" podUID="59d81eeb-a67f-4bc9-a951-3145d53aef60" containerName="istio-proxy"
Nov 01 05:36:01.562483 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.562468    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="59d81eeb-a67f-4bc9-a951-3145d53aef60" containerName="istio-proxy"
Nov 01 05:36:01.562483 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.562478    4864 cpu_manager.go:414] "containerMap: removing container" podUID="e3695950-3ae2-48f4-a299-8a398f7640d9" containerName="istio-proxy"
Nov 01 05:36:01.562483 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.562483    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3695950-3ae2-48f4-a299-8a398f7640d9" containerName="istio-proxy"
Nov 01 05:36:01.562715 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.562554    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="0acea0d8-ee39-4724-8c86-4396234a5654" containerName="istio-proxy"
Nov 01 05:36:01.562715 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.562564    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="11336c88-aa11-40b2-8770-1a7253bc8dd0" containerName="maas-api"
Nov 01 05:36:01.562715 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.562571    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="59d81eeb-a67f-4bc9-a951-3145d53aef60" containerName="istio-proxy"
Nov 01 05:36:01.562715 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.562578    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3695950-3ae2-48f4-a299-8a398f7640d9" containerName="istio-proxy"
Nov 01 05:36:01.566163 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.566143    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt"
Nov 01 05:36:01.624298 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.624268    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt"]
Nov 01 05:36:01.669316 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.669223    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-extmod-shr-user-b73da0\"/\"kube-root-ca.crt\""
Nov 01 05:36:01.669316 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.669289    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-extmod-shr-user-b73da0\"/\"e2e-usage-sim-59820a-cert\""
Nov 01 05:36:01.669316 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.669308    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-extmod-shr-user-b73da0\"/\"openshift-service-ca.crt\""
Nov 01 05:36:01.669639 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.669408    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-extmod-shr-user-b73da0\"/\"default-dockercfg-s5zhv\""
Nov 01 05:36:01.700600 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.700556    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/0df126d4-02ca-453d-92eb-2ce949558897-tls\") pod \"e2e-usage-sim-59820a-6cb498f45f-xklnt\" (UID: \"0df126d4-02ca-453d-92eb-2ce949558897\") " pod="e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt"
Nov 01 05:36:01.700828 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.700638    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qw76\" (UniqueName: \"kubernetes.io/projected/0df126d4-02ca-453d-92eb-2ce949558897-kube-api-access-6qw76\") pod \"e2e-usage-sim-59820a-6cb498f45f-xklnt\" (UID: \"0df126d4-02ca-453d-92eb-2ce949558897\") " pod="e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt"
Nov 01 05:36:01.801798 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.801754    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/0df126d4-02ca-453d-92eb-2ce949558897-tls\") pod \"e2e-usage-sim-59820a-6cb498f45f-xklnt\" (UID: \"0df126d4-02ca-453d-92eb-2ce949558897\") " pod="e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt"
Nov 01 05:36:01.801980 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.801825    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qw76\" (UniqueName: \"kubernetes.io/projected/0df126d4-02ca-453d-92eb-2ce949558897-kube-api-access-6qw76\") pod \"e2e-usage-sim-59820a-6cb498f45f-xklnt\" (UID: \"0df126d4-02ca-453d-92eb-2ce949558897\") " pod="e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt"
Nov 01 05:36:01.804663 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.804634    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls\" (UniqueName: \"kubernetes.io/secret/0df126d4-02ca-453d-92eb-2ce949558897-tls\") pod \"e2e-usage-sim-59820a-6cb498f45f-xklnt\" (UID: \"0df126d4-02ca-453d-92eb-2ce949558897\") " pod="e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt"
Nov 01 05:36:01.863230 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.863140    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qw76\" (UniqueName: \"kubernetes.io/projected/0df126d4-02ca-453d-92eb-2ce949558897-kube-api-access-6qw76\") pod \"e2e-usage-sim-59820a-6cb498f45f-xklnt\" (UID: \"0df126d4-02ca-453d-92eb-2ce949558897\") " pod="e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt"
Nov 01 05:36:01.881003 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:01.880974    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt"
Nov 01 05:36:02.260611 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:02.260581    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt"]
Nov 01 05:36:02.262101 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:36:02.262068    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0df126d4_02ca_453d_92eb_2ce949558897.slice/crio-6766e0d5d2df552879ea52a81385473f3a3d5c844cabbb22bb267b94b19079b0 WatchSource:0}: Error finding container 6766e0d5d2df552879ea52a81385473f3a3d5c844cabbb22bb267b94b19079b0: Status 404 returned error can't find the container with id 6766e0d5d2df552879ea52a81385473f3a3d5c844cabbb22bb267b94b19079b0
Nov 01 05:36:02.600922 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:02.600882    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt" event={"ID":"0df126d4-02ca-453d-92eb-2ce949558897","Type":"ContainerStarted","Data":"6d32dc50ea36d4c1845c97acb57f34402336ff7c31a5cc9bf7ae88121bbcea0e"}
Nov 01 05:36:02.600922 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:02.600926    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt" event={"ID":"0df126d4-02ca-453d-92eb-2ce949558897","Type":"ContainerStarted","Data":"6766e0d5d2df552879ea52a81385473f3a3d5c844cabbb22bb267b94b19079b0"}
Nov 01 05:36:02.601465 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:02.601032    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt"
Nov 01 05:36:02.606802 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:02.606765    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt"
Nov 01 05:36:02.716894 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:02.716843    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt" podStartSLOduration=1.716778634 podStartE2EDuration="1.716778634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:36:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:36:02.715773127 +0000 UTC m=+1979.606149103" watchObservedRunningTime="2026-11-01 05:36:02.716778634 +0000 UTC m=+1979.607154638"
Nov 01 05:36:06.002400 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.002364    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"]
Nov 01 05:36:06.007565 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.007541    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.112149 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.112110    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"]
Nov 01 05:36:06.148185 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.148148    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.148388 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.148206    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bthms\" (UniqueName: \"kubernetes.io/projected/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-kube-api-access-bthms\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.148388 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.148339    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.148520 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.148384    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.148520 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.148508    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.148605 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.148553    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.148660 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.148649    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.148699 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.148667    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.148758 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.148708    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.249325 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.249287    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.249484 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.249363    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.249484 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.249390    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.249484 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.249449    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.249484 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.249479    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.249710 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.249520    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bthms\" (UniqueName: \"kubernetes.io/projected/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-kube-api-access-bthms\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.249710 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.249551    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.249710 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.249674    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.249916 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.249819    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.249916 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.249901    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.250026 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.249958    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.250083 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.250046    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.250140 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.250124    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.250331 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.250310    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.252056 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.252033    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.252494 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.252452    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.310396 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.310357    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.311009 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.310983    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bthms\" (UniqueName: \"kubernetes.io/projected/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-kube-api-access-bthms\") pod \"maas-default-gateway-openshift-default-58d898bd55-xsdd7\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.321674 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.321637    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:06.753018 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.752995    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"]
Nov 01 05:36:06.754226 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:36:06.754201    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc37f8bd_cf86_47e5_823d_9e9d357ddf13.slice/crio-89adc441efa829bf9e4744bb8e95ce1e55be6a6d34776ba058438e6ef418759b WatchSource:0}: Error finding container 89adc441efa829bf9e4744bb8e95ce1e55be6a6d34776ba058438e6ef418759b: Status 404 returned error can't find the container with id 89adc441efa829bf9e4744bb8e95ce1e55be6a6d34776ba058438e6ef418759b
Nov 01 05:36:06.756679 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.756649    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:36:06.756746 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.756715    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:36:06.756812 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:06.756744    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:36:07.637871 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:07.637835    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7" event={"ID":"dc37f8bd-cf86-47e5-823d-9e9d357ddf13","Type":"ContainerStarted","Data":"0336bcdf655f593c34d79fee5b5ef12ff3b45aaeef308567f464c552866ae28b"}
Nov 01 05:36:07.638340 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:07.637899    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7" event={"ID":"dc37f8bd-cf86-47e5-823d-9e9d357ddf13","Type":"ContainerStarted","Data":"89adc441efa829bf9e4744bb8e95ce1e55be6a6d34776ba058438e6ef418759b"}
Nov 01 05:36:07.767409 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:07.767339    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7" podStartSLOduration=2.767317568 podStartE2EDuration="2.767317568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:36:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:36:07.76223906 +0000 UTC m=+1984.652615036" watchObservedRunningTime="2026-11-01 05:36:07.767317568 +0000 UTC m=+1984.657693544"
Nov 01 05:36:08.321979 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:08.321936    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:08.324064 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:08.324041    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-xsdd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 01 05:36:08.324147 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:08.324093    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7" podUID="dc37f8bd-cf86-47e5-823d-9e9d357ddf13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 01 05:36:09.322279 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:09.322243    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-xsdd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 01 05:36:09.322834 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:09.322295    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7" podUID="dc37f8bd-cf86-47e5-823d-9e9d357ddf13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 01 05:36:10.322246 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:10.322213    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-xsdd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 01 05:36:10.322401 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:10.322265    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7" podUID="dc37f8bd-cf86-47e5-823d-9e9d357ddf13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 01 05:36:11.322136 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:11.322091    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-xsdd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 01 05:36:11.322362 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:11.322154    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7" podUID="dc37f8bd-cf86-47e5-823d-9e9d357ddf13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 01 05:36:12.322230 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:12.322183    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-xsdd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 01 05:36:12.322768 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:12.322241    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7" podUID="dc37f8bd-cf86-47e5-823d-9e9d357ddf13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 01 05:36:13.258237 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:13.258208    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"]
Nov 01 05:36:13.262360 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:13.262339    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"
Nov 01 05:36:13.321870 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:13.321837    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"]
Nov 01 05:36:13.322243 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:13.322217    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-xsdd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 01 05:36:13.322411 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:13.322389    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7" podUID="dc37f8bd-cf86-47e5-823d-9e9d357ddf13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 01 05:36:13.366465 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:13.366428    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-c6c6fdb9-serving-cert\""
Nov 01 05:36:13.376173 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:13.376133    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7668fa0-e004-4890-89a7-04d6efe40724-metrics-certs\") pod \"maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss\" (UID: \"a7668fa0-e004-4890-89a7-04d6efe40724\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"
Nov 01 05:36:13.376400 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:13.376231    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7668fa0-e004-4890-89a7-04d6efe40724-openshift-service-ca\") pod \"maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss\" (UID: \"a7668fa0-e004-4890-89a7-04d6efe40724\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"
Nov 01 05:36:13.376400 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:13.376296    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzmk2\" (UniqueName: \"kubernetes.io/projected/a7668fa0-e004-4890-89a7-04d6efe40724-kube-api-access-wzmk2\") pod \"maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss\" (UID: \"a7668fa0-e004-4890-89a7-04d6efe40724\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"
Nov 01 05:36:13.376583 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:13.376468    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7668fa0-e004-4890-89a7-04d6efe40724-maas-api-tls\") pod \"maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss\" (UID: \"a7668fa0-e004-4890-89a7-04d6efe40724\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"
Nov 01 05:36:13.478686 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:13.478644    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7668fa0-e004-4890-89a7-04d6efe40724-maas-api-tls\") pod \"maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss\" (UID: \"a7668fa0-e004-4890-89a7-04d6efe40724\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"
Nov 01 05:36:13.478953 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:13.478709    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7668fa0-e004-4890-89a7-04d6efe40724-metrics-certs\") pod \"maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss\" (UID: \"a7668fa0-e004-4890-89a7-04d6efe40724\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"
Nov 01 05:36:13.478953 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:13.478744    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7668fa0-e004-4890-89a7-04d6efe40724-openshift-service-ca\") pod \"maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss\" (UID: \"a7668fa0-e004-4890-89a7-04d6efe40724\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"
Nov 01 05:36:13.478953 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:13.478812    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzmk2\" (UniqueName: \"kubernetes.io/projected/a7668fa0-e004-4890-89a7-04d6efe40724-kube-api-access-wzmk2\") pod \"maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss\" (UID: \"a7668fa0-e004-4890-89a7-04d6efe40724\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"
Nov 01 05:36:13.478953 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:36:13.478830    4864 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-serving-cert: secret "maas-api-e2e-migrate-c6c6fdb9-serving-cert" not found
Nov 01 05:36:13.478953 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:36:13.478907    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7668fa0-e004-4890-89a7-04d6efe40724-maas-api-tls podName:a7668fa0-e004-4890-89a7-04d6efe40724 nodeName:}" failed. No retries permitted until 2026-11-01 05:36:13.978885861 +0000 UTC m=+1990.869261828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a7668fa0-e004-4890-89a7-04d6efe40724-maas-api-tls") pod "maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss" (UID: "a7668fa0-e004-4890-89a7-04d6efe40724") : secret "maas-api-e2e-migrate-c6c6fdb9-serving-cert" not found
Nov 01 05:36:13.479495 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:13.479474    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7668fa0-e004-4890-89a7-04d6efe40724-openshift-service-ca\") pod \"maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss\" (UID: \"a7668fa0-e004-4890-89a7-04d6efe40724\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"
Nov 01 05:36:13.481711 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:13.481684    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7668fa0-e004-4890-89a7-04d6efe40724-metrics-certs\") pod \"maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss\" (UID: \"a7668fa0-e004-4890-89a7-04d6efe40724\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"
Nov 01 05:36:13.559871 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:13.559836    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzmk2\" (UniqueName: \"kubernetes.io/projected/a7668fa0-e004-4890-89a7-04d6efe40724-kube-api-access-wzmk2\") pod \"maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss\" (UID: \"a7668fa0-e004-4890-89a7-04d6efe40724\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"
Nov 01 05:36:13.985065 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:13.984983    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7668fa0-e004-4890-89a7-04d6efe40724-maas-api-tls\") pod \"maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss\" (UID: \"a7668fa0-e004-4890-89a7-04d6efe40724\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"
Nov 01 05:36:13.987935 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:13.987903    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7668fa0-e004-4890-89a7-04d6efe40724-maas-api-tls\") pod \"maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss\" (UID: \"a7668fa0-e004-4890-89a7-04d6efe40724\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"
Nov 01 05:36:14.180222 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:14.180181    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"
Nov 01 05:36:14.322906 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:14.322860    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-xsdd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 01 05:36:14.323391 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:14.322930    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7" podUID="dc37f8bd-cf86-47e5-823d-9e9d357ddf13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 01 05:36:14.568488 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:14.568374    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"]
Nov 01 05:36:14.569709 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:36:14.569682    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7668fa0_e004_4890_89a7_04d6efe40724.slice/crio-0aa3d1107604c567cb9555825abd3e839bce462f191c2435de5143c592cfe965 WatchSource:0}: Error finding container 0aa3d1107604c567cb9555825abd3e839bce462f191c2435de5143c592cfe965: Status 404 returned error can't find the container with id 0aa3d1107604c567cb9555825abd3e839bce462f191c2435de5143c592cfe965
Nov 01 05:36:14.571313 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:14.571293    4864 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 05:36:14.709963 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:14.709902    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss" event={"ID":"a7668fa0-e004-4890-89a7-04d6efe40724","Type":"ContainerStarted","Data":"0aa3d1107604c567cb9555825abd3e839bce462f191c2435de5143c592cfe965"}
Nov 01 05:36:15.637228 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:15.636134    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:15.637228 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:15.636751    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:15.640492 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:15.640470    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 05:36:15.722136 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:15.722094    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss" event={"ID":"a7668fa0-e004-4890-89a7-04d6efe40724","Type":"ContainerStarted","Data":"ec3d4be58a0a250771c6945b7dda2a67f1e52fe75da39bc2c91285d34083790a"}
Nov 01 05:36:15.722341 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:15.722147    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"
Nov 01 05:36:15.915610 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:15.915543    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss" podStartSLOduration=1.960263318 podStartE2EDuration="2.915524876s" totalImagesPullingTime="955.261558ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:36:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:36:15.910730674 +0000 UTC m=+1992.801106645" watchObservedRunningTime="2026-11-01 05:36:15.915524876 +0000 UTC m=+1992.805900855"
Nov 01 05:36:21.733199 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:21.733172    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"
Nov 01 05:36:22.525095 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:36:22.525036    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:36:36.254231 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.254190    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"]
Nov 01 05:36:36.258368 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.258346    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.364148 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.364110    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.364346 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.364157    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.364346 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.364316    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.364346 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.364340    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.364516 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.364359    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.364516 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.364400    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.364516 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.364437    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9x5t\" (UniqueName: \"kubernetes.io/projected/51619ded-3924-4581-a8ae-8d2cb8dd9609-kube-api-access-h9x5t\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.364516 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.364480    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.364722 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.364522    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51619ded-3924-4581-a8ae-8d2cb8dd9609-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.466043 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.466004    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"]
Nov 01 05:36:36.466355 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.466189    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.466684 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.466658    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.466869 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.466852    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.467005 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.466991    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.467112 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.467096    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9x5t\" (UniqueName: \"kubernetes.io/projected/51619ded-3924-4581-a8ae-8d2cb8dd9609-kube-api-access-h9x5t\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.467433 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.467383    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.467556 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.467536    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51619ded-3924-4581-a8ae-8d2cb8dd9609-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.467688 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.467571    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.467688 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.467637    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.468239 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.468216    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.468518 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.468491    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.468518 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.468510    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.468769 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.468740    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.469048 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.468740    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51619ded-3924-4581-a8ae-8d2cb8dd9609-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.471675 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.471646    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.473299 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.473269    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.528622 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.528543    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.528812 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.528643    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9x5t\" (UniqueName: \"kubernetes.io/projected/51619ded-3924-4581-a8ae-8d2cb8dd9609-kube-api-access-h9x5t\") pod \"maas-default-gateway-openshift-default-58d898bd55-9gnnl\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:36.572628 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:36.572589    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:37.009010 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:37.008974    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"]
Nov 01 05:36:37.010460 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:36:37.010417    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51619ded_3924_4581_a8ae_8d2cb8dd9609.slice/crio-d093ff081788c03db35f37729c9795225e6525b842e35958604a60f831b52add WatchSource:0}: Error finding container d093ff081788c03db35f37729c9795225e6525b842e35958604a60f831b52add: Status 404 returned error can't find the container with id d093ff081788c03db35f37729c9795225e6525b842e35958604a60f831b52add
Nov 01 05:36:37.013199 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:37.013167    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:36:37.013273 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:37.013237    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:36:37.013273 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:37.013264    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:36:37.892157 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:37.892117    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl" event={"ID":"51619ded-3924-4581-a8ae-8d2cb8dd9609","Type":"ContainerStarted","Data":"9d12fac431256cd89730835ced6eb81440f525cac8db8ea004b9f70eaec3467c"}
Nov 01 05:36:37.892666 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:37.892179    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl" event={"ID":"51619ded-3924-4581-a8ae-8d2cb8dd9609","Type":"ContainerStarted","Data":"d093ff081788c03db35f37729c9795225e6525b842e35958604a60f831b52add"}
Nov 01 05:36:38.575223 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:38.575190    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:38.578923 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:38.578891    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-9gnnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 01 05:36:38.579168 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:38.579139    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl" podUID="51619ded-3924-4581-a8ae-8d2cb8dd9609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 01 05:36:39.573811 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:39.573209    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-9gnnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 01 05:36:39.573811 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:39.573289    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl" podUID="51619ded-3924-4581-a8ae-8d2cb8dd9609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 01 05:36:40.573256 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:40.573215    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-9gnnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 01 05:36:40.573568 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:40.573539    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl" podUID="51619ded-3924-4581-a8ae-8d2cb8dd9609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 01 05:36:41.576169 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:41.576133    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-9gnnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 01 05:36:41.576760 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:41.576730    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl" podUID="51619ded-3924-4581-a8ae-8d2cb8dd9609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 01 05:36:42.575220 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:42.575177    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-9gnnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 01 05:36:42.575629 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:42.575598    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl" podUID="51619ded-3924-4581-a8ae-8d2cb8dd9609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 01 05:36:43.573311 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:43.573267    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-9gnnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 01 05:36:43.573870 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:43.573331    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl" podUID="51619ded-3924-4581-a8ae-8d2cb8dd9609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 01 05:36:44.573155 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:44.573106    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-9gnnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 01 05:36:44.573343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:44.573177    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl" podUID="51619ded-3924-4581-a8ae-8d2cb8dd9609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 01 05:36:45.577732 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:45.577695    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:45.578220 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:45.578155    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:45.578857 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:45.578839    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 05:36:45.753817 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:36:45.753744    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl" podStartSLOduration=9.753730713 podStartE2EDuration="9.753730713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:36:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:36:38.063116949 +0000 UTC m=+2014.953492926" watchObservedRunningTime="2026-11-01 05:36:45.753730713 +0000 UTC m=+2022.644106689"
Nov 01 05:36:52.524769 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:36:52.524727    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:37:22.525102 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:37:22.525063    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:37:24.349193 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:24.349152    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"]
Nov 01 05:37:24.349674 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:24.349461    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t" podUID="4002ccda-eb43-49ff-90e6-5ce4e72a70c1" containerName="istio-proxy" containerID="cri-o://e73a32d3862b9699f1ad9edc69179c51ae484cdb8653112949dc2cf6c1c1298b" gracePeriod=30
Nov 01 05:37:29.709581 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.709549    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:37:29.730707 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.730671    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istiod-ca-cert\") pod \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") "
Nov 01 05:37:29.730929 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.730761    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-envoy\") pod \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") "
Nov 01 05:37:29.731136 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.731112    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istiod-ca-cert" pod "4002ccda-eb43-49ff-90e6-5ce4e72a70c1" (UID: "4002ccda-eb43-49ff-90e6-5ce4e72a70c1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:37:29.731401 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.731363    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-kube-api-access-bv4fx\" (UniqueName: \"kubernetes.io/projected/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-kube-api-access-bv4fx\") pod \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") "
Nov 01 05:37:29.731515 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.731429    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-token\" (UniqueName: \"kubernetes.io/projected/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-token\") pod \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") "
Nov 01 05:37:29.731515 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.731524    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-podinfo\") pod \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") "
Nov 01 05:37:29.731805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.731571    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-data\") pod \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") "
Nov 01 05:37:29.731805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.731605    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-credential-socket\") pod \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") "
Nov 01 05:37:29.731805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.731703    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-workload-certs\") pod \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") "
Nov 01 05:37:29.731805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.731749    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-workload-socket\") pod \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\" (UID: \"4002ccda-eb43-49ff-90e6-5ce4e72a70c1\") "
Nov 01 05:37:29.732632 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.732604    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-credential-socket" pod "4002ccda-eb43-49ff-90e6-5ce4e72a70c1" (UID: "4002ccda-eb43-49ff-90e6-5ce4e72a70c1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:37:29.732844 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.732821    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-data" pod "4002ccda-eb43-49ff-90e6-5ce4e72a70c1" (UID: "4002ccda-eb43-49ff-90e6-5ce4e72a70c1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:37:29.733416 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.733383    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-workload-certs" pod "4002ccda-eb43-49ff-90e6-5ce4e72a70c1" (UID: "4002ccda-eb43-49ff-90e6-5ce4e72a70c1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:37:29.733687 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.733662    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-workload-socket" pod "4002ccda-eb43-49ff-90e6-5ce4e72a70c1" (UID: "4002ccda-eb43-49ff-90e6-5ce4e72a70c1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:37:29.764329 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.757991    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-envoy" pod "4002ccda-eb43-49ff-90e6-5ce4e72a70c1" (UID: "4002ccda-eb43-49ff-90e6-5ce4e72a70c1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:37:29.764329 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.758204    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-token" pod "4002ccda-eb43-49ff-90e6-5ce4e72a70c1" (UID: "4002ccda-eb43-49ff-90e6-5ce4e72a70c1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:37:29.764329 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.758531    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:37:29.764329 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.758555    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:37:29.764329 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.758572    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:37:29.764329 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.758588    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:37:29.764329 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.758601    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:37:29.764329 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.758616    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:37:29.764329 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.758629    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:37:29.773884 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.772150    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-podinfo" pod "4002ccda-eb43-49ff-90e6-5ce4e72a70c1" (UID: "4002ccda-eb43-49ff-90e6-5ce4e72a70c1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 05:37:29.815430 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.815355    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-kube-api-access-bv4fx" pod "4002ccda-eb43-49ff-90e6-5ce4e72a70c1" (UID: "4002ccda-eb43-49ff-90e6-5ce4e72a70c1"). InnerVolumeSpecName "kube-api-access-bv4fx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:37:29.860335 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.860288    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bv4fx\" (UniqueName: \"kubernetes.io/projected/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-kube-api-access-bv4fx\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:37:29.860335 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:29.860335    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4002ccda-eb43-49ff-90e6-5ce4e72a70c1-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:37:30.258034 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:30.257998    4864 generic.go:356] "Generic (PLEG): container finished" podID="4002ccda-eb43-49ff-90e6-5ce4e72a70c1" containerID="e73a32d3862b9699f1ad9edc69179c51ae484cdb8653112949dc2cf6c1c1298b" exitCode=0
Nov 01 05:37:30.258299 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:30.258078    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t" event={"ID":"4002ccda-eb43-49ff-90e6-5ce4e72a70c1","Type":"ContainerDied","Data":"e73a32d3862b9699f1ad9edc69179c51ae484cdb8653112949dc2cf6c1c1298b"}
Nov 01 05:37:30.258410 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:30.258336    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t" event={"ID":"4002ccda-eb43-49ff-90e6-5ce4e72a70c1","Type":"ContainerDied","Data":"3c3626f071d50abc1e2a27d664b89f0d39845a5d671a510eae2266077eb627b4"}
Nov 01 05:37:30.258410 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:30.258153    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"
Nov 01 05:37:30.258410 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:30.258372    4864 scope.go:122] "RemoveContainer" containerID="e73a32d3862b9699f1ad9edc69179c51ae484cdb8653112949dc2cf6c1c1298b"
Nov 01 05:37:30.271561 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:30.271183    4864 scope.go:122] "RemoveContainer" containerID="e73a32d3862b9699f1ad9edc69179c51ae484cdb8653112949dc2cf6c1c1298b"
Nov 01 05:37:30.272002 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:37:30.271897    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e73a32d3862b9699f1ad9edc69179c51ae484cdb8653112949dc2cf6c1c1298b\": container with ID starting with e73a32d3862b9699f1ad9edc69179c51ae484cdb8653112949dc2cf6c1c1298b not found: ID does not exist" containerID="e73a32d3862b9699f1ad9edc69179c51ae484cdb8653112949dc2cf6c1c1298b"
Nov 01 05:37:30.272002 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:30.271935    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e73a32d3862b9699f1ad9edc69179c51ae484cdb8653112949dc2cf6c1c1298b"} err="failed to get container status \"e73a32d3862b9699f1ad9edc69179c51ae484cdb8653112949dc2cf6c1c1298b\": rpc error: code = NotFound desc = could not find container \"e73a32d3862b9699f1ad9edc69179c51ae484cdb8653112949dc2cf6c1c1298b\": container with ID starting with e73a32d3862b9699f1ad9edc69179c51ae484cdb8653112949dc2cf6c1c1298b not found: ID does not exist"
Nov 01 05:37:30.429228 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:30.429198    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"]
Nov 01 05:37:30.476595 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:30.476565    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t"]
Nov 01 05:37:31.247945 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:31.247912    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4002ccda-eb43-49ff-90e6-5ce4e72a70c1" path="/var/lib/kubelet/pods/4002ccda-eb43-49ff-90e6-5ce4e72a70c1/volumes"
Nov 01 05:37:31.627979 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:31.627937    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-9gnnl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 05:37:31.628175 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:31.628004    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl" podUID="51619ded-3924-4581-a8ae-8d2cb8dd9609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 05:37:31.684364 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:31.684327    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-xsdd7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 05:37:31.684683 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:37:31.684657    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7" podUID="dc37f8bd-cf86-47e5-823d-9e9d357ddf13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 05:37:52.529453 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:37:52.529406    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:38:00.134971 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.134917    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"]
Nov 01 05:38:00.135556 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.135233    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss" podUID="a7668fa0-e004-4890-89a7-04d6efe40724" containerName="maas-api" containerID="cri-o://ec3d4be58a0a250771c6945b7dda2a67f1e52fe75da39bc2c91285d34083790a" gracePeriod=30
Nov 01 05:38:00.444314 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.444279    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"
Nov 01 05:38:00.540810 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.525150    4864 generic.go:356] "Generic (PLEG): container finished" podID="a7668fa0-e004-4890-89a7-04d6efe40724" containerID="ec3d4be58a0a250771c6945b7dda2a67f1e52fe75da39bc2c91285d34083790a" exitCode=0
Nov 01 05:38:00.540810 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.525343    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss" event={"ID":"a7668fa0-e004-4890-89a7-04d6efe40724","Type":"ContainerDied","Data":"ec3d4be58a0a250771c6945b7dda2a67f1e52fe75da39bc2c91285d34083790a"}
Nov 01 05:38:00.540810 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.525395    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss" event={"ID":"a7668fa0-e004-4890-89a7-04d6efe40724","Type":"ContainerDied","Data":"0aa3d1107604c567cb9555825abd3e839bce462f191c2435de5143c592cfe965"}
Nov 01 05:38:00.540810 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.525425    4864 scope.go:122] "RemoveContainer" containerID="ec3d4be58a0a250771c6945b7dda2a67f1e52fe75da39bc2c91285d34083790a"
Nov 01 05:38:00.540810 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.525600    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"
Nov 01 05:38:00.549478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.549436    4864 scope.go:122] "RemoveContainer" containerID="ec3d4be58a0a250771c6945b7dda2a67f1e52fe75da39bc2c91285d34083790a"
Nov 01 05:38:00.550411 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:38:00.550377    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec3d4be58a0a250771c6945b7dda2a67f1e52fe75da39bc2c91285d34083790a\": container with ID starting with ec3d4be58a0a250771c6945b7dda2a67f1e52fe75da39bc2c91285d34083790a not found: ID does not exist" containerID="ec3d4be58a0a250771c6945b7dda2a67f1e52fe75da39bc2c91285d34083790a"
Nov 01 05:38:00.550570 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.550425    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec3d4be58a0a250771c6945b7dda2a67f1e52fe75da39bc2c91285d34083790a"} err="failed to get container status \"ec3d4be58a0a250771c6945b7dda2a67f1e52fe75da39bc2c91285d34083790a\": rpc error: code = NotFound desc = could not find container \"ec3d4be58a0a250771c6945b7dda2a67f1e52fe75da39bc2c91285d34083790a\": container with ID starting with ec3d4be58a0a250771c6945b7dda2a67f1e52fe75da39bc2c91285d34083790a not found: ID does not exist"
Nov 01 05:38:00.571433 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.571400    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7668fa0-e004-4890-89a7-04d6efe40724-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7668fa0-e004-4890-89a7-04d6efe40724-openshift-service-ca\") pod \"a7668fa0-e004-4890-89a7-04d6efe40724\" (UID: \"a7668fa0-e004-4890-89a7-04d6efe40724\") "
Nov 01 05:38:00.571619 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.571552    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a7668fa0-e004-4890-89a7-04d6efe40724-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7668fa0-e004-4890-89a7-04d6efe40724-metrics-certs\") pod \"a7668fa0-e004-4890-89a7-04d6efe40724\" (UID: \"a7668fa0-e004-4890-89a7-04d6efe40724\") "
Nov 01 05:38:00.571694 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.571624    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7668fa0-e004-4890-89a7-04d6efe40724-kube-api-access-wzmk2\" (UniqueName: \"kubernetes.io/projected/a7668fa0-e004-4890-89a7-04d6efe40724-kube-api-access-wzmk2\") pod \"a7668fa0-e004-4890-89a7-04d6efe40724\" (UID: \"a7668fa0-e004-4890-89a7-04d6efe40724\") "
Nov 01 05:38:00.571694 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.571657    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a7668fa0-e004-4890-89a7-04d6efe40724-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7668fa0-e004-4890-89a7-04d6efe40724-maas-api-tls\") pod \"a7668fa0-e004-4890-89a7-04d6efe40724\" (UID: \"a7668fa0-e004-4890-89a7-04d6efe40724\") "
Nov 01 05:38:00.571885 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.571857    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7668fa0-e004-4890-89a7-04d6efe40724-openshift-service-ca" pod "a7668fa0-e004-4890-89a7-04d6efe40724" (UID: "a7668fa0-e004-4890-89a7-04d6efe40724"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:38:00.572149 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.572122    4864 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7668fa0-e004-4890-89a7-04d6efe40724-openshift-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:00.574030 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.574006    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7668fa0-e004-4890-89a7-04d6efe40724-kube-api-access-wzmk2" pod "a7668fa0-e004-4890-89a7-04d6efe40724" (UID: "a7668fa0-e004-4890-89a7-04d6efe40724"). InnerVolumeSpecName "kube-api-access-wzmk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:38:00.574423 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.574401    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7668fa0-e004-4890-89a7-04d6efe40724-metrics-certs" pod "a7668fa0-e004-4890-89a7-04d6efe40724" (UID: "a7668fa0-e004-4890-89a7-04d6efe40724"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:38:00.574551 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.574529    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7668fa0-e004-4890-89a7-04d6efe40724-maas-api-tls" pod "a7668fa0-e004-4890-89a7-04d6efe40724" (UID: "a7668fa0-e004-4890-89a7-04d6efe40724"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:38:00.673926 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.673826    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7668fa0-e004-4890-89a7-04d6efe40724-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:00.673926 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.673865    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzmk2\" (UniqueName: \"kubernetes.io/projected/a7668fa0-e004-4890-89a7-04d6efe40724-kube-api-access-wzmk2\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:00.673926 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.673882    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7668fa0-e004-4890-89a7-04d6efe40724-maas-api-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:00.945660 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.945581    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"]
Nov 01 05:38:00.949657 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:00.949630    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c6c6fdb9-7c466d8449-qmsss"]
Nov 01 05:38:01.248176 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:01.248100    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a7668fa0-e004-4890-89a7-04d6efe40724" path="/var/lib/kubelet/pods/a7668fa0-e004-4890-89a7-04d6efe40724/volumes"
Nov 01 05:38:01.665404 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:01.665364    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"]
Nov 01 05:38:01.665645 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:01.665620    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv" podUID="a25440ad-796b-4ef8-bf27-18ef58ca955a" containerName="istio-proxy" containerID="cri-o://8eed477bd81d2f951d6489cbadbb9012104989091f7e99fc4116964b831808ca" gracePeriod=30
Nov 01 05:38:01.900806 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:01.900749    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 05:38:01.901070 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:01.900853    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv" podUID="a25440ad-796b-4ef8-bf27-18ef58ca955a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 05:38:02.690643 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:02.690600    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"]
Nov 01 05:38:02.691225 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:02.690936    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm" podUID="fb7c1a34-f9cf-445b-b14a-058fc937a6cb" containerName="istio-proxy" containerID="cri-o://f2fde46837394499cdacc51fb4b7fe04c32877c814486e331690753aa5d8248a" gracePeriod=30
Nov 01 05:38:03.238052 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:03.238015    4864 patch_prober.go:29] interesting pod/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 05:38:03.238227 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:03.238068    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm" podUID="fb7c1a34-f9cf-445b-b14a-058fc937a6cb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 05:38:03.873934 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:03.873900    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:38:03.874511 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:03.874491    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:38:03.883915 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:03.883891    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:38:03.884709 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:03.884688    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:38:06.940376 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:06.940312    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:38:07.055486 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.055453    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a25440ad-796b-4ef8-bf27-18ef58ca955a-kube-api-access-bt4zw\" (UniqueName: \"kubernetes.io/projected/a25440ad-796b-4ef8-bf27-18ef58ca955a-kube-api-access-bt4zw\") pod \"a25440ad-796b-4ef8-bf27-18ef58ca955a\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") "
Nov 01 05:38:07.055699 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.055538    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-credential-socket\") pod \"a25440ad-796b-4ef8-bf27-18ef58ca955a\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") "
Nov 01 05:38:07.055699 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.055581    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-token\" (UniqueName: \"kubernetes.io/projected/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-token\") pod \"a25440ad-796b-4ef8-bf27-18ef58ca955a\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") "
Nov 01 05:38:07.055699 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.055629    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-podinfo\") pod \"a25440ad-796b-4ef8-bf27-18ef58ca955a\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") "
Nov 01 05:38:07.055699 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.055672    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-data\") pod \"a25440ad-796b-4ef8-bf27-18ef58ca955a\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") "
Nov 01 05:38:07.055944 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.055723    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a25440ad-796b-4ef8-bf27-18ef58ca955a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a25440ad-796b-4ef8-bf27-18ef58ca955a-istiod-ca-cert\") pod \"a25440ad-796b-4ef8-bf27-18ef58ca955a\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") "
Nov 01 05:38:07.055944 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.055757    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-workload-socket\") pod \"a25440ad-796b-4ef8-bf27-18ef58ca955a\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") "
Nov 01 05:38:07.055944 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.055820    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-envoy\") pod \"a25440ad-796b-4ef8-bf27-18ef58ca955a\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") "
Nov 01 05:38:07.055944 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.055925    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-workload-certs\") pod \"a25440ad-796b-4ef8-bf27-18ef58ca955a\" (UID: \"a25440ad-796b-4ef8-bf27-18ef58ca955a\") "
Nov 01 05:38:07.056135 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.055928    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-credential-socket" pod "a25440ad-796b-4ef8-bf27-18ef58ca955a" (UID: "a25440ad-796b-4ef8-bf27-18ef58ca955a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:38:07.056135 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.056020    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-data" pod "a25440ad-796b-4ef8-bf27-18ef58ca955a" (UID: "a25440ad-796b-4ef8-bf27-18ef58ca955a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:38:07.056234 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.056144    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-workload-socket" pod "a25440ad-796b-4ef8-bf27-18ef58ca955a" (UID: "a25440ad-796b-4ef8-bf27-18ef58ca955a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:38:07.056234 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.056166    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a25440ad-796b-4ef8-bf27-18ef58ca955a-istiod-ca-cert" pod "a25440ad-796b-4ef8-bf27-18ef58ca955a" (UID: "a25440ad-796b-4ef8-bf27-18ef58ca955a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:38:07.056502 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.056482    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:07.056590 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.056506    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:07.056590 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.056521    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a25440ad-796b-4ef8-bf27-18ef58ca955a-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:07.056590 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.056535    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:07.056590 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.056540    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-workload-certs" pod "a25440ad-796b-4ef8-bf27-18ef58ca955a" (UID: "a25440ad-796b-4ef8-bf27-18ef58ca955a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:38:07.058555 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.058521    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-token" pod "a25440ad-796b-4ef8-bf27-18ef58ca955a" (UID: "a25440ad-796b-4ef8-bf27-18ef58ca955a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:38:07.058697 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.058676    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-podinfo" pod "a25440ad-796b-4ef8-bf27-18ef58ca955a" (UID: "a25440ad-796b-4ef8-bf27-18ef58ca955a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 05:38:07.058761 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.058702    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a25440ad-796b-4ef8-bf27-18ef58ca955a-kube-api-access-bt4zw" pod "a25440ad-796b-4ef8-bf27-18ef58ca955a" (UID: "a25440ad-796b-4ef8-bf27-18ef58ca955a"). InnerVolumeSpecName "kube-api-access-bt4zw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:38:07.058853 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.058834    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-envoy" pod "a25440ad-796b-4ef8-bf27-18ef58ca955a" (UID: "a25440ad-796b-4ef8-bf27-18ef58ca955a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:38:07.158294 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.158254    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:07.158294 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.158294    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bt4zw\" (UniqueName: \"kubernetes.io/projected/a25440ad-796b-4ef8-bf27-18ef58ca955a-kube-api-access-bt4zw\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:07.158495 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.158312    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:07.158495 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.158326    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:07.158495 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.158340    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a25440ad-796b-4ef8-bf27-18ef58ca955a-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:07.580662 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.580622    4864 generic.go:356] "Generic (PLEG): container finished" podID="a25440ad-796b-4ef8-bf27-18ef58ca955a" containerID="8eed477bd81d2f951d6489cbadbb9012104989091f7e99fc4116964b831808ca" exitCode=0
Nov 01 05:38:07.580884 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.580677    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv" event={"ID":"a25440ad-796b-4ef8-bf27-18ef58ca955a","Type":"ContainerDied","Data":"8eed477bd81d2f951d6489cbadbb9012104989091f7e99fc4116964b831808ca"}
Nov 01 05:38:07.580884 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.580686    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"
Nov 01 05:38:07.580884 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.580730    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv" event={"ID":"a25440ad-796b-4ef8-bf27-18ef58ca955a","Type":"ContainerDied","Data":"a4027ce001f913ce8a1b1ad6287600d5cdf9ee6c309901b22321185f3bc4ea6e"}
Nov 01 05:38:07.580884 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.580759    4864 scope.go:122] "RemoveContainer" containerID="8eed477bd81d2f951d6489cbadbb9012104989091f7e99fc4116964b831808ca"
Nov 01 05:38:07.593464 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.593433    4864 scope.go:122] "RemoveContainer" containerID="8eed477bd81d2f951d6489cbadbb9012104989091f7e99fc4116964b831808ca"
Nov 01 05:38:07.593745 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:38:07.593722    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8eed477bd81d2f951d6489cbadbb9012104989091f7e99fc4116964b831808ca\": container with ID starting with 8eed477bd81d2f951d6489cbadbb9012104989091f7e99fc4116964b831808ca not found: ID does not exist" containerID="8eed477bd81d2f951d6489cbadbb9012104989091f7e99fc4116964b831808ca"
Nov 01 05:38:07.593844 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.593760    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eed477bd81d2f951d6489cbadbb9012104989091f7e99fc4116964b831808ca"} err="failed to get container status \"8eed477bd81d2f951d6489cbadbb9012104989091f7e99fc4116964b831808ca\": rpc error: code = NotFound desc = could not find container \"8eed477bd81d2f951d6489cbadbb9012104989091f7e99fc4116964b831808ca\": container with ID starting with 8eed477bd81d2f951d6489cbadbb9012104989091f7e99fc4116964b831808ca not found: ID does not exist"
Nov 01 05:38:07.749311 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.749282    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"]
Nov 01 05:38:07.798063 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.798027    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv"]
Nov 01 05:38:07.984117 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:07.984091    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:38:08.069440 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.069400    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-workload-socket\") pod \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") "
Nov 01 05:38:08.069625 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.069474    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-envoy\") pod \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") "
Nov 01 05:38:08.069625 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.069577    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-credential-socket\") pod \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") "
Nov 01 05:38:08.069729 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.069626    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istiod-ca-cert\") pod \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") "
Nov 01 05:38:08.069729 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.069687    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-workload-certs\") pod \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") "
Nov 01 05:38:08.069729 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.069719    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-workload-socket" pod "fb7c1a34-f9cf-445b-b14a-058fc937a6cb" (UID: "fb7c1a34-f9cf-445b-b14a-058fc937a6cb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:38:08.069883 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.069753    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-podinfo\") pod \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") "
Nov 01 05:38:08.069883 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.069779    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-token\" (UniqueName: \"kubernetes.io/projected/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-token\") pod \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") "
Nov 01 05:38:08.069883 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.069865    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-data\") pod \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") "
Nov 01 05:38:08.070044 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.069901    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-kube-api-access-fvdnv\" (UniqueName: \"kubernetes.io/projected/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-kube-api-access-fvdnv\") pod \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\" (UID: \"fb7c1a34-f9cf-445b-b14a-058fc937a6cb\") "
Nov 01 05:38:08.070044 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.069945    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-credential-socket" pod "fb7c1a34-f9cf-445b-b14a-058fc937a6cb" (UID: "fb7c1a34-f9cf-445b-b14a-058fc937a6cb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:38:08.070044 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.070034    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-workload-certs" pod "fb7c1a34-f9cf-445b-b14a-058fc937a6cb" (UID: "fb7c1a34-f9cf-445b-b14a-058fc937a6cb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:38:08.070198 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.070119    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istiod-ca-cert" pod "fb7c1a34-f9cf-445b-b14a-058fc937a6cb" (UID: "fb7c1a34-f9cf-445b-b14a-058fc937a6cb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:38:08.070317 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.070249    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-data" pod "fb7c1a34-f9cf-445b-b14a-058fc937a6cb" (UID: "fb7c1a34-f9cf-445b-b14a-058fc937a6cb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:38:08.070390 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.070345    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:08.070390 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.070358    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:08.070390 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.070372    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:08.070390 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.070385    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:08.070759 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.070398    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:08.072132 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.072107    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-envoy" pod "fb7c1a34-f9cf-445b-b14a-058fc937a6cb" (UID: "fb7c1a34-f9cf-445b-b14a-058fc937a6cb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:38:08.072132 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.072124    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-podinfo" pod "fb7c1a34-f9cf-445b-b14a-058fc937a6cb" (UID: "fb7c1a34-f9cf-445b-b14a-058fc937a6cb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 05:38:08.072449 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.072431    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-token" pod "fb7c1a34-f9cf-445b-b14a-058fc937a6cb" (UID: "fb7c1a34-f9cf-445b-b14a-058fc937a6cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:38:08.072538 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.072519    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-kube-api-access-fvdnv" pod "fb7c1a34-f9cf-445b-b14a-058fc937a6cb" (UID: "fb7c1a34-f9cf-445b-b14a-058fc937a6cb"). InnerVolumeSpecName "kube-api-access-fvdnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:38:08.171558 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.171473    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:08.171558 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.171501    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvdnv\" (UniqueName: \"kubernetes.io/projected/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-kube-api-access-fvdnv\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:08.171558 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.171511    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:08.171558 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.171522    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb7c1a34-f9cf-445b-b14a-058fc937a6cb-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:38:08.588920 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.588883    4864 generic.go:356] "Generic (PLEG): container finished" podID="fb7c1a34-f9cf-445b-b14a-058fc937a6cb" containerID="f2fde46837394499cdacc51fb4b7fe04c32877c814486e331690753aa5d8248a" exitCode=0
Nov 01 05:38:08.588920 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.588922    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm" event={"ID":"fb7c1a34-f9cf-445b-b14a-058fc937a6cb","Type":"ContainerDied","Data":"f2fde46837394499cdacc51fb4b7fe04c32877c814486e331690753aa5d8248a"}
Nov 01 05:38:08.589172 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.588937    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"
Nov 01 05:38:08.589172 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.588972    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm" event={"ID":"fb7c1a34-f9cf-445b-b14a-058fc937a6cb","Type":"ContainerDied","Data":"5ba6fc1776bfc3b72f22ad863cd900d2211fbf858fbfde617538f42500ce87e5"}
Nov 01 05:38:08.589172 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.588998    4864 scope.go:122] "RemoveContainer" containerID="f2fde46837394499cdacc51fb4b7fe04c32877c814486e331690753aa5d8248a"
Nov 01 05:38:08.603468 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.603445    4864 scope.go:122] "RemoveContainer" containerID="f2fde46837394499cdacc51fb4b7fe04c32877c814486e331690753aa5d8248a"
Nov 01 05:38:08.603848 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:38:08.603823    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2fde46837394499cdacc51fb4b7fe04c32877c814486e331690753aa5d8248a\": container with ID starting with f2fde46837394499cdacc51fb4b7fe04c32877c814486e331690753aa5d8248a not found: ID does not exist" containerID="f2fde46837394499cdacc51fb4b7fe04c32877c814486e331690753aa5d8248a"
Nov 01 05:38:08.603928 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.603863    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2fde46837394499cdacc51fb4b7fe04c32877c814486e331690753aa5d8248a"} err="failed to get container status \"f2fde46837394499cdacc51fb4b7fe04c32877c814486e331690753aa5d8248a\": rpc error: code = NotFound desc = could not find container \"f2fde46837394499cdacc51fb4b7fe04c32877c814486e331690753aa5d8248a\": container with ID starting with f2fde46837394499cdacc51fb4b7fe04c32877c814486e331690753aa5d8248a not found: ID does not exist"
Nov 01 05:38:08.759480 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.759442    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"]
Nov 01 05:38:08.808828 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:08.808772    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm"]
Nov 01 05:38:09.248585 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:09.248552    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a25440ad-796b-4ef8-bf27-18ef58ca955a" path="/var/lib/kubelet/pods/a25440ad-796b-4ef8-bf27-18ef58ca955a/volumes"
Nov 01 05:38:09.249022 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:09.249009    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fb7c1a34-f9cf-445b-b14a-058fc937a6cb" path="/var/lib/kubelet/pods/fb7c1a34-f9cf-445b-b14a-058fc937a6cb/volumes"
Nov 01 05:38:22.525702 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:38:22.525656    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:38:43.297576 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.297526    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"]
Nov 01 05:38:43.300456 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.298056    4864 cpu_manager.go:414] "containerMap: removing container" podUID="fb7c1a34-f9cf-445b-b14a-058fc937a6cb" containerName="istio-proxy"
Nov 01 05:38:43.300456 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.298069    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb7c1a34-f9cf-445b-b14a-058fc937a6cb" containerName="istio-proxy"
Nov 01 05:38:43.300456 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.298096    4864 cpu_manager.go:414] "containerMap: removing container" podUID="a25440ad-796b-4ef8-bf27-18ef58ca955a" containerName="istio-proxy"
Nov 01 05:38:43.300456 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.298101    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a25440ad-796b-4ef8-bf27-18ef58ca955a" containerName="istio-proxy"
Nov 01 05:38:43.300456 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.298109    4864 cpu_manager.go:414] "containerMap: removing container" podUID="4002ccda-eb43-49ff-90e6-5ce4e72a70c1" containerName="istio-proxy"
Nov 01 05:38:43.300456 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.298113    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4002ccda-eb43-49ff-90e6-5ce4e72a70c1" containerName="istio-proxy"
Nov 01 05:38:43.300456 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.298119    4864 cpu_manager.go:414] "containerMap: removing container" podUID="a7668fa0-e004-4890-89a7-04d6efe40724" containerName="maas-api"
Nov 01 05:38:43.300456 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.298124    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7668fa0-e004-4890-89a7-04d6efe40724" containerName="maas-api"
Nov 01 05:38:43.300456 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.298270    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="a25440ad-796b-4ef8-bf27-18ef58ca955a" containerName="istio-proxy"
Nov 01 05:38:43.300456 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.298286    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="4002ccda-eb43-49ff-90e6-5ce4e72a70c1" containerName="istio-proxy"
Nov 01 05:38:43.300456 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.298296    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7668fa0-e004-4890-89a7-04d6efe40724" containerName="maas-api"
Nov 01 05:38:43.300456 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.298320    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb7c1a34-f9cf-445b-b14a-058fc937a6cb" containerName="istio-proxy"
Nov 01 05:38:43.301565 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.301549    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.354588 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.354537    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5975aa80-aa20-4896-b06f-222a39b0d6d2-istiod-ca-cert\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.354815 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.354630    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-workload-certs\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.354815 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.354704    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-token\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.354815 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.354771    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-data\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.355114 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.354883    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-envoy\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.355114 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.354997    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a08eb536-openshift-default-dockercfg-lwx88\""
Nov 01 05:38:43.355274 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.355250    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-podinfo\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.355481 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.355462    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq5n7\" (UniqueName: \"kubernetes.io/projected/5975aa80-aa20-4896-b06f-222a39b0d6d2-kube-api-access-zq5n7\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.355689 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.355672    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-credential-socket\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.355868 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.355850    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-workload-socket\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.416882 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.415573    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"]
Nov 01 05:38:43.457731 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.457690    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-podinfo\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.457947 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.457743    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq5n7\" (UniqueName: \"kubernetes.io/projected/5975aa80-aa20-4896-b06f-222a39b0d6d2-kube-api-access-zq5n7\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.457947 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.457880    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-credential-socket\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.457947 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.457929    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-workload-socket\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.458179 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.457964    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5975aa80-aa20-4896-b06f-222a39b0d6d2-istiod-ca-cert\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.458179 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.458018    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-workload-certs\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.458179 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.458063    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-token\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.458179 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.458127    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-data\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.458499 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.458182    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-envoy\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.458996 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.458970    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-workload-socket\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.459455 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.459436    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-workload-certs\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.459694 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.459639    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-credential-socket\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.459945 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.459921    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-data\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.462643 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.461051    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5975aa80-aa20-4896-b06f-222a39b0d6d2-istiod-ca-cert\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.495682 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.489775    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-envoy\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.500376 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.494178    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-podinfo\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.520906 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.520872    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-token\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.521096 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.521023    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq5n7\" (UniqueName: \"kubernetes.io/projected/5975aa80-aa20-4896-b06f-222a39b0d6d2-kube-api-access-zq5n7\") pod \"e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") " pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:43.617599 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:43.617559    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:44.053475 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:44.053440    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"]
Nov 01 05:38:44.054213 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:38:44.054170    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5975aa80_aa20_4896_b06f_222a39b0d6d2.slice/crio-7eaafef72520604c6976c3b10142b487334e7330f56720dcd46172dd1c0cd50e WatchSource:0}: Error finding container 7eaafef72520604c6976c3b10142b487334e7330f56720dcd46172dd1c0cd50e: Status 404 returned error can't find the container with id 7eaafef72520604c6976c3b10142b487334e7330f56720dcd46172dd1c0cd50e
Nov 01 05:38:44.057837 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:44.057774    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:38:44.057967 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:44.057895    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:38:44.057967 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:44.057928    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:38:44.825539 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:44.825504    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g" event={"ID":"5975aa80-aa20-4896-b06f-222a39b0d6d2","Type":"ContainerStarted","Data":"0e94704ca54db39389dae8c6b9af813af1294161fe5872f22cf0af9dde1c4f46"}
Nov 01 05:38:44.826010 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:44.825554    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g" event={"ID":"5975aa80-aa20-4896-b06f-222a39b0d6d2","Type":"ContainerStarted","Data":"7eaafef72520604c6976c3b10142b487334e7330f56720dcd46172dd1c0cd50e"}
Nov 01 05:38:44.946320 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:44.946258    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g" podStartSLOduration=1.9462432509999998 podStartE2EDuration="1.946243251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:38:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:38:44.944374168 +0000 UTC m=+2141.834750145" watchObservedRunningTime="2026-11-01 05:38:44.946243251 +0000 UTC m=+2141.836619224"
Nov 01 05:38:45.617816 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:45.617767    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:45.623262 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:45.623237    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:45.833634 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:45.833594    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:45.834681 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:45.834662    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:38:52.527280 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:38:52.527243    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:38:55.923067 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:55.923032    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"]
Nov 01 05:38:55.929273 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:55.929237    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"
Nov 01 05:38:55.985192 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:55.985158    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-4cbcb62b-serving-cert\""
Nov 01 05:38:56.003812 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:56.003760    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d26bc32-acde-4f20-916e-effc027e3eb9-metrics-certs\") pod \"maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg\" (UID: \"3d26bc32-acde-4f20-916e-effc027e3eb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"
Nov 01 05:38:56.004015 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:56.003919    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d26bc32-acde-4f20-916e-effc027e3eb9-maas-api-tls\") pod \"maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg\" (UID: \"3d26bc32-acde-4f20-916e-effc027e3eb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"
Nov 01 05:38:56.004015 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:56.003972    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnlch\" (UniqueName: \"kubernetes.io/projected/3d26bc32-acde-4f20-916e-effc027e3eb9-kube-api-access-tnlch\") pod \"maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg\" (UID: \"3d26bc32-acde-4f20-916e-effc027e3eb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"
Nov 01 05:38:56.004137 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:56.004100    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d26bc32-acde-4f20-916e-effc027e3eb9-openshift-service-ca\") pod \"maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg\" (UID: \"3d26bc32-acde-4f20-916e-effc027e3eb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"
Nov 01 05:38:56.037598 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:56.037565    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"]
Nov 01 05:38:56.105303 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:56.105257    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d26bc32-acde-4f20-916e-effc027e3eb9-metrics-certs\") pod \"maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg\" (UID: \"3d26bc32-acde-4f20-916e-effc027e3eb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"
Nov 01 05:38:56.105460 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:56.105351    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d26bc32-acde-4f20-916e-effc027e3eb9-maas-api-tls\") pod \"maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg\" (UID: \"3d26bc32-acde-4f20-916e-effc027e3eb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"
Nov 01 05:38:56.105460 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:56.105390    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnlch\" (UniqueName: \"kubernetes.io/projected/3d26bc32-acde-4f20-916e-effc027e3eb9-kube-api-access-tnlch\") pod \"maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg\" (UID: \"3d26bc32-acde-4f20-916e-effc027e3eb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"
Nov 01 05:38:56.105560 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:38:56.105494    4864 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-serving-cert: secret "maas-api-e2e-ait-4cbcb62b-serving-cert" not found
Nov 01 05:38:56.105619 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:38:56.105562    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d26bc32-acde-4f20-916e-effc027e3eb9-maas-api-tls podName:3d26bc32-acde-4f20-916e-effc027e3eb9 nodeName:}" failed. No retries permitted until 2026-11-01 05:38:56.605545822 +0000 UTC m=+2153.495921812 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3d26bc32-acde-4f20-916e-effc027e3eb9-maas-api-tls") pod "maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg" (UID: "3d26bc32-acde-4f20-916e-effc027e3eb9") : secret "maas-api-e2e-ait-4cbcb62b-serving-cert" not found
Nov 01 05:38:56.105619 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:56.105505    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d26bc32-acde-4f20-916e-effc027e3eb9-openshift-service-ca\") pod \"maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg\" (UID: \"3d26bc32-acde-4f20-916e-effc027e3eb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"
Nov 01 05:38:56.106204 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:56.106183    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d26bc32-acde-4f20-916e-effc027e3eb9-openshift-service-ca\") pod \"maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg\" (UID: \"3d26bc32-acde-4f20-916e-effc027e3eb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"
Nov 01 05:38:56.107964 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:56.107947    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d26bc32-acde-4f20-916e-effc027e3eb9-metrics-certs\") pod \"maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg\" (UID: \"3d26bc32-acde-4f20-916e-effc027e3eb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"
Nov 01 05:38:56.168531 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:56.168494    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnlch\" (UniqueName: \"kubernetes.io/projected/3d26bc32-acde-4f20-916e-effc027e3eb9-kube-api-access-tnlch\") pod \"maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg\" (UID: \"3d26bc32-acde-4f20-916e-effc027e3eb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"
Nov 01 05:38:56.611332 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:56.611298    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d26bc32-acde-4f20-916e-effc027e3eb9-maas-api-tls\") pod \"maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg\" (UID: \"3d26bc32-acde-4f20-916e-effc027e3eb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"
Nov 01 05:38:56.614016 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:56.613989    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d26bc32-acde-4f20-916e-effc027e3eb9-maas-api-tls\") pod \"maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg\" (UID: \"3d26bc32-acde-4f20-916e-effc027e3eb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"
Nov 01 05:38:56.845831 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:56.845776    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"
Nov 01 05:38:57.227362 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:57.227337    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"]
Nov 01 05:38:57.230031 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:38:57.230000    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d26bc32_acde_4f20_916e_effc027e3eb9.slice/crio-3a7d2ea98fccf66a298d48948e3ecae681994c5fd335040b9eaa17891cc7fd50 WatchSource:0}: Error finding container 3a7d2ea98fccf66a298d48948e3ecae681994c5fd335040b9eaa17891cc7fd50: Status 404 returned error can't find the container with id 3a7d2ea98fccf66a298d48948e3ecae681994c5fd335040b9eaa17891cc7fd50
Nov 01 05:38:57.924570 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:57.924531    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg" event={"ID":"3d26bc32-acde-4f20-916e-effc027e3eb9","Type":"ContainerStarted","Data":"3a7d2ea98fccf66a298d48948e3ecae681994c5fd335040b9eaa17891cc7fd50"}
Nov 01 05:38:58.932295 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:58.932258    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg" event={"ID":"3d26bc32-acde-4f20-916e-effc027e3eb9","Type":"ContainerStarted","Data":"7d4b3badb24e760a6d4573b16c081395a2f96d00044316d609714a0bf76384cd"}
Nov 01 05:38:58.932715 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:58.932434    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"
Nov 01 05:38:59.051353 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:38:59.051297    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg" podStartSLOduration=3.134485728 podStartE2EDuration="4.051281836s" totalImagesPullingTime="916.796108ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:38:59.049052969 +0000 UTC m=+2155.939428945" watchObservedRunningTime="2026-11-01 05:38:59.051281836 +0000 UTC m=+2155.941657811"
Nov 01 05:39:04.944368 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:04.944339    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"
Nov 01 05:39:22.524219 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:39:22.524178    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:39:28.491458 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:28.491415    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7440-fed722"]
Nov 01 05:39:28.495483 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:28.495457    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7440-fed722"
Nov 01 05:39:28.551861 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:28.551833    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7440-fed722"]
Nov 01 05:39:28.564127 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:28.564094    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btldg\" (UniqueName: \"kubernetes.io/projected/ca0a27da-a9ee-4c46-8ac1-d25d474cad3f-kube-api-access-btldg\") pod \"test-curl-7440-fed722\" (UID: \"ca0a27da-a9ee-4c46-8ac1-d25d474cad3f\") " pod="opendatahub/test-curl-7440-fed722"
Nov 01 05:39:28.597529 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:28.597492    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-wfzxk\""
Nov 01 05:39:28.665247 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:28.665213    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btldg\" (UniqueName: \"kubernetes.io/projected/ca0a27da-a9ee-4c46-8ac1-d25d474cad3f-kube-api-access-btldg\") pod \"test-curl-7440-fed722\" (UID: \"ca0a27da-a9ee-4c46-8ac1-d25d474cad3f\") " pod="opendatahub/test-curl-7440-fed722"
Nov 01 05:39:28.726349 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:28.726319    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btldg\" (UniqueName: \"kubernetes.io/projected/ca0a27da-a9ee-4c46-8ac1-d25d474cad3f-kube-api-access-btldg\") pod \"test-curl-7440-fed722\" (UID: \"ca0a27da-a9ee-4c46-8ac1-d25d474cad3f\") " pod="opendatahub/test-curl-7440-fed722"
Nov 01 05:39:28.808342 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:28.808308    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7440-fed722"
Nov 01 05:39:29.186278 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:29.186251    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7440-fed722"]
Nov 01 05:39:29.187743 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:39:29.187713    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca0a27da_a9ee_4c46_8ac1_d25d474cad3f.slice/crio-ebae7709fd013ba002dc4d8219e4fb6c1a67990e3122275409930bebb9a6dc54 WatchSource:0}: Error finding container ebae7709fd013ba002dc4d8219e4fb6c1a67990e3122275409930bebb9a6dc54: Status 404 returned error can't find the container with id ebae7709fd013ba002dc4d8219e4fb6c1a67990e3122275409930bebb9a6dc54
Nov 01 05:39:30.158824 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:30.158741    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7440-fed722" event={"ID":"ca0a27da-a9ee-4c46-8ac1-d25d474cad3f","Type":"ContainerStarted","Data":"f2aa337f8cde313c2b58c272fe3a7afe2ce149eb07885bba9b59495ae43e3168"}
Nov 01 05:39:30.159352 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:30.158835    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7440-fed722" event={"ID":"ca0a27da-a9ee-4c46-8ac1-d25d474cad3f","Type":"ContainerStarted","Data":"ebae7709fd013ba002dc4d8219e4fb6c1a67990e3122275409930bebb9a6dc54"}
Nov 01 05:39:30.281457 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:30.281388    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7440-fed722" podStartSLOduration=2.281368481 podStartE2EDuration="2.281368481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:39:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:39:30.278962734 +0000 UTC m=+2187.169338708" watchObservedRunningTime="2026-11-01 05:39:30.281368481 +0000 UTC m=+2187.171744457"
Nov 01 05:39:31.807203 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:31.807168    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7440-fed722"]
Nov 01 05:39:31.818056 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:31.818022    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7440-fed722"]
Nov 01 05:39:32.175757 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:32.175634    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7440-fed722" podUID="ca0a27da-a9ee-4c46-8ac1-d25d474cad3f" containerName="test-curl-7440-fed722" containerID="cri-o://f2aa337f8cde313c2b58c272fe3a7afe2ce149eb07885bba9b59495ae43e3168" gracePeriod=2
Nov 01 05:39:32.226470 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:39:32.226434    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7440-fed722\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="ca0a27da-a9ee-4c46-8ac1-d25d474cad3f" pod="opendatahub/test-curl-7440-fed722"
Nov 01 05:39:33.294539 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:39:33.294495    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7440-fed722\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="ca0a27da-a9ee-4c46-8ac1-d25d474cad3f" pod="opendatahub/test-curl-7440-fed722"
Nov 01 05:39:34.434544 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:34.434482    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7440-fed722"
Nov 01 05:39:34.536342 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:34.536306    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca0a27da-a9ee-4c46-8ac1-d25d474cad3f-kube-api-access-btldg\" (UniqueName: \"kubernetes.io/projected/ca0a27da-a9ee-4c46-8ac1-d25d474cad3f-kube-api-access-btldg\") pod \"ca0a27da-a9ee-4c46-8ac1-d25d474cad3f\" (UID: \"ca0a27da-a9ee-4c46-8ac1-d25d474cad3f\") "
Nov 01 05:39:34.538544 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:34.538520    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca0a27da-a9ee-4c46-8ac1-d25d474cad3f-kube-api-access-btldg" pod "ca0a27da-a9ee-4c46-8ac1-d25d474cad3f" (UID: "ca0a27da-a9ee-4c46-8ac1-d25d474cad3f"). InnerVolumeSpecName "kube-api-access-btldg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:39:34.637687 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:34.637652    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btldg\" (UniqueName: \"kubernetes.io/projected/ca0a27da-a9ee-4c46-8ac1-d25d474cad3f-kube-api-access-btldg\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:39:35.199631 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:35.199580    4864 generic.go:356] "Generic (PLEG): container finished" podID="ca0a27da-a9ee-4c46-8ac1-d25d474cad3f" containerID="f2aa337f8cde313c2b58c272fe3a7afe2ce149eb07885bba9b59495ae43e3168" exitCode=137
Nov 01 05:39:35.199876 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:35.199706    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7440-fed722"
Nov 01 05:39:35.199876 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:35.199752    4864 scope.go:122] "RemoveContainer" containerID="f2aa337f8cde313c2b58c272fe3a7afe2ce149eb07885bba9b59495ae43e3168"
Nov 01 05:39:35.209267 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:35.209250    4864 scope.go:122] "RemoveContainer" containerID="f2aa337f8cde313c2b58c272fe3a7afe2ce149eb07885bba9b59495ae43e3168"
Nov 01 05:39:35.209569 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:39:35.209550    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2aa337f8cde313c2b58c272fe3a7afe2ce149eb07885bba9b59495ae43e3168\": container with ID starting with f2aa337f8cde313c2b58c272fe3a7afe2ce149eb07885bba9b59495ae43e3168 not found: ID does not exist" containerID="f2aa337f8cde313c2b58c272fe3a7afe2ce149eb07885bba9b59495ae43e3168"
Nov 01 05:39:35.209644 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:35.209581    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2aa337f8cde313c2b58c272fe3a7afe2ce149eb07885bba9b59495ae43e3168"} err="failed to get container status \"f2aa337f8cde313c2b58c272fe3a7afe2ce149eb07885bba9b59495ae43e3168\": rpc error: code = NotFound desc = could not find container \"f2aa337f8cde313c2b58c272fe3a7afe2ce149eb07885bba9b59495ae43e3168\": container with ID starting with f2aa337f8cde313c2b58c272fe3a7afe2ce149eb07885bba9b59495ae43e3168 not found: ID does not exist"
Nov 01 05:39:35.248015 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:39:35.247976    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca0a27da-a9ee-4c46-8ac1-d25d474cad3f" path="/var/lib/kubelet/pods/ca0a27da-a9ee-4c46-8ac1-d25d474cad3f/volumes"
Nov 01 05:39:35.262664 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:39:35.262616    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7440-fed722\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="ca0a27da-a9ee-4c46-8ac1-d25d474cad3f" pod="opendatahub/test-curl-7440-fed722"
Nov 01 05:39:52.525383 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:39:52.525305    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:40:22.524680 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:40:22.524642    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:40:28.558380 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:28.558341    4864 generic.go:356] "Generic (PLEG): container finished" podID="5975aa80-aa20-4896-b06f-222a39b0d6d2" containerID="0e94704ca54db39389dae8c6b9af813af1294161fe5872f22cf0af9dde1c4f46" exitCode=0
Nov 01 05:40:28.558847 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:28.558409    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g" event={"ID":"5975aa80-aa20-4896-b06f-222a39b0d6d2","Type":"ContainerDied","Data":"0e94704ca54db39389dae8c6b9af813af1294161fe5872f22cf0af9dde1c4f46"}
Nov 01 05:40:28.558847 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:28.558820    4864 scope.go:122] "RemoveContainer" containerID="0e94704ca54db39389dae8c6b9af813af1294161fe5872f22cf0af9dde1c4f46"
Nov 01 05:40:28.560309 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:28.560267    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:40:28.560442 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:28.560357    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:40:28.560521 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:28.560438    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:40:29.587959 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:29.587919    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g" event={"ID":"5975aa80-aa20-4896-b06f-222a39b0d6d2","Type":"ContainerStarted","Data":"eaa611d49034de9eb92d3113d615d2771012bff59b56daadbfeaf530f8414fe9"}
Nov 01 05:40:29.618815 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:29.618771    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:40:29.619814 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:29.619776    4864 patch_prober.go:29] interesting pod/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 01 05:40:29.619928 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:29.619838    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g" podUID="5975aa80-aa20-4896-b06f-222a39b0d6d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 01 05:40:30.618020 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:30.617991    4864 patch_prober.go:29] interesting pod/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 01 05:40:30.618409 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:30.618037    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g" podUID="5975aa80-aa20-4896-b06f-222a39b0d6d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 01 05:40:30.833535 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:30.833495    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:40:31.617975 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:31.617946    4864 patch_prober.go:29] interesting pod/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 01 05:40:31.618149 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:31.617996    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g" podUID="5975aa80-aa20-4896-b06f-222a39b0d6d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 01 05:40:32.619484 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:32.619454    4864 patch_prober.go:29] interesting pod/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 01 05:40:32.619990 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:32.619966    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g" podUID="5975aa80-aa20-4896-b06f-222a39b0d6d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 01 05:40:33.618262 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:33.618229    4864 patch_prober.go:29] interesting pod/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 01 05:40:33.618442 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:33.618282    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g" podUID="5975aa80-aa20-4896-b06f-222a39b0d6d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 01 05:40:34.618806 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:34.618747    4864 patch_prober.go:29] interesting pod/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 01 05:40:34.619204 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:34.618812    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g" podUID="5975aa80-aa20-4896-b06f-222a39b0d6d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 01 05:40:35.623992 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:35.623959    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:40:35.625233 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:35.625212    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:40:52.526030 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:40:52.525988    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:40:53.668604 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:53.668573    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"]
Nov 01 05:40:53.669115 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:53.668761    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg" podUID="3d26bc32-acde-4f20-916e-effc027e3eb9" containerName="maas-api" containerID="cri-o://7d4b3badb24e760a6d4573b16c081395a2f96d00044316d609714a0bf76384cd" gracePeriod=30
Nov 01 05:40:53.990441 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:53.990420    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"
Nov 01 05:40:54.077410 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:54.077381    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3d26bc32-acde-4f20-916e-effc027e3eb9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d26bc32-acde-4f20-916e-effc027e3eb9-metrics-certs\") pod \"3d26bc32-acde-4f20-916e-effc027e3eb9\" (UID: \"3d26bc32-acde-4f20-916e-effc027e3eb9\") "
Nov 01 05:40:54.077590 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:54.077469    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3d26bc32-acde-4f20-916e-effc027e3eb9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d26bc32-acde-4f20-916e-effc027e3eb9-maas-api-tls\") pod \"3d26bc32-acde-4f20-916e-effc027e3eb9\" (UID: \"3d26bc32-acde-4f20-916e-effc027e3eb9\") "
Nov 01 05:40:54.077590 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:54.077548    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3d26bc32-acde-4f20-916e-effc027e3eb9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d26bc32-acde-4f20-916e-effc027e3eb9-openshift-service-ca\") pod \"3d26bc32-acde-4f20-916e-effc027e3eb9\" (UID: \"3d26bc32-acde-4f20-916e-effc027e3eb9\") "
Nov 01 05:40:54.077729 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:54.077651    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d26bc32-acde-4f20-916e-effc027e3eb9-kube-api-access-tnlch\" (UniqueName: \"kubernetes.io/projected/3d26bc32-acde-4f20-916e-effc027e3eb9-kube-api-access-tnlch\") pod \"3d26bc32-acde-4f20-916e-effc027e3eb9\" (UID: \"3d26bc32-acde-4f20-916e-effc027e3eb9\") "
Nov 01 05:40:54.077930 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:54.077900    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d26bc32-acde-4f20-916e-effc027e3eb9-openshift-service-ca" pod "3d26bc32-acde-4f20-916e-effc027e3eb9" (UID: "3d26bc32-acde-4f20-916e-effc027e3eb9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:40:54.078141 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:54.078112    4864 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d26bc32-acde-4f20-916e-effc027e3eb9-openshift-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:40:54.080146 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:54.080116    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d26bc32-acde-4f20-916e-effc027e3eb9-kube-api-access-tnlch" pod "3d26bc32-acde-4f20-916e-effc027e3eb9" (UID: "3d26bc32-acde-4f20-916e-effc027e3eb9"). InnerVolumeSpecName "kube-api-access-tnlch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:40:54.080371 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:54.080345    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d26bc32-acde-4f20-916e-effc027e3eb9-maas-api-tls" pod "3d26bc32-acde-4f20-916e-effc027e3eb9" (UID: "3d26bc32-acde-4f20-916e-effc027e3eb9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:40:54.080371 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:54.080364    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d26bc32-acde-4f20-916e-effc027e3eb9-metrics-certs" pod "3d26bc32-acde-4f20-916e-effc027e3eb9" (UID: "3d26bc32-acde-4f20-916e-effc027e3eb9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:40:54.179769 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:54.179718    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnlch\" (UniqueName: \"kubernetes.io/projected/3d26bc32-acde-4f20-916e-effc027e3eb9-kube-api-access-tnlch\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:40:54.179769 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:54.179768    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d26bc32-acde-4f20-916e-effc027e3eb9-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:40:54.179769 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:54.179806    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d26bc32-acde-4f20-916e-effc027e3eb9-maas-api-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:40:54.783011 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:54.782977    4864 generic.go:356] "Generic (PLEG): container finished" podID="3d26bc32-acde-4f20-916e-effc027e3eb9" containerID="7d4b3badb24e760a6d4573b16c081395a2f96d00044316d609714a0bf76384cd" exitCode=0
Nov 01 05:40:54.783519 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:54.783025    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"
Nov 01 05:40:54.783519 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:54.783044    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg" event={"ID":"3d26bc32-acde-4f20-916e-effc027e3eb9","Type":"ContainerDied","Data":"7d4b3badb24e760a6d4573b16c081395a2f96d00044316d609714a0bf76384cd"}
Nov 01 05:40:54.783519 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:54.783107    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg" event={"ID":"3d26bc32-acde-4f20-916e-effc027e3eb9","Type":"ContainerDied","Data":"3a7d2ea98fccf66a298d48948e3ecae681994c5fd335040b9eaa17891cc7fd50"}
Nov 01 05:40:54.783519 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:54.783139    4864 scope.go:122] "RemoveContainer" containerID="7d4b3badb24e760a6d4573b16c081395a2f96d00044316d609714a0bf76384cd"
Nov 01 05:40:54.793985 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:54.793964    4864 scope.go:122] "RemoveContainer" containerID="7d4b3badb24e760a6d4573b16c081395a2f96d00044316d609714a0bf76384cd"
Nov 01 05:40:54.794245 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:40:54.794223    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d4b3badb24e760a6d4573b16c081395a2f96d00044316d609714a0bf76384cd\": container with ID starting with 7d4b3badb24e760a6d4573b16c081395a2f96d00044316d609714a0bf76384cd not found: ID does not exist" containerID="7d4b3badb24e760a6d4573b16c081395a2f96d00044316d609714a0bf76384cd"
Nov 01 05:40:54.794308 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:54.794258    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d4b3badb24e760a6d4573b16c081395a2f96d00044316d609714a0bf76384cd"} err="failed to get container status \"7d4b3badb24e760a6d4573b16c081395a2f96d00044316d609714a0bf76384cd\": rpc error: code = NotFound desc = could not find container \"7d4b3badb24e760a6d4573b16c081395a2f96d00044316d609714a0bf76384cd\": container with ID starting with 7d4b3badb24e760a6d4573b16c081395a2f96d00044316d609714a0bf76384cd not found: ID does not exist"
Nov 01 05:40:54.947231 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:54.947197    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"]
Nov 01 05:40:54.998453 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:54.998421    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4cbcb62b-76fddcdc55-cttqg"]
Nov 01 05:40:55.249842 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:40:55.249804    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3d26bc32-acde-4f20-916e-effc027e3eb9" path="/var/lib/kubelet/pods/3d26bc32-acde-4f20-916e-effc027e3eb9/volumes"
Nov 01 05:41:19.771342 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:19.771264    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt"]
Nov 01 05:41:19.771762 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:19.771450    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt" podUID="0df126d4-02ca-453d-92eb-2ce949558897" containerName="sim" containerID="cri-o://6d32dc50ea36d4c1845c97acb57f34402336ff7c31a5cc9bf7ae88121bbcea0e" gracePeriod=30
Nov 01 05:41:19.972982 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:19.972950    4864 generic.go:356] "Generic (PLEG): container finished" podID="0df126d4-02ca-453d-92eb-2ce949558897" containerID="6d32dc50ea36d4c1845c97acb57f34402336ff7c31a5cc9bf7ae88121bbcea0e" exitCode=0
Nov 01 05:41:19.973161 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:19.973116    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt" event={"ID":"0df126d4-02ca-453d-92eb-2ce949558897","Type":"ContainerDied","Data":"6d32dc50ea36d4c1845c97acb57f34402336ff7c31a5cc9bf7ae88121bbcea0e"}
Nov 01 05:41:20.071254 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:20.071227    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt"
Nov 01 05:41:20.191026 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:20.190994    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0df126d4-02ca-453d-92eb-2ce949558897-tls\" (UniqueName: \"kubernetes.io/secret/0df126d4-02ca-453d-92eb-2ce949558897-tls\") pod \"0df126d4-02ca-453d-92eb-2ce949558897\" (UID: \"0df126d4-02ca-453d-92eb-2ce949558897\") "
Nov 01 05:41:20.191215 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:20.191139    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0df126d4-02ca-453d-92eb-2ce949558897-kube-api-access-6qw76\" (UniqueName: \"kubernetes.io/projected/0df126d4-02ca-453d-92eb-2ce949558897-kube-api-access-6qw76\") pod \"0df126d4-02ca-453d-92eb-2ce949558897\" (UID: \"0df126d4-02ca-453d-92eb-2ce949558897\") "
Nov 01 05:41:20.193825 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:20.193775    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0df126d4-02ca-453d-92eb-2ce949558897-tls" pod "0df126d4-02ca-453d-92eb-2ce949558897" (UID: "0df126d4-02ca-453d-92eb-2ce949558897"). InnerVolumeSpecName "tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:41:20.193975 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:20.193848    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0df126d4-02ca-453d-92eb-2ce949558897-kube-api-access-6qw76" pod "0df126d4-02ca-453d-92eb-2ce949558897" (UID: "0df126d4-02ca-453d-92eb-2ce949558897"). InnerVolumeSpecName "kube-api-access-6qw76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:41:20.292860 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:20.292831    4864 reconciler_common.go:299] "Volume detached for volume \"tls\" (UniqueName: \"kubernetes.io/secret/0df126d4-02ca-453d-92eb-2ce949558897-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:41:20.292860 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:20.292856    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qw76\" (UniqueName: \"kubernetes.io/projected/0df126d4-02ca-453d-92eb-2ce949558897-kube-api-access-6qw76\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:41:20.981094 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:20.981061    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt"
Nov 01 05:41:20.981094 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:20.981081    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt" event={"ID":"0df126d4-02ca-453d-92eb-2ce949558897","Type":"ContainerDied","Data":"6766e0d5d2df552879ea52a81385473f3a3d5c844cabbb22bb267b94b19079b0"}
Nov 01 05:41:20.981647 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:20.981140    4864 scope.go:122] "RemoveContainer" containerID="6d32dc50ea36d4c1845c97acb57f34402336ff7c31a5cc9bf7ae88121bbcea0e"
Nov 01 05:41:21.146922 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:21.146884    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt"]
Nov 01 05:41:21.150356 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:21.150329    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-extmod-shr-user-b73da0/e2e-usage-sim-59820a-6cb498f45f-xklnt"]
Nov 01 05:41:21.247611 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:21.247530    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0df126d4-02ca-453d-92eb-2ce949558897" path="/var/lib/kubelet/pods/0df126d4-02ca-453d-92eb-2ce949558897/volumes"
Nov 01 05:41:22.524119 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:41:22.524085    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:41:35.023035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.023001    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"]
Nov 01 05:41:35.023747 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.023723    4864 cpu_manager.go:414] "containerMap: removing container" podUID="ca0a27da-a9ee-4c46-8ac1-d25d474cad3f" containerName="test-curl-7440-fed722"
Nov 01 05:41:35.023747 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.023745    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca0a27da-a9ee-4c46-8ac1-d25d474cad3f" containerName="test-curl-7440-fed722"
Nov 01 05:41:35.024000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.023773    4864 cpu_manager.go:414] "containerMap: removing container" podUID="0df126d4-02ca-453d-92eb-2ce949558897" containerName="sim"
Nov 01 05:41:35.024000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.023797    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0df126d4-02ca-453d-92eb-2ce949558897" containerName="sim"
Nov 01 05:41:35.024000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.023821    4864 cpu_manager.go:414] "containerMap: removing container" podUID="3d26bc32-acde-4f20-916e-effc027e3eb9" containerName="maas-api"
Nov 01 05:41:35.024000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.023828    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d26bc32-acde-4f20-916e-effc027e3eb9" containerName="maas-api"
Nov 01 05:41:35.024000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.023976    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca0a27da-a9ee-4c46-8ac1-d25d474cad3f" containerName="test-curl-7440-fed722"
Nov 01 05:41:35.024000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.023992    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="0df126d4-02ca-453d-92eb-2ce949558897" containerName="sim"
Nov 01 05:41:35.024000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.024003    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d26bc32-acde-4f20-916e-effc027e3eb9" containerName="maas-api"
Nov 01 05:41:35.027424 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.027404    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"
Nov 01 05:41:35.082349 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.082322    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-faa8ff-serving-cert\""
Nov 01 05:41:35.132013 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.131990    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"]
Nov 01 05:41:35.160062 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.160035    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b33a7a5-8d14-4281-8639-32a1ab752cf8-openshift-service-ca\") pod \"maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w\" (UID: \"1b33a7a5-8d14-4281-8639-32a1ab752cf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"
Nov 01 05:41:35.160200 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.160168    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvn5g\" (UniqueName: \"kubernetes.io/projected/1b33a7a5-8d14-4281-8639-32a1ab752cf8-kube-api-access-wvn5g\") pod \"maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w\" (UID: \"1b33a7a5-8d14-4281-8639-32a1ab752cf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"
Nov 01 05:41:35.160249 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.160219    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b33a7a5-8d14-4281-8639-32a1ab752cf8-maas-api-tls\") pod \"maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w\" (UID: \"1b33a7a5-8d14-4281-8639-32a1ab752cf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"
Nov 01 05:41:35.160302 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.160279    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b33a7a5-8d14-4281-8639-32a1ab752cf8-metrics-certs\") pod \"maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w\" (UID: \"1b33a7a5-8d14-4281-8639-32a1ab752cf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"
Nov 01 05:41:35.261306 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.261274    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b33a7a5-8d14-4281-8639-32a1ab752cf8-openshift-service-ca\") pod \"maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w\" (UID: \"1b33a7a5-8d14-4281-8639-32a1ab752cf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"
Nov 01 05:41:35.261463 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.261352    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvn5g\" (UniqueName: \"kubernetes.io/projected/1b33a7a5-8d14-4281-8639-32a1ab752cf8-kube-api-access-wvn5g\") pod \"maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w\" (UID: \"1b33a7a5-8d14-4281-8639-32a1ab752cf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"
Nov 01 05:41:35.261463 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.261382    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b33a7a5-8d14-4281-8639-32a1ab752cf8-maas-api-tls\") pod \"maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w\" (UID: \"1b33a7a5-8d14-4281-8639-32a1ab752cf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"
Nov 01 05:41:35.261463 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.261434    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b33a7a5-8d14-4281-8639-32a1ab752cf8-metrics-certs\") pod \"maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w\" (UID: \"1b33a7a5-8d14-4281-8639-32a1ab752cf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"
Nov 01 05:41:35.262082 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.262056    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b33a7a5-8d14-4281-8639-32a1ab752cf8-openshift-service-ca\") pod \"maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w\" (UID: \"1b33a7a5-8d14-4281-8639-32a1ab752cf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"
Nov 01 05:41:35.264157 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.264136    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b33a7a5-8d14-4281-8639-32a1ab752cf8-maas-api-tls\") pod \"maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w\" (UID: \"1b33a7a5-8d14-4281-8639-32a1ab752cf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"
Nov 01 05:41:35.264314 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.264291    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b33a7a5-8d14-4281-8639-32a1ab752cf8-metrics-certs\") pod \"maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w\" (UID: \"1b33a7a5-8d14-4281-8639-32a1ab752cf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"
Nov 01 05:41:35.322148 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.322127    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvn5g\" (UniqueName: \"kubernetes.io/projected/1b33a7a5-8d14-4281-8639-32a1ab752cf8-kube-api-access-wvn5g\") pod \"maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w\" (UID: \"1b33a7a5-8d14-4281-8639-32a1ab752cf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"
Nov 01 05:41:35.341022 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.341004    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"
Nov 01 05:41:35.721242 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.721216    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"]
Nov 01 05:41:35.723370 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:41:35.723340    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b33a7a5_8d14_4281_8639_32a1ab752cf8.slice/crio-a55cb867b899111c2d7f505ecb7fd053319d51ed5f97aac45095eecd9c4bb0a5 WatchSource:0}: Error finding container a55cb867b899111c2d7f505ecb7fd053319d51ed5f97aac45095eecd9c4bb0a5: Status 404 returned error can't find the container with id a55cb867b899111c2d7f505ecb7fd053319d51ed5f97aac45095eecd9c4bb0a5
Nov 01 05:41:35.724938 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:35.724920    4864 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 05:41:36.085474 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:36.085437    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w" event={"ID":"1b33a7a5-8d14-4281-8639-32a1ab752cf8","Type":"ContainerStarted","Data":"a55cb867b899111c2d7f505ecb7fd053319d51ed5f97aac45095eecd9c4bb0a5"}
Nov 01 05:41:37.097642 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:37.097602    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w" event={"ID":"1b33a7a5-8d14-4281-8639-32a1ab752cf8","Type":"ContainerStarted","Data":"1156f732eb1fb344da493bdfc77be0884488a716ba8c5d4b1da5f5639c5c92a3"}
Nov 01 05:41:37.098187 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:37.097672    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"
Nov 01 05:41:37.216182 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:37.216130    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w" podStartSLOduration=2.319616993 podStartE2EDuration="3.216114798s" totalImagesPullingTime="896.497805ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:41:37.213859955 +0000 UTC m=+2314.104235931" watchObservedRunningTime="2026-11-01 05:41:37.216114798 +0000 UTC m=+2314.106490773"
Nov 01 05:41:41.307218 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:41.307181    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"]
Nov 01 05:41:41.307723 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:41.307409    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx" podUID="28515788-ceb7-439c-829f-9de5067031f9" containerName="istio-proxy" containerID="cri-o://055dcbb3d8d97810e1ab7fc7589f33bea5b1c859957492d7fba72eb8847ccfb4" gracePeriod=30
Nov 01 05:41:43.108552 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:43.108524    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"
Nov 01 05:41:45.318225 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:45.318190    4864 patch_prober.go:29] interesting pod/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 05:41:45.318620 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:45.318259    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx" podUID="28515788-ceb7-439c-829f-9de5067031f9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 05:41:46.571966 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.571907    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:41:46.678242 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.678209    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-istio-data\") pod \"28515788-ceb7-439c-829f-9de5067031f9\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") "
Nov 01 05:41:46.678392 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.678291    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-credential-socket\") pod \"28515788-ceb7-439c-829f-9de5067031f9\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") "
Nov 01 05:41:46.678449 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.678391    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28515788-ceb7-439c-829f-9de5067031f9-istio-token\" (UniqueName: \"kubernetes.io/projected/28515788-ceb7-439c-829f-9de5067031f9-istio-token\") pod \"28515788-ceb7-439c-829f-9de5067031f9\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") "
Nov 01 05:41:46.678515 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.678475    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-istio-data" pod "28515788-ceb7-439c-829f-9de5067031f9" (UID: "28515788-ceb7-439c-829f-9de5067031f9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:41:46.678515 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.678481    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/28515788-ceb7-439c-829f-9de5067031f9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28515788-ceb7-439c-829f-9de5067031f9-istiod-ca-cert\") pod \"28515788-ceb7-439c-829f-9de5067031f9\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") "
Nov 01 05:41:46.678626 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.678540    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-workload-certs\") pod \"28515788-ceb7-439c-829f-9de5067031f9\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") "
Nov 01 05:41:46.678626 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.678567    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/28515788-ceb7-439c-829f-9de5067031f9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28515788-ceb7-439c-829f-9de5067031f9-istio-podinfo\") pod \"28515788-ceb7-439c-829f-9de5067031f9\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") "
Nov 01 05:41:46.678626 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.678606    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28515788-ceb7-439c-829f-9de5067031f9-kube-api-access-tqrt9\" (UniqueName: \"kubernetes.io/projected/28515788-ceb7-439c-829f-9de5067031f9-kube-api-access-tqrt9\") pod \"28515788-ceb7-439c-829f-9de5067031f9\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") "
Nov 01 05:41:46.678840 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.678691    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-workload-socket\") pod \"28515788-ceb7-439c-829f-9de5067031f9\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") "
Nov 01 05:41:46.678840 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.678775    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-istio-envoy\") pod \"28515788-ceb7-439c-829f-9de5067031f9\" (UID: \"28515788-ceb7-439c-829f-9de5067031f9\") "
Nov 01 05:41:46.678965 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.678902    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28515788-ceb7-439c-829f-9de5067031f9-istiod-ca-cert" pod "28515788-ceb7-439c-829f-9de5067031f9" (UID: "28515788-ceb7-439c-829f-9de5067031f9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:41:46.679321 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.679254    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-credential-socket" pod "28515788-ceb7-439c-829f-9de5067031f9" (UID: "28515788-ceb7-439c-829f-9de5067031f9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:41:46.679443 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.679335    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:41:46.679443 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.679359    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28515788-ceb7-439c-829f-9de5067031f9-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:41:46.679443 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.679421    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-workload-certs" pod "28515788-ceb7-439c-829f-9de5067031f9" (UID: "28515788-ceb7-439c-829f-9de5067031f9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:41:46.679625 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.679468    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-workload-socket" pod "28515788-ceb7-439c-829f-9de5067031f9" (UID: "28515788-ceb7-439c-829f-9de5067031f9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:41:46.681626 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.681600    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28515788-ceb7-439c-829f-9de5067031f9-istio-token" pod "28515788-ceb7-439c-829f-9de5067031f9" (UID: "28515788-ceb7-439c-829f-9de5067031f9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:41:46.681733 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.681664    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28515788-ceb7-439c-829f-9de5067031f9-kube-api-access-tqrt9" pod "28515788-ceb7-439c-829f-9de5067031f9" (UID: "28515788-ceb7-439c-829f-9de5067031f9"). InnerVolumeSpecName "kube-api-access-tqrt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:41:46.681923 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.681895    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/28515788-ceb7-439c-829f-9de5067031f9-istio-podinfo" pod "28515788-ceb7-439c-829f-9de5067031f9" (UID: "28515788-ceb7-439c-829f-9de5067031f9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 05:41:46.681923 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.681911    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-istio-envoy" pod "28515788-ceb7-439c-829f-9de5067031f9" (UID: "28515788-ceb7-439c-829f-9de5067031f9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:41:46.781018 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.780985    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28515788-ceb7-439c-829f-9de5067031f9-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:41:46.781160 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.781023    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:41:46.781160 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.781040    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28515788-ceb7-439c-829f-9de5067031f9-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:41:46.781160 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.781054    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqrt9\" (UniqueName: \"kubernetes.io/projected/28515788-ceb7-439c-829f-9de5067031f9-kube-api-access-tqrt9\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:41:46.781160 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.781070    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:41:46.781160 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.781082    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:41:46.781160 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:46.781095    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28515788-ceb7-439c-829f-9de5067031f9-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:41:47.165425 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:47.165390    4864 generic.go:356] "Generic (PLEG): container finished" podID="28515788-ceb7-439c-829f-9de5067031f9" containerID="055dcbb3d8d97810e1ab7fc7589f33bea5b1c859957492d7fba72eb8847ccfb4" exitCode=0
Nov 01 05:41:47.165647 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:47.165452    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"
Nov 01 05:41:47.165647 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:47.165457    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx" event={"ID":"28515788-ceb7-439c-829f-9de5067031f9","Type":"ContainerDied","Data":"055dcbb3d8d97810e1ab7fc7589f33bea5b1c859957492d7fba72eb8847ccfb4"}
Nov 01 05:41:47.165647 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:47.165612    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx" event={"ID":"28515788-ceb7-439c-829f-9de5067031f9","Type":"ContainerDied","Data":"e8592e78e9f6a301751edc1cf78abf45bd7197e51a032064fdea85874b41b243"}
Nov 01 05:41:47.165847 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:47.165650    4864 scope.go:122] "RemoveContainer" containerID="055dcbb3d8d97810e1ab7fc7589f33bea5b1c859957492d7fba72eb8847ccfb4"
Nov 01 05:41:47.178380 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:47.178358    4864 scope.go:122] "RemoveContainer" containerID="055dcbb3d8d97810e1ab7fc7589f33bea5b1c859957492d7fba72eb8847ccfb4"
Nov 01 05:41:47.178686 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:41:47.178668    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"055dcbb3d8d97810e1ab7fc7589f33bea5b1c859957492d7fba72eb8847ccfb4\": container with ID starting with 055dcbb3d8d97810e1ab7fc7589f33bea5b1c859957492d7fba72eb8847ccfb4 not found: ID does not exist" containerID="055dcbb3d8d97810e1ab7fc7589f33bea5b1c859957492d7fba72eb8847ccfb4"
Nov 01 05:41:47.178760 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:47.178694    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"055dcbb3d8d97810e1ab7fc7589f33bea5b1c859957492d7fba72eb8847ccfb4"} err="failed to get container status \"055dcbb3d8d97810e1ab7fc7589f33bea5b1c859957492d7fba72eb8847ccfb4\": rpc error: code = NotFound desc = could not find container \"055dcbb3d8d97810e1ab7fc7589f33bea5b1c859957492d7fba72eb8847ccfb4\": container with ID starting with 055dcbb3d8d97810e1ab7fc7589f33bea5b1c859957492d7fba72eb8847ccfb4 not found: ID does not exist"
Nov 01 05:41:47.330000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:47.329972    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"]
Nov 01 05:41:47.379898 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:47.379861    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx"]
Nov 01 05:41:49.246681 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:49.246644    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28515788-ceb7-439c-829f-9de5067031f9" path="/var/lib/kubelet/pods/28515788-ceb7-439c-829f-9de5067031f9/volumes"
Nov 01 05:41:52.524364 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:41:52.524332    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:41:59.085088 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:59.085056    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"]
Nov 01 05:41:59.085577 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:59.085562    4864 cpu_manager.go:414] "containerMap: removing container" podUID="28515788-ceb7-439c-829f-9de5067031f9" containerName="istio-proxy"
Nov 01 05:41:59.085621 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:59.085578    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28515788-ceb7-439c-829f-9de5067031f9" containerName="istio-proxy"
Nov 01 05:41:59.085672 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:59.085663    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="28515788-ceb7-439c-829f-9de5067031f9" containerName="istio-proxy"
Nov 01 05:41:59.089000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:59.088973    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"
Nov 01 05:41:59.115217 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:59.115194    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7497187-82da-4486-9b88-c5ebb34e565f-openshift-service-ca\") pod \"maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g\" (UID: \"d7497187-82da-4486-9b88-c5ebb34e565f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"
Nov 01 05:41:59.115327 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:59.115241    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7497187-82da-4486-9b88-c5ebb34e565f-metrics-certs\") pod \"maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g\" (UID: \"d7497187-82da-4486-9b88-c5ebb34e565f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"
Nov 01 05:41:59.115369 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:59.115351    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7497187-82da-4486-9b88-c5ebb34e565f-maas-api-tls\") pod \"maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g\" (UID: \"d7497187-82da-4486-9b88-c5ebb34e565f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"
Nov 01 05:41:59.115456 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:59.115441    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkgrc\" (UniqueName: \"kubernetes.io/projected/d7497187-82da-4486-9b88-c5ebb34e565f-kube-api-access-lkgrc\") pod \"maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g\" (UID: \"d7497187-82da-4486-9b88-c5ebb34e565f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"
Nov 01 05:41:59.140701 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:59.140652    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ad90c52d-serving-cert\""
Nov 01 05:41:59.200891 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:59.200862    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"]
Nov 01 05:41:59.216081 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:59.216052    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkgrc\" (UniqueName: \"kubernetes.io/projected/d7497187-82da-4486-9b88-c5ebb34e565f-kube-api-access-lkgrc\") pod \"maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g\" (UID: \"d7497187-82da-4486-9b88-c5ebb34e565f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"
Nov 01 05:41:59.216250 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:59.216108    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7497187-82da-4486-9b88-c5ebb34e565f-openshift-service-ca\") pod \"maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g\" (UID: \"d7497187-82da-4486-9b88-c5ebb34e565f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"
Nov 01 05:41:59.216250 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:59.216158    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7497187-82da-4486-9b88-c5ebb34e565f-metrics-certs\") pod \"maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g\" (UID: \"d7497187-82da-4486-9b88-c5ebb34e565f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"
Nov 01 05:41:59.216250 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:59.216215    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7497187-82da-4486-9b88-c5ebb34e565f-maas-api-tls\") pod \"maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g\" (UID: \"d7497187-82da-4486-9b88-c5ebb34e565f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"
Nov 01 05:41:59.216441 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:41:59.216328    4864 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-serving-cert: secret "maas-api-e2e-ait-ad90c52d-serving-cert" not found
Nov 01 05:41:59.216441 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:41:59.216378    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7497187-82da-4486-9b88-c5ebb34e565f-maas-api-tls podName:d7497187-82da-4486-9b88-c5ebb34e565f nodeName:}" failed. No retries permitted until 2026-11-01 05:41:59.716360611 +0000 UTC m=+2336.606736579 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d7497187-82da-4486-9b88-c5ebb34e565f-maas-api-tls") pod "maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g" (UID: "d7497187-82da-4486-9b88-c5ebb34e565f") : secret "maas-api-e2e-ait-ad90c52d-serving-cert" not found
Nov 01 05:41:59.217434 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:59.217407    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7497187-82da-4486-9b88-c5ebb34e565f-openshift-service-ca\") pod \"maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g\" (UID: \"d7497187-82da-4486-9b88-c5ebb34e565f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"
Nov 01 05:41:59.219511 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:59.219481    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7497187-82da-4486-9b88-c5ebb34e565f-metrics-certs\") pod \"maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g\" (UID: \"d7497187-82da-4486-9b88-c5ebb34e565f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"
Nov 01 05:41:59.277305 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:59.277268    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkgrc\" (UniqueName: \"kubernetes.io/projected/d7497187-82da-4486-9b88-c5ebb34e565f-kube-api-access-lkgrc\") pod \"maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g\" (UID: \"d7497187-82da-4486-9b88-c5ebb34e565f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"
Nov 01 05:41:59.721501 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:59.721466    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7497187-82da-4486-9b88-c5ebb34e565f-maas-api-tls\") pod \"maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g\" (UID: \"d7497187-82da-4486-9b88-c5ebb34e565f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"
Nov 01 05:41:59.724115 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:41:59.724086    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7497187-82da-4486-9b88-c5ebb34e565f-maas-api-tls\") pod \"maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g\" (UID: \"d7497187-82da-4486-9b88-c5ebb34e565f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"
Nov 01 05:42:00.003722 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:42:00.003639    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"
Nov 01 05:42:00.393760 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:42:00.393737    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"]
Nov 01 05:42:00.396487 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:42:00.396457    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7497187_82da_4486_9b88_c5ebb34e565f.slice/crio-ab4675051a3bf8ce7789c11d588be8d76285354dbef800f4b809e3e0c323a9d4 WatchSource:0}: Error finding container ab4675051a3bf8ce7789c11d588be8d76285354dbef800f4b809e3e0c323a9d4: Status 404 returned error can't find the container with id ab4675051a3bf8ce7789c11d588be8d76285354dbef800f4b809e3e0c323a9d4
Nov 01 05:42:01.270700 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:42:01.270590    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g" event={"ID":"d7497187-82da-4486-9b88-c5ebb34e565f","Type":"ContainerStarted","Data":"ab4675051a3bf8ce7789c11d588be8d76285354dbef800f4b809e3e0c323a9d4"}
Nov 01 05:42:02.278718 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:42:02.278671    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g" event={"ID":"d7497187-82da-4486-9b88-c5ebb34e565f","Type":"ContainerStarted","Data":"acc01a7bc3e1c04c03b54c8a2c60637c255091ca8c10409e7aa5e5368866c2e3"}
Nov 01 05:42:02.279114 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:42:02.278748    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"
Nov 01 05:42:02.400771 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:42:02.400685    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g" podStartSLOduration=2.548871124 podStartE2EDuration="3.400667095s" totalImagesPullingTime="851.795971ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:41:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:42:02.395720287 +0000 UTC m=+2339.286096264" watchObservedRunningTime="2026-11-01 05:42:02.400667095 +0000 UTC m=+2339.291043071"
Nov 01 05:42:08.290714 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:42:08.290682    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"
Nov 01 05:42:22.524479 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:42:22.524445    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:42:52.524722 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:42:52.524635    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:43:00.920008 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:00.919961    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"]
Nov 01 05:43:00.920484 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:00.920186    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g" podUID="5975aa80-aa20-4896-b06f-222a39b0d6d2" containerName="istio-proxy" containerID="cri-o://eaa611d49034de9eb92d3113d615d2771012bff59b56daadbfeaf530f8414fe9" gracePeriod=30
Nov 01 05:43:03.920911 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:03.920880    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:43:03.923668 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:03.923642    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:43:03.929405 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:03.929383    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:43:03.931894 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:03.931874    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:43:05.627114 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:05.627076    4864 patch_prober.go:29] interesting pod/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 05:43:05.627567 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:05.627149    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g" podUID="5975aa80-aa20-4896-b06f-222a39b0d6d2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 05:43:06.187125 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.187073    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:43:06.283719 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.283691    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-podinfo\") pod \"5975aa80-aa20-4896-b06f-222a39b0d6d2\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") "
Nov 01 05:43:06.283898 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.283739    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-credential-socket\") pod \"5975aa80-aa20-4896-b06f-222a39b0d6d2\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") "
Nov 01 05:43:06.283898 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.283810    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-token\" (UniqueName: \"kubernetes.io/projected/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-token\") pod \"5975aa80-aa20-4896-b06f-222a39b0d6d2\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") "
Nov 01 05:43:06.283898 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.283851    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-data\") pod \"5975aa80-aa20-4896-b06f-222a39b0d6d2\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") "
Nov 01 05:43:06.284041 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.283963    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5975aa80-aa20-4896-b06f-222a39b0d6d2-kube-api-access-zq5n7\" (UniqueName: \"kubernetes.io/projected/5975aa80-aa20-4896-b06f-222a39b0d6d2-kube-api-access-zq5n7\") pod \"5975aa80-aa20-4896-b06f-222a39b0d6d2\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") "
Nov 01 05:43:06.284091 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.284063    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-workload-socket\") pod \"5975aa80-aa20-4896-b06f-222a39b0d6d2\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") "
Nov 01 05:43:06.284147 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.284118    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-data" pod "5975aa80-aa20-4896-b06f-222a39b0d6d2" (UID: "5975aa80-aa20-4896-b06f-222a39b0d6d2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:43:06.284200 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.284133    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-credential-socket" pod "5975aa80-aa20-4896-b06f-222a39b0d6d2" (UID: "5975aa80-aa20-4896-b06f-222a39b0d6d2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:43:06.284200 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.284160    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-workload-certs\") pod \"5975aa80-aa20-4896-b06f-222a39b0d6d2\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") "
Nov 01 05:43:06.284200 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.284190    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5975aa80-aa20-4896-b06f-222a39b0d6d2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5975aa80-aa20-4896-b06f-222a39b0d6d2-istiod-ca-cert\") pod \"5975aa80-aa20-4896-b06f-222a39b0d6d2\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") "
Nov 01 05:43:06.284356 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.284300    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-envoy\") pod \"5975aa80-aa20-4896-b06f-222a39b0d6d2\" (UID: \"5975aa80-aa20-4896-b06f-222a39b0d6d2\") "
Nov 01 05:43:06.284356 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.284309    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-workload-socket" pod "5975aa80-aa20-4896-b06f-222a39b0d6d2" (UID: "5975aa80-aa20-4896-b06f-222a39b0d6d2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:43:06.284461 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.284419    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-workload-certs" pod "5975aa80-aa20-4896-b06f-222a39b0d6d2" (UID: "5975aa80-aa20-4896-b06f-222a39b0d6d2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:43:06.284629 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.284607    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5975aa80-aa20-4896-b06f-222a39b0d6d2-istiod-ca-cert" pod "5975aa80-aa20-4896-b06f-222a39b0d6d2" (UID: "5975aa80-aa20-4896-b06f-222a39b0d6d2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:43:06.284825 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.284748    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:43:06.285051 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.284830    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5975aa80-aa20-4896-b06f-222a39b0d6d2-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:43:06.285051 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.284846    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:43:06.285051 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.284857    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:43:06.285051 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.284870    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:43:06.286432 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.286405    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-token" pod "5975aa80-aa20-4896-b06f-222a39b0d6d2" (UID: "5975aa80-aa20-4896-b06f-222a39b0d6d2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:43:06.286709 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.286683    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-envoy" pod "5975aa80-aa20-4896-b06f-222a39b0d6d2" (UID: "5975aa80-aa20-4896-b06f-222a39b0d6d2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:43:06.286709 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.286698    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-podinfo" pod "5975aa80-aa20-4896-b06f-222a39b0d6d2" (UID: "5975aa80-aa20-4896-b06f-222a39b0d6d2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 05:43:06.286870 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.286844    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5975aa80-aa20-4896-b06f-222a39b0d6d2-kube-api-access-zq5n7" pod "5975aa80-aa20-4896-b06f-222a39b0d6d2" (UID: "5975aa80-aa20-4896-b06f-222a39b0d6d2"). InnerVolumeSpecName "kube-api-access-zq5n7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:43:06.385900 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.385865    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:43:06.385900 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.385902    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zq5n7\" (UniqueName: \"kubernetes.io/projected/5975aa80-aa20-4896-b06f-222a39b0d6d2-kube-api-access-zq5n7\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:43:06.386098 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.385915    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:43:06.386098 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.385923    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5975aa80-aa20-4896-b06f-222a39b0d6d2-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:43:06.711067 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.711034    4864 generic.go:356] "Generic (PLEG): container finished" podID="5975aa80-aa20-4896-b06f-222a39b0d6d2" containerID="eaa611d49034de9eb92d3113d615d2771012bff59b56daadbfeaf530f8414fe9" exitCode=0
Nov 01 05:43:06.711543 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.711086    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g" event={"ID":"5975aa80-aa20-4896-b06f-222a39b0d6d2","Type":"ContainerDied","Data":"eaa611d49034de9eb92d3113d615d2771012bff59b56daadbfeaf530f8414fe9"}
Nov 01 05:43:06.711543 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.711097    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"
Nov 01 05:43:06.711543 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.711128    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g" event={"ID":"5975aa80-aa20-4896-b06f-222a39b0d6d2","Type":"ContainerDied","Data":"7eaafef72520604c6976c3b10142b487334e7330f56720dcd46172dd1c0cd50e"}
Nov 01 05:43:06.711543 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.711151    4864 scope.go:122] "RemoveContainer" containerID="eaa611d49034de9eb92d3113d615d2771012bff59b56daadbfeaf530f8414fe9"
Nov 01 05:43:06.722662 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.722640    4864 scope.go:122] "RemoveContainer" containerID="0e94704ca54db39389dae8c6b9af813af1294161fe5872f22cf0af9dde1c4f46"
Nov 01 05:43:06.733578 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.733559    4864 scope.go:122] "RemoveContainer" containerID="eaa611d49034de9eb92d3113d615d2771012bff59b56daadbfeaf530f8414fe9"
Nov 01 05:43:06.733911 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:43:06.733890    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaa611d49034de9eb92d3113d615d2771012bff59b56daadbfeaf530f8414fe9\": container with ID starting with eaa611d49034de9eb92d3113d615d2771012bff59b56daadbfeaf530f8414fe9 not found: ID does not exist" containerID="eaa611d49034de9eb92d3113d615d2771012bff59b56daadbfeaf530f8414fe9"
Nov 01 05:43:06.733977 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.733920    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaa611d49034de9eb92d3113d615d2771012bff59b56daadbfeaf530f8414fe9"} err="failed to get container status \"eaa611d49034de9eb92d3113d615d2771012bff59b56daadbfeaf530f8414fe9\": rpc error: code = NotFound desc = could not find container \"eaa611d49034de9eb92d3113d615d2771012bff59b56daadbfeaf530f8414fe9\": container with ID starting with eaa611d49034de9eb92d3113d615d2771012bff59b56daadbfeaf530f8414fe9 not found: ID does not exist"
Nov 01 05:43:06.733977 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.733940    4864 scope.go:122] "RemoveContainer" containerID="0e94704ca54db39389dae8c6b9af813af1294161fe5872f22cf0af9dde1c4f46"
Nov 01 05:43:06.734171 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:43:06.734152    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e94704ca54db39389dae8c6b9af813af1294161fe5872f22cf0af9dde1c4f46\": container with ID starting with 0e94704ca54db39389dae8c6b9af813af1294161fe5872f22cf0af9dde1c4f46 not found: ID does not exist" containerID="0e94704ca54db39389dae8c6b9af813af1294161fe5872f22cf0af9dde1c4f46"
Nov 01 05:43:06.734221 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.734176    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e94704ca54db39389dae8c6b9af813af1294161fe5872f22cf0af9dde1c4f46"} err="failed to get container status \"0e94704ca54db39389dae8c6b9af813af1294161fe5872f22cf0af9dde1c4f46\": rpc error: code = NotFound desc = could not find container \"0e94704ca54db39389dae8c6b9af813af1294161fe5872f22cf0af9dde1c4f46\": container with ID starting with 0e94704ca54db39389dae8c6b9af813af1294161fe5872f22cf0af9dde1c4f46 not found: ID does not exist"
Nov 01 05:43:06.877385 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.877346    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"]
Nov 01 05:43:06.880284 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:06.880262    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g"]
Nov 01 05:43:07.247681 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:43:07.247639    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5975aa80-aa20-4896-b06f-222a39b0d6d2" path="/var/lib/kubelet/pods/5975aa80-aa20-4896-b06f-222a39b0d6d2/volumes"
Nov 01 05:43:22.541051 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:43:22.541004    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:43:52.524635 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:43:52.524601    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:44:03.879207 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:03.879173    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"]
Nov 01 05:44:03.879710 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:03.879417    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g" podUID="d7497187-82da-4486-9b88-c5ebb34e565f" containerName="maas-api" containerID="cri-o://acc01a7bc3e1c04c03b54c8a2c60637c255091ca8c10409e7aa5e5368866c2e3" gracePeriod=30
Nov 01 05:44:04.144444 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:04.144139    4864 generic.go:356] "Generic (PLEG): container finished" podID="d7497187-82da-4486-9b88-c5ebb34e565f" containerID="acc01a7bc3e1c04c03b54c8a2c60637c255091ca8c10409e7aa5e5368866c2e3" exitCode=0
Nov 01 05:44:04.144444 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:04.144327    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g" event={"ID":"d7497187-82da-4486-9b88-c5ebb34e565f","Type":"ContainerDied","Data":"acc01a7bc3e1c04c03b54c8a2c60637c255091ca8c10409e7aa5e5368866c2e3"}
Nov 01 05:44:04.211827 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:04.211769    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"
Nov 01 05:44:04.239979 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:04.239951    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7497187-82da-4486-9b88-c5ebb34e565f-kube-api-access-lkgrc\" (UniqueName: \"kubernetes.io/projected/d7497187-82da-4486-9b88-c5ebb34e565f-kube-api-access-lkgrc\") pod \"d7497187-82da-4486-9b88-c5ebb34e565f\" (UID: \"d7497187-82da-4486-9b88-c5ebb34e565f\") "
Nov 01 05:44:04.240165 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:04.239993    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d7497187-82da-4486-9b88-c5ebb34e565f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7497187-82da-4486-9b88-c5ebb34e565f-maas-api-tls\") pod \"d7497187-82da-4486-9b88-c5ebb34e565f\" (UID: \"d7497187-82da-4486-9b88-c5ebb34e565f\") "
Nov 01 05:44:04.240165 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:04.240064    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7497187-82da-4486-9b88-c5ebb34e565f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7497187-82da-4486-9b88-c5ebb34e565f-openshift-service-ca\") pod \"d7497187-82da-4486-9b88-c5ebb34e565f\" (UID: \"d7497187-82da-4486-9b88-c5ebb34e565f\") "
Nov 01 05:44:04.240165 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:04.240130    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d7497187-82da-4486-9b88-c5ebb34e565f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7497187-82da-4486-9b88-c5ebb34e565f-metrics-certs\") pod \"d7497187-82da-4486-9b88-c5ebb34e565f\" (UID: \"d7497187-82da-4486-9b88-c5ebb34e565f\") "
Nov 01 05:44:04.240965 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:04.240931    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7497187-82da-4486-9b88-c5ebb34e565f-openshift-service-ca" pod "d7497187-82da-4486-9b88-c5ebb34e565f" (UID: "d7497187-82da-4486-9b88-c5ebb34e565f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:44:04.243620 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:04.243294    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7497187-82da-4486-9b88-c5ebb34e565f-metrics-certs" pod "d7497187-82da-4486-9b88-c5ebb34e565f" (UID: "d7497187-82da-4486-9b88-c5ebb34e565f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:44:04.243620 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:04.243579    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7497187-82da-4486-9b88-c5ebb34e565f-kube-api-access-lkgrc" pod "d7497187-82da-4486-9b88-c5ebb34e565f" (UID: "d7497187-82da-4486-9b88-c5ebb34e565f"). InnerVolumeSpecName "kube-api-access-lkgrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:44:04.244658 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:04.244639    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7497187-82da-4486-9b88-c5ebb34e565f-maas-api-tls" pod "d7497187-82da-4486-9b88-c5ebb34e565f" (UID: "d7497187-82da-4486-9b88-c5ebb34e565f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:44:04.341859 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:04.341815    4864 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7497187-82da-4486-9b88-c5ebb34e565f-openshift-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:44:04.341859 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:04.341857    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7497187-82da-4486-9b88-c5ebb34e565f-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:44:04.342141 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:04.341872    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkgrc\" (UniqueName: \"kubernetes.io/projected/d7497187-82da-4486-9b88-c5ebb34e565f-kube-api-access-lkgrc\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:44:04.342141 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:04.341887    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7497187-82da-4486-9b88-c5ebb34e565f-maas-api-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:44:05.152715 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:05.152677    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g" event={"ID":"d7497187-82da-4486-9b88-c5ebb34e565f","Type":"ContainerDied","Data":"ab4675051a3bf8ce7789c11d588be8d76285354dbef800f4b809e3e0c323a9d4"}
Nov 01 05:44:05.152715 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:05.152711    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"
Nov 01 05:44:05.153382 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:05.152757    4864 scope.go:122] "RemoveContainer" containerID="acc01a7bc3e1c04c03b54c8a2c60637c255091ca8c10409e7aa5e5368866c2e3"
Nov 01 05:44:05.316281 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:05.316246    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"]
Nov 01 05:44:05.320603 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:05.320572    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ad90c52d-57bd996767-n4x4g"]
Nov 01 05:44:07.247272 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:07.247238    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7497187-82da-4486-9b88-c5ebb34e565f" path="/var/lib/kubelet/pods/d7497187-82da-4486-9b88-c5ebb34e565f/volumes"
Nov 01 05:44:08.184719 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.184683    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"]
Nov 01 05:44:08.185286 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.185268    4864 cpu_manager.go:414] "containerMap: removing container" podUID="5975aa80-aa20-4896-b06f-222a39b0d6d2" containerName="istio-proxy"
Nov 01 05:44:08.185361 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.185288    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5975aa80-aa20-4896-b06f-222a39b0d6d2" containerName="istio-proxy"
Nov 01 05:44:08.185361 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.185339    4864 cpu_manager.go:414] "containerMap: removing container" podUID="d7497187-82da-4486-9b88-c5ebb34e565f" containerName="maas-api"
Nov 01 05:44:08.185361 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.185345    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7497187-82da-4486-9b88-c5ebb34e565f" containerName="maas-api"
Nov 01 05:44:08.185361 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.185352    4864 cpu_manager.go:414] "containerMap: removing container" podUID="5975aa80-aa20-4896-b06f-222a39b0d6d2" containerName="istio-proxy"
Nov 01 05:44:08.185361 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.185357    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5975aa80-aa20-4896-b06f-222a39b0d6d2" containerName="istio-proxy"
Nov 01 05:44:08.185524 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.185439    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7497187-82da-4486-9b88-c5ebb34e565f" containerName="maas-api"
Nov 01 05:44:08.185524 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.185446    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="5975aa80-aa20-4896-b06f-222a39b0d6d2" containerName="istio-proxy"
Nov 01 05:44:08.189911 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.189891    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.241758 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.241728    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-31e728ff-openshift-default-dockercfg-x5p5z\""
Nov 01 05:44:08.294832 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.294800    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"]
Nov 01 05:44:08.300183 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.300157    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-data\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.300328 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.300309    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v2wk\" (UniqueName: \"kubernetes.io/projected/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-kube-api-access-7v2wk\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.300475 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.300452    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-envoy\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.300584 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.300567    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-workload-socket\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.300648 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.300620    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-token\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.300747 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.300729    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-credential-socket\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.300800 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.300754    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-podinfo\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.300854 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.300800    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-workload-certs\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.300934 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.300913    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istiod-ca-cert\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.401919 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.401882    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-token\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.402124 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.401937    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-credential-socket\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.402124 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.402042    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-podinfo\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.402124 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.402078    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-workload-certs\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.402124 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.402109    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istiod-ca-cert\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.402360 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.402161    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-data\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.402360 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.402249    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v2wk\" (UniqueName: \"kubernetes.io/projected/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-kube-api-access-7v2wk\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.402360 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.402303    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-credential-socket\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.402360 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.402308    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-envoy\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.402580 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.402363    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-workload-socket\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.402580 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.402363    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-workload-certs\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.402698 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.402595    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-workload-socket\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.402698 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.402656    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-data\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.403148 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.403122    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istiod-ca-cert\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.404715 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.404690    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-envoy\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.404899 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.404883    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-podinfo\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.462924 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.462847    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-token\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.463172 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.463151    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v2wk\" (UniqueName: \"kubernetes.io/projected/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-kube-api-access-7v2wk\") pod \"e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") " pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.502882 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.502850    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:08.986981 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.986952    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"]
Nov 01 05:44:08.988412 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:44:08.988378    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e9c0e6d_f282_45d2_a1e3_8753edbbc355.slice/crio-19fcac466997b21ec2c6c6718ef333f5a9e69a717872adc7c06d91b734418b3d WatchSource:0}: Error finding container 19fcac466997b21ec2c6c6718ef333f5a9e69a717872adc7c06d91b734418b3d: Status 404 returned error can't find the container with id 19fcac466997b21ec2c6c6718ef333f5a9e69a717872adc7c06d91b734418b3d
Nov 01 05:44:08.990701 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.990667    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:44:08.990805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.990730    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:44:08.990805 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:08.990759    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:44:09.184364 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:09.184326    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698" event={"ID":"0e9c0e6d-f282-45d2-a1e3-8753edbbc355","Type":"ContainerStarted","Data":"8c4858fcd84c6b12145d47343f54e18f3d2b4a8049cad2edb4a7bd1f8d3005f9"}
Nov 01 05:44:09.184555 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:09.184386    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698" event={"ID":"0e9c0e6d-f282-45d2-a1e3-8753edbbc355","Type":"ContainerStarted","Data":"19fcac466997b21ec2c6c6718ef333f5a9e69a717872adc7c06d91b734418b3d"}
Nov 01 05:44:09.307378 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:09.307307    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698" podStartSLOduration=1.307288549 podStartE2EDuration="1.307288549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:44:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:44:09.30677314 +0000 UTC m=+2466.197149130" watchObservedRunningTime="2026-11-01 05:44:09.307288549 +0000 UTC m=+2466.197664527"
Nov 01 05:44:09.503891 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:09.503851    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:10.509101 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:10.509072    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:11.197668 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:11.197635    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:11.198901 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:11.198879    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:44:22.525284 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:44:22.525195    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:44:25.427297 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:25.427267    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"]
Nov 01 05:44:25.427926 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:25.427910    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="5975aa80-aa20-4896-b06f-222a39b0d6d2" containerName="istio-proxy"
Nov 01 05:44:25.431328 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:25.431313    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"
Nov 01 05:44:25.483107 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:25.483077    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-25565930-serving-cert\""
Nov 01 05:44:25.502083 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:25.502056    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-maas-api-tls\") pod \"maas-api-e2e-mt-25565930-847c47b46f-6mnzx\" (UID: \"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"
Nov 01 05:44:25.502222 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:25.502094    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-metrics-certs\") pod \"maas-api-e2e-mt-25565930-847c47b46f-6mnzx\" (UID: \"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"
Nov 01 05:44:25.502266 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:25.502219    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87mqb\" (UniqueName: \"kubernetes.io/projected/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-kube-api-access-87mqb\") pod \"maas-api-e2e-mt-25565930-847c47b46f-6mnzx\" (UID: \"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"
Nov 01 05:44:25.502308 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:25.502278    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-openshift-service-ca\") pod \"maas-api-e2e-mt-25565930-847c47b46f-6mnzx\" (UID: \"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"
Nov 01 05:44:25.535616 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:25.535592    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"]
Nov 01 05:44:25.603608 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:25.603572    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-maas-api-tls\") pod \"maas-api-e2e-mt-25565930-847c47b46f-6mnzx\" (UID: \"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"
Nov 01 05:44:25.603818 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:25.603623    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-metrics-certs\") pod \"maas-api-e2e-mt-25565930-847c47b46f-6mnzx\" (UID: \"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"
Nov 01 05:44:25.603818 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:25.603654    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87mqb\" (UniqueName: \"kubernetes.io/projected/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-kube-api-access-87mqb\") pod \"maas-api-e2e-mt-25565930-847c47b46f-6mnzx\" (UID: \"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"
Nov 01 05:44:25.603818 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:25.603688    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-openshift-service-ca\") pod \"maas-api-e2e-mt-25565930-847c47b46f-6mnzx\" (UID: \"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"
Nov 01 05:44:25.603818 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:44:25.603745    4864 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-25565930-serving-cert: secret "maas-api-e2e-mt-25565930-serving-cert" not found
Nov 01 05:44:25.604052 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:44:25.603882    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-maas-api-tls podName:a148b0c7-891c-4eba-8f2e-f5dfeaaec96b nodeName:}" failed. No retries permitted until 2026-11-01 05:44:26.103858008 +0000 UTC m=+2482.994233981 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-maas-api-tls") pod "maas-api-e2e-mt-25565930-847c47b46f-6mnzx" (UID: "a148b0c7-891c-4eba-8f2e-f5dfeaaec96b") : secret "maas-api-e2e-mt-25565930-serving-cert" not found
Nov 01 05:44:25.604438 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:25.604411    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-openshift-service-ca\") pod \"maas-api-e2e-mt-25565930-847c47b46f-6mnzx\" (UID: \"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"
Nov 01 05:44:25.606321 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:25.606293    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-metrics-certs\") pod \"maas-api-e2e-mt-25565930-847c47b46f-6mnzx\" (UID: \"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"
Nov 01 05:44:25.664581 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:25.664552    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87mqb\" (UniqueName: \"kubernetes.io/projected/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-kube-api-access-87mqb\") pod \"maas-api-e2e-mt-25565930-847c47b46f-6mnzx\" (UID: \"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"
Nov 01 05:44:26.109272 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:26.109240    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-maas-api-tls\") pod \"maas-api-e2e-mt-25565930-847c47b46f-6mnzx\" (UID: \"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"
Nov 01 05:44:26.111870 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:26.111848    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-maas-api-tls\") pod \"maas-api-e2e-mt-25565930-847c47b46f-6mnzx\" (UID: \"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"
Nov 01 05:44:26.345654 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:26.345620    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"
Nov 01 05:44:26.731378 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:26.731343    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"]
Nov 01 05:44:26.732832 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:44:26.732755    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda148b0c7_891c_4eba_8f2e_f5dfeaaec96b.slice/crio-d3a290e5d8e93245bc9fec51dbcc5f964961a243eaaf23ba0ba2e069fbab8546 WatchSource:0}: Error finding container d3a290e5d8e93245bc9fec51dbcc5f964961a243eaaf23ba0ba2e069fbab8546: Status 404 returned error can't find the container with id d3a290e5d8e93245bc9fec51dbcc5f964961a243eaaf23ba0ba2e069fbab8546
Nov 01 05:44:27.299814 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:27.299761    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx" event={"ID":"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b","Type":"ContainerStarted","Data":"d3a290e5d8e93245bc9fec51dbcc5f964961a243eaaf23ba0ba2e069fbab8546"}
Nov 01 05:44:28.308419 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:28.308374    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx" event={"ID":"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b","Type":"ContainerStarted","Data":"b48c2517cb57dde5d2f0817d80d07278b67de73a62d2fb036a01756c62b82c77"}
Nov 01 05:44:28.308926 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:28.308437    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"
Nov 01 05:44:28.427159 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:28.427089    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx" podStartSLOduration=2.583195044 podStartE2EDuration="3.427069162s" totalImagesPullingTime="843.874118ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:44:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:44:28.424202624 +0000 UTC m=+2485.314578601" watchObservedRunningTime="2026-11-01 05:44:28.427069162 +0000 UTC m=+2485.317445137"
Nov 01 05:44:34.327362 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:44:34.327335    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"
Nov 01 05:44:52.525857 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:44:52.525817    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:45:00.196586 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.196550    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5"]
Nov 01 05:45:00.199284 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.199254    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5"
Nov 01 05:45:00.199433 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.199324    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn"]
Nov 01 05:45:00.201876 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.201853    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn"
Nov 01 05:45:00.220266 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.220243    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w"]
Nov 01 05:45:00.222562 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.222547    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w"
Nov 01 05:45:00.285555 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.285528    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8a9f52e-e39d-4a35-930b-23d1521646d1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn\" (UID: \"c8a9f52e-e39d-4a35-930b-23d1521646d1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn"
Nov 01 05:45:00.285678 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.285572    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sppmd\" (UniqueName: \"kubernetes.io/projected/c4b34f80-4eb0-4d2a-aee8-b2136ed54f83-kube-api-access-sppmd\") pod \"maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5\" (UID: \"c4b34f80-4eb0-4d2a-aee8-b2136ed54f83\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5"
Nov 01 05:45:00.285750 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.285695    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4b34f80-4eb0-4d2a-aee8-b2136ed54f83-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5\" (UID: \"c4b34f80-4eb0-4d2a-aee8-b2136ed54f83\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5"
Nov 01 05:45:00.285750 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.285738    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl7qs\" (UniqueName: \"kubernetes.io/projected/c8a9f52e-e39d-4a35-930b-23d1521646d1-kube-api-access-cl7qs\") pod \"maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn\" (UID: \"c8a9f52e-e39d-4a35-930b-23d1521646d1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn"
Nov 01 05:45:00.326737 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.326707    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn"]
Nov 01 05:45:00.360833 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.360779    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sp8g2\""
Nov 01 05:45:00.369865 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.369842    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w"]
Nov 01 05:45:00.386961 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.386934    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4b34f80-4eb0-4d2a-aee8-b2136ed54f83-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5\" (UID: \"c4b34f80-4eb0-4d2a-aee8-b2136ed54f83\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5"
Nov 01 05:45:00.387080 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.387003    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqtjg\" (UniqueName: \"kubernetes.io/projected/c6b1ddd9-f87e-458f-9f38-e84b141883d1-kube-api-access-rqtjg\") pod \"maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w\" (UID: \"c6b1ddd9-f87e-458f-9f38-e84b141883d1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w"
Nov 01 05:45:00.387080 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.387030    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl7qs\" (UniqueName: \"kubernetes.io/projected/c8a9f52e-e39d-4a35-930b-23d1521646d1-kube-api-access-cl7qs\") pod \"maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn\" (UID: \"c8a9f52e-e39d-4a35-930b-23d1521646d1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn"
Nov 01 05:45:00.387183 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.387159    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6b1ddd9-f87e-458f-9f38-e84b141883d1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w\" (UID: \"c6b1ddd9-f87e-458f-9f38-e84b141883d1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w"
Nov 01 05:45:00.387361 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.387336    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8a9f52e-e39d-4a35-930b-23d1521646d1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn\" (UID: \"c8a9f52e-e39d-4a35-930b-23d1521646d1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn"
Nov 01 05:45:00.387438 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.387422    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sppmd\" (UniqueName: \"kubernetes.io/projected/c4b34f80-4eb0-4d2a-aee8-b2136ed54f83-kube-api-access-sppmd\") pod \"maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5\" (UID: \"c4b34f80-4eb0-4d2a-aee8-b2136ed54f83\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5"
Nov 01 05:45:00.387652 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.387633    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4b34f80-4eb0-4d2a-aee8-b2136ed54f83-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5\" (UID: \"c4b34f80-4eb0-4d2a-aee8-b2136ed54f83\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5"
Nov 01 05:45:00.387980 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.387962    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8a9f52e-e39d-4a35-930b-23d1521646d1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn\" (UID: \"c8a9f52e-e39d-4a35-930b-23d1521646d1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn"
Nov 01 05:45:00.398029 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.398008    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5"]
Nov 01 05:45:00.448067 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.448012    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl7qs\" (UniqueName: \"kubernetes.io/projected/c8a9f52e-e39d-4a35-930b-23d1521646d1-kube-api-access-cl7qs\") pod \"maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn\" (UID: \"c8a9f52e-e39d-4a35-930b-23d1521646d1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn"
Nov 01 05:45:00.448210 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.448190    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sppmd\" (UniqueName: \"kubernetes.io/projected/c4b34f80-4eb0-4d2a-aee8-b2136ed54f83-kube-api-access-sppmd\") pod \"maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5\" (UID: \"c4b34f80-4eb0-4d2a-aee8-b2136ed54f83\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5"
Nov 01 05:45:00.488126 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.488101    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqtjg\" (UniqueName: \"kubernetes.io/projected/c6b1ddd9-f87e-458f-9f38-e84b141883d1-kube-api-access-rqtjg\") pod \"maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w\" (UID: \"c6b1ddd9-f87e-458f-9f38-e84b141883d1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w"
Nov 01 05:45:00.488273 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.488170    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6b1ddd9-f87e-458f-9f38-e84b141883d1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w\" (UID: \"c6b1ddd9-f87e-458f-9f38-e84b141883d1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w"
Nov 01 05:45:00.488739 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.488717    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6b1ddd9-f87e-458f-9f38-e84b141883d1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w\" (UID: \"c6b1ddd9-f87e-458f-9f38-e84b141883d1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w"
Nov 01 05:45:00.516895 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.516873    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5"
Nov 01 05:45:00.523204 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.523181    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn"
Nov 01 05:45:00.549583 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.549559    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqtjg\" (UniqueName: \"kubernetes.io/projected/c6b1ddd9-f87e-458f-9f38-e84b141883d1-kube-api-access-rqtjg\") pod \"maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w\" (UID: \"c6b1ddd9-f87e-458f-9f38-e84b141883d1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w"
Nov 01 05:45:00.833727 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.833697    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w"
Nov 01 05:45:00.905318 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:45:00.905285    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4b34f80_4eb0_4d2a_aee8_b2136ed54f83.slice/crio-c9dfd164eb4c34bd64344baccf7dcdf6c1fbb21188d6c4f64aa6a05e8c1899e0 WatchSource:0}: Error finding container c9dfd164eb4c34bd64344baccf7dcdf6c1fbb21188d6c4f64aa6a05e8c1899e0: Status 404 returned error can't find the container with id c9dfd164eb4c34bd64344baccf7dcdf6c1fbb21188d6c4f64aa6a05e8c1899e0
Nov 01 05:45:00.913192 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.912663    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5"]
Nov 01 05:45:00.947231 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:00.947204    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn"]
Nov 01 05:45:01.235657 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:01.235627    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w"]
Nov 01 05:45:01.236630 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:45:01.236600    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6b1ddd9_f87e_458f_9f38_e84b141883d1.slice/crio-5e7680f1f987b1e03dceeb5c51cca7d5ecf8a3e59613d87b91f78ef16abae80e WatchSource:0}: Error finding container 5e7680f1f987b1e03dceeb5c51cca7d5ecf8a3e59613d87b91f78ef16abae80e: Status 404 returned error can't find the container with id 5e7680f1f987b1e03dceeb5c51cca7d5ecf8a3e59613d87b91f78ef16abae80e
Nov 01 05:45:01.566519 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:01.566484    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w" event={"ID":"c6b1ddd9-f87e-458f-9f38-e84b141883d1","Type":"ContainerStarted","Data":"5e7680f1f987b1e03dceeb5c51cca7d5ecf8a3e59613d87b91f78ef16abae80e"}
Nov 01 05:45:01.568114 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:01.568089    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn" event={"ID":"c8a9f52e-e39d-4a35-930b-23d1521646d1","Type":"ContainerStarted","Data":"52ffb8b44e8ff9537facc814d4dbb798e4c599a4d6988d38b9a67d70c723c976"}
Nov 01 05:45:01.568228 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:01.568153    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn" event={"ID":"c8a9f52e-e39d-4a35-930b-23d1521646d1","Type":"ContainerStarted","Data":"73da6b01cb28fade1c22422049cb75acc56322b4687e4afa891f942f0603cfd1"}
Nov 01 05:45:01.570302 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:01.570264    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5" event={"ID":"c4b34f80-4eb0-4d2a-aee8-b2136ed54f83","Type":"ContainerStarted","Data":"0b59e7fad1a007874bbad8cbcf9050dc832b7f94f507a29a1b70de8abdbebd7a"}
Nov 01 05:45:01.570406 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:01.570320    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5" event={"ID":"c4b34f80-4eb0-4d2a-aee8-b2136ed54f83","Type":"ContainerStarted","Data":"c9dfd164eb4c34bd64344baccf7dcdf6c1fbb21188d6c4f64aa6a05e8c1899e0"}
Nov 01 05:45:01.683937 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:01.683883    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn" podStartSLOduration=1.683866614 podStartE2EDuration="1.683866614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:45:01.68086187 +0000 UTC m=+2518.571237874" watchObservedRunningTime="2026-11-01 05:45:01.683866614 +0000 UTC m=+2518.574242590"
Nov 01 05:45:01.797067 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:01.797012    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5" podStartSLOduration=1.7969961570000001 podStartE2EDuration="1.796996157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:45:01.794996404 +0000 UTC m=+2518.685372380" watchObservedRunningTime="2026-11-01 05:45:01.796996157 +0000 UTC m=+2518.687372132"
Nov 01 05:45:02.578178 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:02.578138    4864 generic.go:356] "Generic (PLEG): container finished" podID="c6b1ddd9-f87e-458f-9f38-e84b141883d1" containerID="0b21fc553ca2426deb940c7101c7ac2a523ec75b5d88d29ce8ac2bf84b0ff778" exitCode=60
Nov 01 05:45:02.578694 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:02.578230    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w" event={"ID":"c6b1ddd9-f87e-458f-9f38-e84b141883d1","Type":"ContainerDied","Data":"0b21fc553ca2426deb940c7101c7ac2a523ec75b5d88d29ce8ac2bf84b0ff778"}
Nov 01 05:45:02.578694 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:02.578511    4864 scope.go:122] "RemoveContainer" containerID="0b21fc553ca2426deb940c7101c7ac2a523ec75b5d88d29ce8ac2bf84b0ff778"
Nov 01 05:45:02.579938 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:02.579915    4864 generic.go:356] "Generic (PLEG): container finished" podID="c8a9f52e-e39d-4a35-930b-23d1521646d1" containerID="52ffb8b44e8ff9537facc814d4dbb798e4c599a4d6988d38b9a67d70c723c976" exitCode=60
Nov 01 05:45:02.580051 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:02.579987    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn" event={"ID":"c8a9f52e-e39d-4a35-930b-23d1521646d1","Type":"ContainerDied","Data":"52ffb8b44e8ff9537facc814d4dbb798e4c599a4d6988d38b9a67d70c723c976"}
Nov 01 05:45:02.580301 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:02.580264    4864 scope.go:122] "RemoveContainer" containerID="52ffb8b44e8ff9537facc814d4dbb798e4c599a4d6988d38b9a67d70c723c976"
Nov 01 05:45:02.582106 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:02.582084    4864 generic.go:356] "Generic (PLEG): container finished" podID="c4b34f80-4eb0-4d2a-aee8-b2136ed54f83" containerID="0b59e7fad1a007874bbad8cbcf9050dc832b7f94f507a29a1b70de8abdbebd7a" exitCode=60
Nov 01 05:45:02.582206 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:02.582183    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5" event={"ID":"c4b34f80-4eb0-4d2a-aee8-b2136ed54f83","Type":"ContainerDied","Data":"0b59e7fad1a007874bbad8cbcf9050dc832b7f94f507a29a1b70de8abdbebd7a"}
Nov 01 05:45:02.582316 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:02.582302    4864 scope.go:122] "RemoveContainer" containerID="0b59e7fad1a007874bbad8cbcf9050dc832b7f94f507a29a1b70de8abdbebd7a"
Nov 01 05:45:03.591321 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:03.591287    4864 generic.go:356] "Generic (PLEG): container finished" podID="c6b1ddd9-f87e-458f-9f38-e84b141883d1" containerID="dd619f6216b57a0a36ca8063447bca62207fd4ba0e34c704af8c427a835fee35" exitCode=60
Nov 01 05:45:03.591771 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:03.591362    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w" event={"ID":"c6b1ddd9-f87e-458f-9f38-e84b141883d1","Type":"ContainerDied","Data":"dd619f6216b57a0a36ca8063447bca62207fd4ba0e34c704af8c427a835fee35"}
Nov 01 05:45:03.591771 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:03.591430    4864 scope.go:122] "RemoveContainer" containerID="0b21fc553ca2426deb940c7101c7ac2a523ec75b5d88d29ce8ac2bf84b0ff778"
Nov 01 05:45:03.591771 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:03.591728    4864 scope.go:122] "RemoveContainer" containerID="dd619f6216b57a0a36ca8063447bca62207fd4ba0e34c704af8c427a835fee35"
Nov 01 05:45:03.592074 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:45:03.592052    4864 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w_odh-ai-gateway-infra(c6b1ddd9-f87e-458f-9f38-e84b141883d1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w" podUID="c6b1ddd9-f87e-458f-9f38-e84b141883d1"
Nov 01 05:45:03.593365 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:03.593335    4864 generic.go:356] "Generic (PLEG): container finished" podID="c8a9f52e-e39d-4a35-930b-23d1521646d1" containerID="b1f4182adcaf1739ecd01dff4f80de2897530fae304e30ccfec87d78e065e449" exitCode=60
Nov 01 05:45:03.593475 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:03.593411    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn" event={"ID":"c8a9f52e-e39d-4a35-930b-23d1521646d1","Type":"ContainerDied","Data":"b1f4182adcaf1739ecd01dff4f80de2897530fae304e30ccfec87d78e065e449"}
Nov 01 05:45:03.593684 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:03.593655    4864 scope.go:122] "RemoveContainer" containerID="b1f4182adcaf1739ecd01dff4f80de2897530fae304e30ccfec87d78e065e449"
Nov 01 05:45:03.594589 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:45:03.594554    4864 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn_odh-ai-gateway-infra(c8a9f52e-e39d-4a35-930b-23d1521646d1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn" podUID="c8a9f52e-e39d-4a35-930b-23d1521646d1"
Nov 01 05:45:03.595300 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:03.595278    4864 generic.go:356] "Generic (PLEG): container finished" podID="c4b34f80-4eb0-4d2a-aee8-b2136ed54f83" containerID="2b2677cedf559292e33b2295a83efa4d40788ce297daa9ebc6b3e265b45d9980" exitCode=60
Nov 01 05:45:03.595398 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:03.595352    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5" event={"ID":"c4b34f80-4eb0-4d2a-aee8-b2136ed54f83","Type":"ContainerDied","Data":"2b2677cedf559292e33b2295a83efa4d40788ce297daa9ebc6b3e265b45d9980"}
Nov 01 05:45:03.595601 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:03.595587    4864 scope.go:122] "RemoveContainer" containerID="2b2677cedf559292e33b2295a83efa4d40788ce297daa9ebc6b3e265b45d9980"
Nov 01 05:45:03.595836 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:45:03.595816    4864 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5_odh-ai-gateway-infra(c4b34f80-4eb0-4d2a-aee8-b2136ed54f83)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5" podUID="c4b34f80-4eb0-4d2a-aee8-b2136ed54f83"
Nov 01 05:45:03.737666 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:03.736532    4864 scope.go:122] "RemoveContainer" containerID="52ffb8b44e8ff9537facc814d4dbb798e4c599a4d6988d38b9a67d70c723c976"
Nov 01 05:45:03.751374 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:03.751352    4864 scope.go:122] "RemoveContainer" containerID="0b59e7fad1a007874bbad8cbcf9050dc832b7f94f507a29a1b70de8abdbebd7a"
Nov 01 05:45:04.603812 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:04.603745    4864 scope.go:122] "RemoveContainer" containerID="dd619f6216b57a0a36ca8063447bca62207fd4ba0e34c704af8c427a835fee35"
Nov 01 05:45:04.604321 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:45:04.604139    4864 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w_odh-ai-gateway-infra(c6b1ddd9-f87e-458f-9f38-e84b141883d1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w" podUID="c6b1ddd9-f87e-458f-9f38-e84b141883d1"
Nov 01 05:45:04.605523 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:04.605503    4864 scope.go:122] "RemoveContainer" containerID="b1f4182adcaf1739ecd01dff4f80de2897530fae304e30ccfec87d78e065e449"
Nov 01 05:45:04.605765 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:45:04.605739    4864 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn_odh-ai-gateway-infra(c8a9f52e-e39d-4a35-930b-23d1521646d1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn" podUID="c8a9f52e-e39d-4a35-930b-23d1521646d1"
Nov 01 05:45:04.607344 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:04.607327    4864 scope.go:122] "RemoveContainer" containerID="2b2677cedf559292e33b2295a83efa4d40788ce297daa9ebc6b3e265b45d9980"
Nov 01 05:45:04.607552 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:45:04.607533    4864 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5_odh-ai-gateway-infra(c4b34f80-4eb0-4d2a-aee8-b2136ed54f83)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5" podUID="c4b34f80-4eb0-4d2a-aee8-b2136ed54f83"
Nov 01 05:45:13.783008 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:13.782972    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"]
Nov 01 05:45:13.788143 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:13.788124    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"
Nov 01 05:45:13.840839 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:13.840771    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28fa7c01-92f6-475d-9362-c0d37edaf50e-metrics-certs\") pod \"maas-api-e2e-derive-31e728ff-6484dd8854-lfd92\" (UID: \"28fa7c01-92f6-475d-9362-c0d37edaf50e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"
Nov 01 05:45:13.841009 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:13.840910    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlrp7\" (UniqueName: \"kubernetes.io/projected/28fa7c01-92f6-475d-9362-c0d37edaf50e-kube-api-access-nlrp7\") pod \"maas-api-e2e-derive-31e728ff-6484dd8854-lfd92\" (UID: \"28fa7c01-92f6-475d-9362-c0d37edaf50e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"
Nov 01 05:45:13.841009 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:13.840961    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28fa7c01-92f6-475d-9362-c0d37edaf50e-maas-api-tls\") pod \"maas-api-e2e-derive-31e728ff-6484dd8854-lfd92\" (UID: \"28fa7c01-92f6-475d-9362-c0d37edaf50e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"
Nov 01 05:45:13.841165 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:13.841143    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28fa7c01-92f6-475d-9362-c0d37edaf50e-openshift-service-ca\") pod \"maas-api-e2e-derive-31e728ff-6484dd8854-lfd92\" (UID: \"28fa7c01-92f6-475d-9362-c0d37edaf50e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"
Nov 01 05:45:13.890007 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:13.889975    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-31e728ff-serving-cert\""
Nov 01 05:45:13.896411 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:13.896381    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"]
Nov 01 05:45:13.942621 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:13.942583    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28fa7c01-92f6-475d-9362-c0d37edaf50e-metrics-certs\") pod \"maas-api-e2e-derive-31e728ff-6484dd8854-lfd92\" (UID: \"28fa7c01-92f6-475d-9362-c0d37edaf50e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"
Nov 01 05:45:13.942800 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:13.942648    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlrp7\" (UniqueName: \"kubernetes.io/projected/28fa7c01-92f6-475d-9362-c0d37edaf50e-kube-api-access-nlrp7\") pod \"maas-api-e2e-derive-31e728ff-6484dd8854-lfd92\" (UID: \"28fa7c01-92f6-475d-9362-c0d37edaf50e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"
Nov 01 05:45:13.942800 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:13.942695    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28fa7c01-92f6-475d-9362-c0d37edaf50e-maas-api-tls\") pod \"maas-api-e2e-derive-31e728ff-6484dd8854-lfd92\" (UID: \"28fa7c01-92f6-475d-9362-c0d37edaf50e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"
Nov 01 05:45:13.942951 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:13.942827    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28fa7c01-92f6-475d-9362-c0d37edaf50e-openshift-service-ca\") pod \"maas-api-e2e-derive-31e728ff-6484dd8854-lfd92\" (UID: \"28fa7c01-92f6-475d-9362-c0d37edaf50e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"
Nov 01 05:45:13.943157 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:45:13.943132    4864 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-serving-cert: secret "maas-api-e2e-derive-31e728ff-serving-cert" not found
Nov 01 05:45:13.943239 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:45:13.943218    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28fa7c01-92f6-475d-9362-c0d37edaf50e-maas-api-tls podName:28fa7c01-92f6-475d-9362-c0d37edaf50e nodeName:}" failed. No retries permitted until 2026-11-01 05:45:14.443195579 +0000 UTC m=+2531.333571551 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/28fa7c01-92f6-475d-9362-c0d37edaf50e-maas-api-tls") pod "maas-api-e2e-derive-31e728ff-6484dd8854-lfd92" (UID: "28fa7c01-92f6-475d-9362-c0d37edaf50e") : secret "maas-api-e2e-derive-31e728ff-serving-cert" not found
Nov 01 05:45:13.943639 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:13.943608    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28fa7c01-92f6-475d-9362-c0d37edaf50e-openshift-service-ca\") pod \"maas-api-e2e-derive-31e728ff-6484dd8854-lfd92\" (UID: \"28fa7c01-92f6-475d-9362-c0d37edaf50e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"
Nov 01 05:45:13.952826 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:13.946630    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28fa7c01-92f6-475d-9362-c0d37edaf50e-metrics-certs\") pod \"maas-api-e2e-derive-31e728ff-6484dd8854-lfd92\" (UID: \"28fa7c01-92f6-475d-9362-c0d37edaf50e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"
Nov 01 05:45:14.006311 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:14.006275    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlrp7\" (UniqueName: \"kubernetes.io/projected/28fa7c01-92f6-475d-9362-c0d37edaf50e-kube-api-access-nlrp7\") pod \"maas-api-e2e-derive-31e728ff-6484dd8854-lfd92\" (UID: \"28fa7c01-92f6-475d-9362-c0d37edaf50e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"
Nov 01 05:45:14.240746 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:14.240717    4864 scope.go:122] "RemoveContainer" containerID="dd619f6216b57a0a36ca8063447bca62207fd4ba0e34c704af8c427a835fee35"
Nov 01 05:45:14.240936 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:14.240875    4864 scope.go:122] "RemoveContainer" containerID="2b2677cedf559292e33b2295a83efa4d40788ce297daa9ebc6b3e265b45d9980"
Nov 01 05:45:14.448858 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:14.448823    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28fa7c01-92f6-475d-9362-c0d37edaf50e-maas-api-tls\") pod \"maas-api-e2e-derive-31e728ff-6484dd8854-lfd92\" (UID: \"28fa7c01-92f6-475d-9362-c0d37edaf50e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"
Nov 01 05:45:14.458028 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:14.457993    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28fa7c01-92f6-475d-9362-c0d37edaf50e-maas-api-tls\") pod \"maas-api-e2e-derive-31e728ff-6484dd8854-lfd92\" (UID: \"28fa7c01-92f6-475d-9362-c0d37edaf50e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"
Nov 01 05:45:14.679684 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:14.679648    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w" event={"ID":"c6b1ddd9-f87e-458f-9f38-e84b141883d1","Type":"ContainerStarted","Data":"668a7d47907bdc5e162fd35fd3517e88d788fe3926bc562a064f1c4a5c8e23c9"}
Nov 01 05:45:14.681974 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:14.681935    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5" event={"ID":"c4b34f80-4eb0-4d2a-aee8-b2136ed54f83","Type":"ContainerStarted","Data":"2d80c06bf1263cbe1749b02034e81a6267f41c501c08770672cb5b050285f97c"}
Nov 01 05:45:14.704714 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:14.704687    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"
Nov 01 05:45:14.913406 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:14.913308    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w" podStartSLOduration=14.91329514 podStartE2EDuration="14.91329514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:45:14.799019341 +0000 UTC m=+2531.689395317" watchObservedRunningTime="2026-11-01 05:45:14.91329514 +0000 UTC m=+2531.803671116"
Nov 01 05:45:15.129848 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:15.129817    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"]
Nov 01 05:45:15.131075 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:45:15.131037    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28fa7c01_92f6_475d_9362_c0d37edaf50e.slice/crio-8f3b15d94089c7906a5dfde1a830da985238aa1eba4535f811379bac34b42639 WatchSource:0}: Error finding container 8f3b15d94089c7906a5dfde1a830da985238aa1eba4535f811379bac34b42639: Status 404 returned error can't find the container with id 8f3b15d94089c7906a5dfde1a830da985238aa1eba4535f811379bac34b42639
Nov 01 05:45:15.240922 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:15.240891    4864 scope.go:122] "RemoveContainer" containerID="b1f4182adcaf1739ecd01dff4f80de2897530fae304e30ccfec87d78e065e449"
Nov 01 05:45:15.690264 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:15.690233    4864 generic.go:356] "Generic (PLEG): container finished" podID="c6b1ddd9-f87e-458f-9f38-e84b141883d1" containerID="668a7d47907bdc5e162fd35fd3517e88d788fe3926bc562a064f1c4a5c8e23c9" exitCode=60
Nov 01 05:45:15.690398 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:15.690302    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w" event={"ID":"c6b1ddd9-f87e-458f-9f38-e84b141883d1","Type":"ContainerDied","Data":"668a7d47907bdc5e162fd35fd3517e88d788fe3926bc562a064f1c4a5c8e23c9"}
Nov 01 05:45:15.690398 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:15.690362    4864 scope.go:122] "RemoveContainer" containerID="dd619f6216b57a0a36ca8063447bca62207fd4ba0e34c704af8c427a835fee35"
Nov 01 05:45:15.690719 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:15.690701    4864 scope.go:122] "RemoveContainer" containerID="668a7d47907bdc5e162fd35fd3517e88d788fe3926bc562a064f1c4a5c8e23c9"
Nov 01 05:45:15.691022 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:45:15.691006    4864 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w_odh-ai-gateway-infra(c6b1ddd9-f87e-458f-9f38-e84b141883d1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w" podUID="c6b1ddd9-f87e-458f-9f38-e84b141883d1"
Nov 01 05:45:15.692351 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:15.692319    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn" event={"ID":"c8a9f52e-e39d-4a35-930b-23d1521646d1","Type":"ContainerStarted","Data":"edcecf49f74cbdb4d2ad0b4d07ee476efdb72774347601c1625ad88ffabd5c06"}
Nov 01 05:45:15.694095 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:15.694079    4864 generic.go:356] "Generic (PLEG): container finished" podID="c4b34f80-4eb0-4d2a-aee8-b2136ed54f83" containerID="2d80c06bf1263cbe1749b02034e81a6267f41c501c08770672cb5b050285f97c" exitCode=60
Nov 01 05:45:15.694193 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:15.694154    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5" event={"ID":"c4b34f80-4eb0-4d2a-aee8-b2136ed54f83","Type":"ContainerDied","Data":"2d80c06bf1263cbe1749b02034e81a6267f41c501c08770672cb5b050285f97c"}
Nov 01 05:45:15.694475 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:15.694455    4864 scope.go:122] "RemoveContainer" containerID="2d80c06bf1263cbe1749b02034e81a6267f41c501c08770672cb5b050285f97c"
Nov 01 05:45:15.694742 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:45:15.694722    4864 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5_odh-ai-gateway-infra(c4b34f80-4eb0-4d2a-aee8-b2136ed54f83)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5" podUID="c4b34f80-4eb0-4d2a-aee8-b2136ed54f83"
Nov 01 05:45:15.695456 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:15.695437    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92" event={"ID":"28fa7c01-92f6-475d-9362-c0d37edaf50e","Type":"ContainerStarted","Data":"8f3b15d94089c7906a5dfde1a830da985238aa1eba4535f811379bac34b42639"}
Nov 01 05:45:15.702778 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:15.702761    4864 scope.go:122] "RemoveContainer" containerID="2b2677cedf559292e33b2295a83efa4d40788ce297daa9ebc6b3e265b45d9980"
Nov 01 05:45:15.807299 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:15.807276    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w"]
Nov 01 05:45:15.923616 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:15.923578    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5"]
Nov 01 05:45:16.705481 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:16.705455    4864 generic.go:356] "Generic (PLEG): container finished" podID="c8a9f52e-e39d-4a35-930b-23d1521646d1" containerID="edcecf49f74cbdb4d2ad0b4d07ee476efdb72774347601c1625ad88ffabd5c06" exitCode=60
Nov 01 05:45:16.705631 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:16.705544    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn" event={"ID":"c8a9f52e-e39d-4a35-930b-23d1521646d1","Type":"ContainerDied","Data":"edcecf49f74cbdb4d2ad0b4d07ee476efdb72774347601c1625ad88ffabd5c06"}
Nov 01 05:45:16.705631 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:16.705618    4864 scope.go:122] "RemoveContainer" containerID="b1f4182adcaf1739ecd01dff4f80de2897530fae304e30ccfec87d78e065e449"
Nov 01 05:45:16.706079 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:16.706058    4864 scope.go:122] "RemoveContainer" containerID="edcecf49f74cbdb4d2ad0b4d07ee476efdb72774347601c1625ad88ffabd5c06"
Nov 01 05:45:16.706427 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:45:16.706400    4864 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn_odh-ai-gateway-infra(c8a9f52e-e39d-4a35-930b-23d1521646d1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn" podUID="c8a9f52e-e39d-4a35-930b-23d1521646d1"
Nov 01 05:45:16.709460 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:16.709439    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92" event={"ID":"28fa7c01-92f6-475d-9362-c0d37edaf50e","Type":"ContainerStarted","Data":"5baa74eda5f0271d7c7016efb2f0f118e482c448d16e321451e849c075f8c6c2"}
Nov 01 05:45:16.709598 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:16.709579    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"
Nov 01 05:45:16.936872 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:16.936822    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92" podStartSLOduration=3.00688435 podStartE2EDuration="3.936805756s" totalImagesPullingTime="929.921406ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:45:16.935945674 +0000 UTC m=+2533.826321681" watchObservedRunningTime="2026-11-01 05:45:16.936805756 +0000 UTC m=+2533.827181729"
Nov 01 05:45:16.941893 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:16.941818    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w"
Nov 01 05:45:16.949849 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:16.949828    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5"
Nov 01 05:45:16.978347 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:16.978258    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6b1ddd9-f87e-458f-9f38-e84b141883d1-kube-api-access-rqtjg\" (UniqueName: \"kubernetes.io/projected/c6b1ddd9-f87e-458f-9f38-e84b141883d1-kube-api-access-rqtjg\") pod \"c6b1ddd9-f87e-458f-9f38-e84b141883d1\" (UID: \"c6b1ddd9-f87e-458f-9f38-e84b141883d1\") "
Nov 01 05:45:16.978347 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:16.978339    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4b34f80-4eb0-4d2a-aee8-b2136ed54f83-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4b34f80-4eb0-4d2a-aee8-b2136ed54f83-maas-api-service-ca\") pod \"c4b34f80-4eb0-4d2a-aee8-b2136ed54f83\" (UID: \"c4b34f80-4eb0-4d2a-aee8-b2136ed54f83\") "
Nov 01 05:45:16.978548 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:16.978400    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4b34f80-4eb0-4d2a-aee8-b2136ed54f83-kube-api-access-sppmd\" (UniqueName: \"kubernetes.io/projected/c4b34f80-4eb0-4d2a-aee8-b2136ed54f83-kube-api-access-sppmd\") pod \"c4b34f80-4eb0-4d2a-aee8-b2136ed54f83\" (UID: \"c4b34f80-4eb0-4d2a-aee8-b2136ed54f83\") "
Nov 01 05:45:16.978594 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:16.978543    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c6b1ddd9-f87e-458f-9f38-e84b141883d1-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6b1ddd9-f87e-458f-9f38-e84b141883d1-maas-api-service-ca\") pod \"c6b1ddd9-f87e-458f-9f38-e84b141883d1\" (UID: \"c6b1ddd9-f87e-458f-9f38-e84b141883d1\") "
Nov 01 05:45:16.978839 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:16.978810    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4b34f80-4eb0-4d2a-aee8-b2136ed54f83-maas-api-service-ca" pod "c4b34f80-4eb0-4d2a-aee8-b2136ed54f83" (UID: "c4b34f80-4eb0-4d2a-aee8-b2136ed54f83"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:45:16.978958 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:16.978918    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6b1ddd9-f87e-458f-9f38-e84b141883d1-maas-api-service-ca" pod "c6b1ddd9-f87e-458f-9f38-e84b141883d1" (UID: "c6b1ddd9-f87e-458f-9f38-e84b141883d1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:45:16.979025 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:16.979012    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4b34f80-4eb0-4d2a-aee8-b2136ed54f83-maas-api-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:45:16.979072 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:16.979025    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6b1ddd9-f87e-458f-9f38-e84b141883d1-maas-api-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:45:16.980831 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:16.980715    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6b1ddd9-f87e-458f-9f38-e84b141883d1-kube-api-access-rqtjg" pod "c6b1ddd9-f87e-458f-9f38-e84b141883d1" (UID: "c6b1ddd9-f87e-458f-9f38-e84b141883d1"). InnerVolumeSpecName "kube-api-access-rqtjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:45:16.981190 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:16.981159    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4b34f80-4eb0-4d2a-aee8-b2136ed54f83-kube-api-access-sppmd" pod "c4b34f80-4eb0-4d2a-aee8-b2136ed54f83" (UID: "c4b34f80-4eb0-4d2a-aee8-b2136ed54f83"). InnerVolumeSpecName "kube-api-access-sppmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:45:17.079997 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:17.079961    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqtjg\" (UniqueName: \"kubernetes.io/projected/c6b1ddd9-f87e-458f-9f38-e84b141883d1-kube-api-access-rqtjg\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:45:17.079997 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:17.079997    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sppmd\" (UniqueName: \"kubernetes.io/projected/c4b34f80-4eb0-4d2a-aee8-b2136ed54f83-kube-api-access-sppmd\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:45:17.083051 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:17.083019    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn"]
Nov 01 05:45:17.720546 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:17.720506    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w" event={"ID":"c6b1ddd9-f87e-458f-9f38-e84b141883d1","Type":"ContainerDied","Data":"5e7680f1f987b1e03dceeb5c51cca7d5ecf8a3e59613d87b91f78ef16abae80e"}
Nov 01 05:45:17.720761 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:17.720584    4864 scope.go:122] "RemoveContainer" containerID="668a7d47907bdc5e162fd35fd3517e88d788fe3926bc562a064f1c4a5c8e23c9"
Nov 01 05:45:17.720761 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:17.720528    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w"
Nov 01 05:45:17.725112 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:17.725080    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5" event={"ID":"c4b34f80-4eb0-4d2a-aee8-b2136ed54f83","Type":"ContainerDied","Data":"c9dfd164eb4c34bd64344baccf7dcdf6c1fbb21188d6c4f64aa6a05e8c1899e0"}
Nov 01 05:45:17.725210 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:17.725127    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5"
Nov 01 05:45:17.732821 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:17.732770    4864 scope.go:122] "RemoveContainer" containerID="2d80c06bf1263cbe1749b02034e81a6267f41c501c08770672cb5b050285f97c"
Nov 01 05:45:17.885260 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:17.885215    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w"]
Nov 01 05:45:17.887426 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:17.887403    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn"
Nov 01 05:45:17.888941 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:17.888914    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891865-gzt5w"]
Nov 01 05:45:17.990268 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:17.990169    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c8a9f52e-e39d-4a35-930b-23d1521646d1-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8a9f52e-e39d-4a35-930b-23d1521646d1-maas-api-service-ca\") pod \"c8a9f52e-e39d-4a35-930b-23d1521646d1\" (UID: \"c8a9f52e-e39d-4a35-930b-23d1521646d1\") "
Nov 01 05:45:17.990268 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:17.990213    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8a9f52e-e39d-4a35-930b-23d1521646d1-kube-api-access-cl7qs\" (UniqueName: \"kubernetes.io/projected/c8a9f52e-e39d-4a35-930b-23d1521646d1-kube-api-access-cl7qs\") pod \"c8a9f52e-e39d-4a35-930b-23d1521646d1\" (UID: \"c8a9f52e-e39d-4a35-930b-23d1521646d1\") "
Nov 01 05:45:17.990762 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:17.990521    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8a9f52e-e39d-4a35-930b-23d1521646d1-maas-api-service-ca" pod "c8a9f52e-e39d-4a35-930b-23d1521646d1" (UID: "c8a9f52e-e39d-4a35-930b-23d1521646d1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:45:17.993118 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:17.993085    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8a9f52e-e39d-4a35-930b-23d1521646d1-kube-api-access-cl7qs" pod "c8a9f52e-e39d-4a35-930b-23d1521646d1" (UID: "c8a9f52e-e39d-4a35-930b-23d1521646d1"). InnerVolumeSpecName "kube-api-access-cl7qs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:45:17.998938 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:17.998908    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5"]
Nov 01 05:45:18.002062 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:18.002038    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7d1ec3-29891865-r46l5"]
Nov 01 05:45:18.092044 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:18.092010    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8a9f52e-e39d-4a35-930b-23d1521646d1-maas-api-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:45:18.092044 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:18.092037    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cl7qs\" (UniqueName: \"kubernetes.io/projected/c8a9f52e-e39d-4a35-930b-23d1521646d1-kube-api-access-cl7qs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:45:18.733855 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:18.733815    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn" event={"ID":"c8a9f52e-e39d-4a35-930b-23d1521646d1","Type":"ContainerDied","Data":"73da6b01cb28fade1c22422049cb75acc56322b4687e4afa891f942f0603cfd1"}
Nov 01 05:45:18.734040 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:18.733882    4864 scope.go:122] "RemoveContainer" containerID="edcecf49f74cbdb4d2ad0b4d07ee476efdb72774347601c1625ad88ffabd5c06"
Nov 01 05:45:18.734040 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:18.733831    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn"
Nov 01 05:45:18.896885 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:18.896855    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn"]
Nov 01 05:45:18.901573 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:18.901545    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25565930-29891865-r4kkn"]
Nov 01 05:45:19.248085 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:19.248054    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c4b34f80-4eb0-4d2a-aee8-b2136ed54f83" path="/var/lib/kubelet/pods/c4b34f80-4eb0-4d2a-aee8-b2136ed54f83/volumes"
Nov 01 05:45:19.248481 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:19.248448    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c6b1ddd9-f87e-458f-9f38-e84b141883d1" path="/var/lib/kubelet/pods/c6b1ddd9-f87e-458f-9f38-e84b141883d1/volumes"
Nov 01 05:45:19.248825 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:19.248777    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c8a9f52e-e39d-4a35-930b-23d1521646d1" path="/var/lib/kubelet/pods/c8a9f52e-e39d-4a35-930b-23d1521646d1/volumes"
Nov 01 05:45:22.525249 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:45:22.525214    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:45:22.731917 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:22.731891    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"
Nov 01 05:45:29.666809 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:29.666182    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx" podUID="d94a1258-dbb1-44b5-bf57-1b44b66fe286" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 05:45:29.728814 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:29.726511    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c" podUID="61ef19b8-157e-4f82-b624-49971d51b8c7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 05:45:29.901808 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:29.900886    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8" podUID="071c5c98-548c-439b-b8f6-3b07c70df932" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 05:45:30.010857 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:30.010311    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk" podUID="b28a4792-6e4e-4f9b-80e0-737dc18ba934" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 05:45:30.159024 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:30.158959    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx" podUID="7b847150-a3e8-4210-af71-4f2ec0010c03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 05:45:35.933156 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:35.933122    4864 generic.go:356] "Generic (PLEG): container finished" podID="0e9c0e6d-f282-45d2-a1e3-8753edbbc355" containerID="8c4858fcd84c6b12145d47343f54e18f3d2b4a8049cad2edb4a7bd1f8d3005f9" exitCode=0
Nov 01 05:45:35.933619 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:35.933164    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698" event={"ID":"0e9c0e6d-f282-45d2-a1e3-8753edbbc355","Type":"ContainerDied","Data":"8c4858fcd84c6b12145d47343f54e18f3d2b4a8049cad2edb4a7bd1f8d3005f9"}
Nov 01 05:45:35.933619 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:35.933506    4864 scope.go:122] "RemoveContainer" containerID="8c4858fcd84c6b12145d47343f54e18f3d2b4a8049cad2edb4a7bd1f8d3005f9"
Nov 01 05:45:35.935231 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:35.935200    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:45:35.935355 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:35.935266    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:45:35.935355 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:35.935297    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:45:36.942162 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:36.942130    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698" event={"ID":"0e9c0e6d-f282-45d2-a1e3-8753edbbc355","Type":"ContainerStarted","Data":"94ed3ee3b267224c00a199e7258782b232d7dba8ca2ed541a7fafa480c8e4895"}
Nov 01 05:45:37.503243 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:37.503208    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:45:37.504426 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:37.504395    4864 patch_prober.go:29] interesting pod/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 01 05:45:37.504566 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:37.504454    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698" podUID="0e9c0e6d-f282-45d2-a1e3-8753edbbc355" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 01 05:45:38.503362 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:38.503327    4864 patch_prober.go:29] interesting pod/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 01 05:45:38.503770 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:38.503384    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698" podUID="0e9c0e6d-f282-45d2-a1e3-8753edbbc355" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 01 05:45:39.503876 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:39.503845    4864 patch_prober.go:29] interesting pod/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 01 05:45:39.504238 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:39.503896    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698" podUID="0e9c0e6d-f282-45d2-a1e3-8753edbbc355" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 01 05:45:40.503973 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:40.503933    4864 patch_prober.go:29] interesting pod/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 01 05:45:40.504472 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:40.504011    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698" podUID="0e9c0e6d-f282-45d2-a1e3-8753edbbc355" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 01 05:45:41.197838 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:41.197806    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:45:41.503320 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:41.503225    4864 patch_prober.go:29] interesting pod/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 01 05:45:41.503320 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:41.503289    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698" podUID="0e9c0e6d-f282-45d2-a1e3-8753edbbc355" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 01 05:45:42.508576 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:42.508491    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:45:42.509763 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:42.509744    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:45:52.524472 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:45:52.524435    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:45:55.647445 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:55.646980    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"]
Nov 01 05:45:55.649165 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:55.647306    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w" podUID="1b33a7a5-8d14-4281-8639-32a1ab752cf8" containerName="maas-api" containerID="cri-o://1156f732eb1fb344da493bdfc77be0884488a716ba8c5d4b1da5f5639c5c92a3" gracePeriod=30
Nov 01 05:45:55.925645 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:55.925622    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"
Nov 01 05:45:56.009187 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:56.009154    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b33a7a5-8d14-4281-8639-32a1ab752cf8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b33a7a5-8d14-4281-8639-32a1ab752cf8-metrics-certs\") pod \"1b33a7a5-8d14-4281-8639-32a1ab752cf8\" (UID: \"1b33a7a5-8d14-4281-8639-32a1ab752cf8\") "
Nov 01 05:45:56.009357 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:56.009217    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b33a7a5-8d14-4281-8639-32a1ab752cf8-kube-api-access-wvn5g\" (UniqueName: \"kubernetes.io/projected/1b33a7a5-8d14-4281-8639-32a1ab752cf8-kube-api-access-wvn5g\") pod \"1b33a7a5-8d14-4281-8639-32a1ab752cf8\" (UID: \"1b33a7a5-8d14-4281-8639-32a1ab752cf8\") "
Nov 01 05:45:56.009430 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:56.009411    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b33a7a5-8d14-4281-8639-32a1ab752cf8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b33a7a5-8d14-4281-8639-32a1ab752cf8-maas-api-tls\") pod \"1b33a7a5-8d14-4281-8639-32a1ab752cf8\" (UID: \"1b33a7a5-8d14-4281-8639-32a1ab752cf8\") "
Nov 01 05:45:56.009492 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:56.009466    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b33a7a5-8d14-4281-8639-32a1ab752cf8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b33a7a5-8d14-4281-8639-32a1ab752cf8-openshift-service-ca\") pod \"1b33a7a5-8d14-4281-8639-32a1ab752cf8\" (UID: \"1b33a7a5-8d14-4281-8639-32a1ab752cf8\") "
Nov 01 05:45:56.009866 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:56.009839    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b33a7a5-8d14-4281-8639-32a1ab752cf8-openshift-service-ca" pod "1b33a7a5-8d14-4281-8639-32a1ab752cf8" (UID: "1b33a7a5-8d14-4281-8639-32a1ab752cf8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:45:56.012038 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:56.012011    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b33a7a5-8d14-4281-8639-32a1ab752cf8-maas-api-tls" pod "1b33a7a5-8d14-4281-8639-32a1ab752cf8" (UID: "1b33a7a5-8d14-4281-8639-32a1ab752cf8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:45:56.012161 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:56.012096    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b33a7a5-8d14-4281-8639-32a1ab752cf8-metrics-certs" pod "1b33a7a5-8d14-4281-8639-32a1ab752cf8" (UID: "1b33a7a5-8d14-4281-8639-32a1ab752cf8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:45:56.016941 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:56.016900    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b33a7a5-8d14-4281-8639-32a1ab752cf8-kube-api-access-wvn5g" pod "1b33a7a5-8d14-4281-8639-32a1ab752cf8" (UID: "1b33a7a5-8d14-4281-8639-32a1ab752cf8"). InnerVolumeSpecName "kube-api-access-wvn5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:45:56.111423 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:56.111385    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b33a7a5-8d14-4281-8639-32a1ab752cf8-maas-api-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:45:56.111423 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:56.111417    4864 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b33a7a5-8d14-4281-8639-32a1ab752cf8-openshift-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:45:56.111423 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:56.111426    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b33a7a5-8d14-4281-8639-32a1ab752cf8-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:45:56.111659 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:56.111437    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvn5g\" (UniqueName: \"kubernetes.io/projected/1b33a7a5-8d14-4281-8639-32a1ab752cf8-kube-api-access-wvn5g\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:45:56.131508 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:56.131477    4864 generic.go:356] "Generic (PLEG): container finished" podID="1b33a7a5-8d14-4281-8639-32a1ab752cf8" containerID="1156f732eb1fb344da493bdfc77be0884488a716ba8c5d4b1da5f5639c5c92a3" exitCode=0
Nov 01 05:45:56.131626 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:56.131536    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"
Nov 01 05:45:56.131626 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:56.131567    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w" event={"ID":"1b33a7a5-8d14-4281-8639-32a1ab752cf8","Type":"ContainerDied","Data":"1156f732eb1fb344da493bdfc77be0884488a716ba8c5d4b1da5f5639c5c92a3"}
Nov 01 05:45:56.131626 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:56.131621    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w" event={"ID":"1b33a7a5-8d14-4281-8639-32a1ab752cf8","Type":"ContainerDied","Data":"a55cb867b899111c2d7f505ecb7fd053319d51ed5f97aac45095eecd9c4bb0a5"}
Nov 01 05:45:56.132152 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:56.131643    4864 scope.go:122] "RemoveContainer" containerID="1156f732eb1fb344da493bdfc77be0884488a716ba8c5d4b1da5f5639c5c92a3"
Nov 01 05:45:56.142532 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:56.142516    4864 scope.go:122] "RemoveContainer" containerID="1156f732eb1fb344da493bdfc77be0884488a716ba8c5d4b1da5f5639c5c92a3"
Nov 01 05:45:56.142822 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:45:56.142803    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1156f732eb1fb344da493bdfc77be0884488a716ba8c5d4b1da5f5639c5c92a3\": container with ID starting with 1156f732eb1fb344da493bdfc77be0884488a716ba8c5d4b1da5f5639c5c92a3 not found: ID does not exist" containerID="1156f732eb1fb344da493bdfc77be0884488a716ba8c5d4b1da5f5639c5c92a3"
Nov 01 05:45:56.142896 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:56.142832    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1156f732eb1fb344da493bdfc77be0884488a716ba8c5d4b1da5f5639c5c92a3"} err="failed to get container status \"1156f732eb1fb344da493bdfc77be0884488a716ba8c5d4b1da5f5639c5c92a3\": rpc error: code = NotFound desc = could not find container \"1156f732eb1fb344da493bdfc77be0884488a716ba8c5d4b1da5f5639c5c92a3\": container with ID starting with 1156f732eb1fb344da493bdfc77be0884488a716ba8c5d4b1da5f5639c5c92a3 not found: ID does not exist"
Nov 01 05:45:56.295054 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:56.294971    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"]
Nov 01 05:45:56.298528 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:56.298503    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-faa8ff-bd578db8f-9ld8w"]
Nov 01 05:45:57.247549 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:45:57.247518    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b33a7a5-8d14-4281-8639-32a1ab752cf8" path="/var/lib/kubelet/pods/1b33a7a5-8d14-4281-8639-32a1ab752cf8/volumes"
Nov 01 05:46:22.523547 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:46:22.523509    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:46:52.525180 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:46:52.525145    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:47:15.596803 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.596700    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7440-705aa1"]
Nov 01 05:47:15.598455 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.598427    4864 cpu_manager.go:414] "containerMap: removing container" podUID="c4b34f80-4eb0-4d2a-aee8-b2136ed54f83" containerName="cleanup"
Nov 01 05:47:15.598641 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.598611    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4b34f80-4eb0-4d2a-aee8-b2136ed54f83" containerName="cleanup"
Nov 01 05:47:15.598775 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.598649    4864 cpu_manager.go:414] "containerMap: removing container" podUID="c4b34f80-4eb0-4d2a-aee8-b2136ed54f83" containerName="cleanup"
Nov 01 05:47:15.598775 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.598659    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4b34f80-4eb0-4d2a-aee8-b2136ed54f83" containerName="cleanup"
Nov 01 05:47:15.598775 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.598694    4864 cpu_manager.go:414] "containerMap: removing container" podUID="c6b1ddd9-f87e-458f-9f38-e84b141883d1" containerName="cleanup"
Nov 01 05:47:15.598775 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.598701    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6b1ddd9-f87e-458f-9f38-e84b141883d1" containerName="cleanup"
Nov 01 05:47:15.598775 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.598712    4864 cpu_manager.go:414] "containerMap: removing container" podUID="c4b34f80-4eb0-4d2a-aee8-b2136ed54f83" containerName="cleanup"
Nov 01 05:47:15.598775 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.598720    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4b34f80-4eb0-4d2a-aee8-b2136ed54f83" containerName="cleanup"
Nov 01 05:47:15.598775 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.598745    4864 cpu_manager.go:414] "containerMap: removing container" podUID="c6b1ddd9-f87e-458f-9f38-e84b141883d1" containerName="cleanup"
Nov 01 05:47:15.598775 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.598759    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6b1ddd9-f87e-458f-9f38-e84b141883d1" containerName="cleanup"
Nov 01 05:47:15.598775 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.598775    4864 cpu_manager.go:414] "containerMap: removing container" podUID="c8a9f52e-e39d-4a35-930b-23d1521646d1" containerName="cleanup"
Nov 01 05:47:15.598775 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.598799    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8a9f52e-e39d-4a35-930b-23d1521646d1" containerName="cleanup"
Nov 01 05:47:15.599265 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.598839    4864 cpu_manager.go:414] "containerMap: removing container" podUID="c6b1ddd9-f87e-458f-9f38-e84b141883d1" containerName="cleanup"
Nov 01 05:47:15.599265 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.598846    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6b1ddd9-f87e-458f-9f38-e84b141883d1" containerName="cleanup"
Nov 01 05:47:15.599265 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.598867    4864 cpu_manager.go:414] "containerMap: removing container" podUID="c8a9f52e-e39d-4a35-930b-23d1521646d1" containerName="cleanup"
Nov 01 05:47:15.599265 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.598876    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8a9f52e-e39d-4a35-930b-23d1521646d1" containerName="cleanup"
Nov 01 05:47:15.599265 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.598887    4864 cpu_manager.go:414] "containerMap: removing container" podUID="c8a9f52e-e39d-4a35-930b-23d1521646d1" containerName="cleanup"
Nov 01 05:47:15.599265 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.598894    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8a9f52e-e39d-4a35-930b-23d1521646d1" containerName="cleanup"
Nov 01 05:47:15.599265 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.598949    4864 cpu_manager.go:414] "containerMap: removing container" podUID="1b33a7a5-8d14-4281-8639-32a1ab752cf8" containerName="maas-api"
Nov 01 05:47:15.599265 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.598956    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b33a7a5-8d14-4281-8639-32a1ab752cf8" containerName="maas-api"
Nov 01 05:47:15.599265 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.599183    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b33a7a5-8d14-4281-8639-32a1ab752cf8" containerName="maas-api"
Nov 01 05:47:15.599265 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.599203    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4b34f80-4eb0-4d2a-aee8-b2136ed54f83" containerName="cleanup"
Nov 01 05:47:15.599265 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.599212    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4b34f80-4eb0-4d2a-aee8-b2136ed54f83" containerName="cleanup"
Nov 01 05:47:15.599265 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.599222    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6b1ddd9-f87e-458f-9f38-e84b141883d1" containerName="cleanup"
Nov 01 05:47:15.599265 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.599238    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4b34f80-4eb0-4d2a-aee8-b2136ed54f83" containerName="cleanup"
Nov 01 05:47:15.599265 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.599247    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6b1ddd9-f87e-458f-9f38-e84b141883d1" containerName="cleanup"
Nov 01 05:47:15.599265 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.599263    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8a9f52e-e39d-4a35-930b-23d1521646d1" containerName="cleanup"
Nov 01 05:47:15.599945 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.599274    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6b1ddd9-f87e-458f-9f38-e84b141883d1" containerName="cleanup"
Nov 01 05:47:15.599945 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.599289    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8a9f52e-e39d-4a35-930b-23d1521646d1" containerName="cleanup"
Nov 01 05:47:15.599945 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.599300    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8a9f52e-e39d-4a35-930b-23d1521646d1" containerName="cleanup"
Nov 01 05:47:15.604080 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.604060    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7440-705aa1"
Nov 01 05:47:15.654798 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.654755    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7440-705aa1"]
Nov 01 05:47:15.671825 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.671770    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwdll\" (UniqueName: \"kubernetes.io/projected/c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c-kube-api-access-kwdll\") pod \"test-curl-7440-705aa1\" (UID: \"c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c\") " pod="opendatahub/test-curl-7440-705aa1"
Nov 01 05:47:15.703538 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.703506    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-wfzxk\""
Nov 01 05:47:15.773042 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.773009    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwdll\" (UniqueName: \"kubernetes.io/projected/c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c-kube-api-access-kwdll\") pod \"test-curl-7440-705aa1\" (UID: \"c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c\") " pod="opendatahub/test-curl-7440-705aa1"
Nov 01 05:47:15.833778 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.833750    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwdll\" (UniqueName: \"kubernetes.io/projected/c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c-kube-api-access-kwdll\") pod \"test-curl-7440-705aa1\" (UID: \"c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c\") " pod="opendatahub/test-curl-7440-705aa1"
Nov 01 05:47:15.919431 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:15.919363    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7440-705aa1"
Nov 01 05:47:16.296821 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:16.296772    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7440-705aa1"]
Nov 01 05:47:16.298714 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:47:16.298667    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc46bda97_a5fb_44e5_a15a_4d89ff8f3f7c.slice/crio-915412bab0eecc08a3db889fc0276f0b21edc8403e7c655611f598ce8723cdcb WatchSource:0}: Error finding container 915412bab0eecc08a3db889fc0276f0b21edc8403e7c655611f598ce8723cdcb: Status 404 returned error can't find the container with id 915412bab0eecc08a3db889fc0276f0b21edc8403e7c655611f598ce8723cdcb
Nov 01 05:47:16.711855 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:16.711819    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7440-705aa1" event={"ID":"c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c","Type":"ContainerStarted","Data":"39f4a53acd055396ea5dc9f8accd9fdbc356525330ae11e4277a94d90ffe3b31"}
Nov 01 05:47:16.712269 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:16.711869    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7440-705aa1" event={"ID":"c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c","Type":"ContainerStarted","Data":"915412bab0eecc08a3db889fc0276f0b21edc8403e7c655611f598ce8723cdcb"}
Nov 01 05:47:16.826365 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:16.826316    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7440-705aa1" podStartSLOduration=1.8263021259999999 podStartE2EDuration="1.826302126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:47:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:47:16.823983378 +0000 UTC m=+2653.714359365" watchObservedRunningTime="2026-11-01 05:47:16.826302126 +0000 UTC m=+2653.716678102"
Nov 01 05:47:18.328482 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:18.328448    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7440-705aa1"]
Nov 01 05:47:18.333275 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:18.333247    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7440-705aa1"]
Nov 01 05:47:18.502592 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:18.502550    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7440-3ab145"]
Nov 01 05:47:18.503363 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:18.503341    4864 cpu_manager.go:414] "containerMap: removing container" podUID="c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c" containerName="test-curl-7440-705aa1"
Nov 01 05:47:18.503472 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:18.503363    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c" containerName="test-curl-7440-705aa1"
Nov 01 05:47:18.503548 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:18.503496    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c" containerName="test-curl-7440-705aa1"
Nov 01 05:47:18.507617 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:18.507598    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7440-3ab145"
Nov 01 05:47:18.563304 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:18.563270    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7440-3ab145"]
Nov 01 05:47:18.606392 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:18.606312    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbfpt\" (UniqueName: \"kubernetes.io/projected/5f871f72-e78e-4efb-8a1f-e2a54967d474-kube-api-access-fbfpt\") pod \"test-curl-7440-3ab145\" (UID: \"5f871f72-e78e-4efb-8a1f-e2a54967d474\") " pod="opendatahub/test-curl-7440-3ab145"
Nov 01 05:47:18.708051 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:18.708015    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbfpt\" (UniqueName: \"kubernetes.io/projected/5f871f72-e78e-4efb-8a1f-e2a54967d474-kube-api-access-fbfpt\") pod \"test-curl-7440-3ab145\" (UID: \"5f871f72-e78e-4efb-8a1f-e2a54967d474\") " pod="opendatahub/test-curl-7440-3ab145"
Nov 01 05:47:18.725862 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:18.725819    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7440-705aa1" podUID="c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c" containerName="test-curl-7440-705aa1" containerID="cri-o://39f4a53acd055396ea5dc9f8accd9fdbc356525330ae11e4277a94d90ffe3b31" gracePeriod=2
Nov 01 05:47:18.769161 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:18.769134    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbfpt\" (UniqueName: \"kubernetes.io/projected/5f871f72-e78e-4efb-8a1f-e2a54967d474-kube-api-access-fbfpt\") pod \"test-curl-7440-3ab145\" (UID: \"5f871f72-e78e-4efb-8a1f-e2a54967d474\") " pod="opendatahub/test-curl-7440-3ab145"
Nov 01 05:47:18.776734 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:18.776702    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7440-705aa1\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c" pod="opendatahub/test-curl-7440-705aa1"
Nov 01 05:47:18.820496 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:18.820466    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7440-3ab145"
Nov 01 05:47:19.201833 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:19.201812    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7440-3ab145"]
Nov 01 05:47:19.204886 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:47:19.204856    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f871f72_e78e_4efb_8a1f_e2a54967d474.slice/crio-e735630704fd83d1b1ce3671ac800f73751a008bbac3ce51119087f4f71a364c WatchSource:0}: Error finding container e735630704fd83d1b1ce3671ac800f73751a008bbac3ce51119087f4f71a364c: Status 404 returned error can't find the container with id e735630704fd83d1b1ce3671ac800f73751a008bbac3ce51119087f4f71a364c
Nov 01 05:47:19.733558 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:19.733525    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7440-3ab145" event={"ID":"5f871f72-e78e-4efb-8a1f-e2a54967d474","Type":"ContainerStarted","Data":"2c5917f62107e30e7464a8403d5b3fb764f8d16aa20a4a38fe9ca2f9ca4c17d3"}
Nov 01 05:47:19.734000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:19.733570    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7440-3ab145" event={"ID":"5f871f72-e78e-4efb-8a1f-e2a54967d474","Type":"ContainerStarted","Data":"e735630704fd83d1b1ce3671ac800f73751a008bbac3ce51119087f4f71a364c"}
Nov 01 05:47:19.847368 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:19.847317    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7440-3ab145" podStartSLOduration=1.847302454 podStartE2EDuration="1.847302454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:47:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:47:19.84573332 +0000 UTC m=+2656.736109295" watchObservedRunningTime="2026-11-01 05:47:19.847302454 +0000 UTC m=+2656.737678430"
Nov 01 05:47:20.972117 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:20.972092    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7440-705aa1"
Nov 01 05:47:21.033287 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.033200    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c-kube-api-access-kwdll\" (UniqueName: \"kubernetes.io/projected/c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c-kube-api-access-kwdll\") pod \"c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c\" (UID: \"c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c\") "
Nov 01 05:47:21.035579 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.035545    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c-kube-api-access-kwdll" pod "c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c" (UID: "c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c"). InnerVolumeSpecName "kube-api-access-kwdll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:47:21.134676 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.134638    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwdll\" (UniqueName: \"kubernetes.io/projected/c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c-kube-api-access-kwdll\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:47:21.247962 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.247919    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c" path="/var/lib/kubelet/pods/c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c/volumes"
Nov 01 05:47:21.410105 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.410068    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7440-3ab145"]
Nov 01 05:47:21.415083 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.415044    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7440-3ab145"]
Nov 01 05:47:21.569362 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.569331    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7440-108a1d"]
Nov 01 05:47:21.569843 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.569830    4864 cpu_manager.go:414] "containerMap: removing container" podUID="5f871f72-e78e-4efb-8a1f-e2a54967d474" containerName="test-curl-7440-3ab145"
Nov 01 05:47:21.569908 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.569845    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f871f72-e78e-4efb-8a1f-e2a54967d474" containerName="test-curl-7440-3ab145"
Nov 01 05:47:21.569968 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.569957    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f871f72-e78e-4efb-8a1f-e2a54967d474" containerName="test-curl-7440-3ab145"
Nov 01 05:47:21.573337 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.573316    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7440-108a1d"
Nov 01 05:47:21.628176 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.628151    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7440-108a1d"]
Nov 01 05:47:21.640512 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.640486    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbbjn\" (UniqueName: \"kubernetes.io/projected/2308831a-1717-48ae-b6df-f819710d4425-kube-api-access-kbbjn\") pod \"test-curl-7440-108a1d\" (UID: \"2308831a-1717-48ae-b6df-f819710d4425\") " pod="opendatahub/test-curl-7440-108a1d"
Nov 01 05:47:21.741387 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.741309    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbbjn\" (UniqueName: \"kubernetes.io/projected/2308831a-1717-48ae-b6df-f819710d4425-kube-api-access-kbbjn\") pod \"test-curl-7440-108a1d\" (UID: \"2308831a-1717-48ae-b6df-f819710d4425\") " pod="opendatahub/test-curl-7440-108a1d"
Nov 01 05:47:21.748419 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.748392    4864 generic.go:356] "Generic (PLEG): container finished" podID="c46bda97-a5fb-44e5-a15a-4d89ff8f3f7c" containerID="39f4a53acd055396ea5dc9f8accd9fdbc356525330ae11e4277a94d90ffe3b31" exitCode=137
Nov 01 05:47:21.748546 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.748438    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7440-705aa1"
Nov 01 05:47:21.748546 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.748465    4864 scope.go:122] "RemoveContainer" containerID="39f4a53acd055396ea5dc9f8accd9fdbc356525330ae11e4277a94d90ffe3b31"
Nov 01 05:47:21.748714 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.748669    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7440-3ab145" podUID="5f871f72-e78e-4efb-8a1f-e2a54967d474" containerName="test-curl-7440-3ab145" containerID="cri-o://2c5917f62107e30e7464a8403d5b3fb764f8d16aa20a4a38fe9ca2f9ca4c17d3" gracePeriod=2
Nov 01 05:47:21.757865 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.757847    4864 scope.go:122] "RemoveContainer" containerID="39f4a53acd055396ea5dc9f8accd9fdbc356525330ae11e4277a94d90ffe3b31"
Nov 01 05:47:21.758188 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:21.758168    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39f4a53acd055396ea5dc9f8accd9fdbc356525330ae11e4277a94d90ffe3b31\": container with ID starting with 39f4a53acd055396ea5dc9f8accd9fdbc356525330ae11e4277a94d90ffe3b31 not found: ID does not exist" containerID="39f4a53acd055396ea5dc9f8accd9fdbc356525330ae11e4277a94d90ffe3b31"
Nov 01 05:47:21.758276 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.758196    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39f4a53acd055396ea5dc9f8accd9fdbc356525330ae11e4277a94d90ffe3b31"} err="failed to get container status \"39f4a53acd055396ea5dc9f8accd9fdbc356525330ae11e4277a94d90ffe3b31\": rpc error: code = NotFound desc = could not find container \"39f4a53acd055396ea5dc9f8accd9fdbc356525330ae11e4277a94d90ffe3b31\": container with ID starting with 39f4a53acd055396ea5dc9f8accd9fdbc356525330ae11e4277a94d90ffe3b31 not found: ID does not exist"
Nov 01 05:47:21.799502 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:21.799473    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7440-3ab145\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="5f871f72-e78e-4efb-8a1f-e2a54967d474" pod="opendatahub/test-curl-7440-3ab145"
Nov 01 05:47:21.802032 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.802016    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbbjn\" (UniqueName: \"kubernetes.io/projected/2308831a-1717-48ae-b6df-f819710d4425-kube-api-access-kbbjn\") pod \"test-curl-7440-108a1d\" (UID: \"2308831a-1717-48ae-b6df-f819710d4425\") " pod="opendatahub/test-curl-7440-108a1d"
Nov 01 05:47:21.850347 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:21.850317    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7440-3ab145\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="5f871f72-e78e-4efb-8a1f-e2a54967d474" pod="opendatahub/test-curl-7440-3ab145"
Nov 01 05:47:21.885363 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:21.885341    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7440-108a1d"
Nov 01 05:47:22.260098 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:22.260071    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7440-108a1d"]
Nov 01 05:47:22.261519 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:47:22.261496    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2308831a_1717_48ae_b6df_f819710d4425.slice/crio-9c6237845dbb11bd3a72f3fb0ad35f51091e29b86453c8fe29ca0352a2abd81a WatchSource:0}: Error finding container 9c6237845dbb11bd3a72f3fb0ad35f51091e29b86453c8fe29ca0352a2abd81a: Status 404 returned error can't find the container with id 9c6237845dbb11bd3a72f3fb0ad35f51091e29b86453c8fe29ca0352a2abd81a
Nov 01 05:47:22.524275 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:22.524189    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:47:22.669147 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:22.669118    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"]
Nov 01 05:47:22.669369 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:22.669325    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92" podUID="28fa7c01-92f6-475d-9362-c0d37edaf50e" containerName="maas-api" containerID="cri-o://5baa74eda5f0271d7c7016efb2f0f118e482c448d16e321451e849c075f8c6c2" gracePeriod=30
Nov 01 05:47:22.719996 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:22.719956    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7440-3ab145\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="5f871f72-e78e-4efb-8a1f-e2a54967d474" pod="opendatahub/test-curl-7440-3ab145"
Nov 01 05:47:22.726939 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:22.726474    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92" podUID="28fa7c01-92f6-475d-9362-c0d37edaf50e" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.101:8443/health\": dial tcp 10.132.0.101:8443: connect: connection refused"
Nov 01 05:47:22.758930 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:22.758842    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7440-108a1d" event={"ID":"2308831a-1717-48ae-b6df-f819710d4425","Type":"ContainerStarted","Data":"0c407828efdddf498d8502c7930315d93cd2b8d68fdb3671c324b39b30ca3019"}
Nov 01 05:47:22.758930 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:22.758894    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7440-108a1d" event={"ID":"2308831a-1717-48ae-b6df-f819710d4425","Type":"ContainerStarted","Data":"9c6237845dbb11bd3a72f3fb0ad35f51091e29b86453c8fe29ca0352a2abd81a"}
Nov 01 05:47:22.956568 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:22.956508    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7440-108a1d" podStartSLOduration=1.956487039 podStartE2EDuration="1.956487039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:47:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:47:22.95338347 +0000 UTC m=+2659.843759447" watchObservedRunningTime="2026-11-01 05:47:22.956487039 +0000 UTC m=+2659.846863013"
Nov 01 05:47:22.959487 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:22.959467    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"
Nov 01 05:47:23.004144 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:23.004104    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7440-3ab145\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="5f871f72-e78e-4efb-8a1f-e2a54967d474" pod="opendatahub/test-curl-7440-3ab145"
Nov 01 05:47:23.054183 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:23.054153    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/28fa7c01-92f6-475d-9362-c0d37edaf50e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/28fa7c01-92f6-475d-9362-c0d37edaf50e-metrics-certs\") pod \"28fa7c01-92f6-475d-9362-c0d37edaf50e\" (UID: \"28fa7c01-92f6-475d-9362-c0d37edaf50e\") "
Nov 01 05:47:23.054381 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:23.054210    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28fa7c01-92f6-475d-9362-c0d37edaf50e-kube-api-access-nlrp7\" (UniqueName: \"kubernetes.io/projected/28fa7c01-92f6-475d-9362-c0d37edaf50e-kube-api-access-nlrp7\") pod \"28fa7c01-92f6-475d-9362-c0d37edaf50e\" (UID: \"28fa7c01-92f6-475d-9362-c0d37edaf50e\") "
Nov 01 05:47:23.054381 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:23.054336    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/28fa7c01-92f6-475d-9362-c0d37edaf50e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28fa7c01-92f6-475d-9362-c0d37edaf50e-maas-api-tls\") pod \"28fa7c01-92f6-475d-9362-c0d37edaf50e\" (UID: \"28fa7c01-92f6-475d-9362-c0d37edaf50e\") "
Nov 01 05:47:23.054381 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:23.054362    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/28fa7c01-92f6-475d-9362-c0d37edaf50e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28fa7c01-92f6-475d-9362-c0d37edaf50e-openshift-service-ca\") pod \"28fa7c01-92f6-475d-9362-c0d37edaf50e\" (UID: \"28fa7c01-92f6-475d-9362-c0d37edaf50e\") "
Nov 01 05:47:23.054777 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:23.054743    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7440-3ab145\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="5f871f72-e78e-4efb-8a1f-e2a54967d474" pod="opendatahub/test-curl-7440-3ab145"
Nov 01 05:47:23.055113 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:23.055086    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28fa7c01-92f6-475d-9362-c0d37edaf50e-openshift-service-ca" pod "28fa7c01-92f6-475d-9362-c0d37edaf50e" (UID: "28fa7c01-92f6-475d-9362-c0d37edaf50e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:47:23.058049 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:23.058025    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28fa7c01-92f6-475d-9362-c0d37edaf50e-kube-api-access-nlrp7" pod "28fa7c01-92f6-475d-9362-c0d37edaf50e" (UID: "28fa7c01-92f6-475d-9362-c0d37edaf50e"). InnerVolumeSpecName "kube-api-access-nlrp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:47:23.058343 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:23.058317    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28fa7c01-92f6-475d-9362-c0d37edaf50e-maas-api-tls" pod "28fa7c01-92f6-475d-9362-c0d37edaf50e" (UID: "28fa7c01-92f6-475d-9362-c0d37edaf50e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:47:23.058983 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:23.058945    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28fa7c01-92f6-475d-9362-c0d37edaf50e-metrics-certs" pod "28fa7c01-92f6-475d-9362-c0d37edaf50e" (UID: "28fa7c01-92f6-475d-9362-c0d37edaf50e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:47:23.155330 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:23.155292    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28fa7c01-92f6-475d-9362-c0d37edaf50e-maas-api-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:47:23.155330 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:23.155323    4864 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28fa7c01-92f6-475d-9362-c0d37edaf50e-openshift-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:47:23.155330 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:23.155338    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28fa7c01-92f6-475d-9362-c0d37edaf50e-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:47:23.155565 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:23.155350    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlrp7\" (UniqueName: \"kubernetes.io/projected/28fa7c01-92f6-475d-9362-c0d37edaf50e-kube-api-access-nlrp7\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:47:23.294548 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:23.294512    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7440-3ab145\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="5f871f72-e78e-4efb-8a1f-e2a54967d474" pod="opendatahub/test-curl-7440-3ab145"
Nov 01 05:47:23.767506 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:23.767479    4864 generic.go:356] "Generic (PLEG): container finished" podID="28fa7c01-92f6-475d-9362-c0d37edaf50e" containerID="5baa74eda5f0271d7c7016efb2f0f118e482c448d16e321451e849c075f8c6c2" exitCode=0
Nov 01 05:47:23.767663 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:23.767526    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"
Nov 01 05:47:23.767663 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:23.767551    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92" event={"ID":"28fa7c01-92f6-475d-9362-c0d37edaf50e","Type":"ContainerDied","Data":"5baa74eda5f0271d7c7016efb2f0f118e482c448d16e321451e849c075f8c6c2"}
Nov 01 05:47:23.767663 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:23.767608    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92" event={"ID":"28fa7c01-92f6-475d-9362-c0d37edaf50e","Type":"ContainerDied","Data":"8f3b15d94089c7906a5dfde1a830da985238aa1eba4535f811379bac34b42639"}
Nov 01 05:47:23.767663 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:23.767629    4864 scope.go:122] "RemoveContainer" containerID="5baa74eda5f0271d7c7016efb2f0f118e482c448d16e321451e849c075f8c6c2"
Nov 01 05:47:23.823309 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:23.823238    4864 scope.go:122] "RemoveContainer" containerID="5baa74eda5f0271d7c7016efb2f0f118e482c448d16e321451e849c075f8c6c2"
Nov 01 05:47:23.823602 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:23.823573    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5baa74eda5f0271d7c7016efb2f0f118e482c448d16e321451e849c075f8c6c2\": container with ID starting with 5baa74eda5f0271d7c7016efb2f0f118e482c448d16e321451e849c075f8c6c2 not found: ID does not exist" containerID="5baa74eda5f0271d7c7016efb2f0f118e482c448d16e321451e849c075f8c6c2"
Nov 01 05:47:23.823702 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:23.823606    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5baa74eda5f0271d7c7016efb2f0f118e482c448d16e321451e849c075f8c6c2"} err="failed to get container status \"5baa74eda5f0271d7c7016efb2f0f118e482c448d16e321451e849c075f8c6c2\": rpc error: code = NotFound desc = could not find container \"5baa74eda5f0271d7c7016efb2f0f118e482c448d16e321451e849c075f8c6c2\": container with ID starting with 5baa74eda5f0271d7c7016efb2f0f118e482c448d16e321451e849c075f8c6c2 not found: ID does not exist"
Nov 01 05:47:23.935998 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:23.935969    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"]
Nov 01 05:47:23.992057 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:23.992021    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-31e728ff-6484dd8854-lfd92"]
Nov 01 05:47:24.002409 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.002387    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7440-3ab145"
Nov 01 05:47:24.065538 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.065509    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f871f72-e78e-4efb-8a1f-e2a54967d474-kube-api-access-fbfpt\" (UniqueName: \"kubernetes.io/projected/5f871f72-e78e-4efb-8a1f-e2a54967d474-kube-api-access-fbfpt\") pod \"5f871f72-e78e-4efb-8a1f-e2a54967d474\" (UID: \"5f871f72-e78e-4efb-8a1f-e2a54967d474\") "
Nov 01 05:47:24.067753 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.067723    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f871f72-e78e-4efb-8a1f-e2a54967d474-kube-api-access-fbfpt" pod "5f871f72-e78e-4efb-8a1f-e2a54967d474" (UID: "5f871f72-e78e-4efb-8a1f-e2a54967d474"). InnerVolumeSpecName "kube-api-access-fbfpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:47:24.167201 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.167173    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbfpt\" (UniqueName: \"kubernetes.io/projected/5f871f72-e78e-4efb-8a1f-e2a54967d474-kube-api-access-fbfpt\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:47:24.413363 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.413282    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7440-108a1d"]
Nov 01 05:47:24.418300 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.418270    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7440-108a1d"]
Nov 01 05:47:24.575556 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.575518    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7440-1d3127"]
Nov 01 05:47:24.576138 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.576122    4864 cpu_manager.go:414] "containerMap: removing container" podUID="2308831a-1717-48ae-b6df-f819710d4425" containerName="test-curl-7440-108a1d"
Nov 01 05:47:24.576195 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.576141    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2308831a-1717-48ae-b6df-f819710d4425" containerName="test-curl-7440-108a1d"
Nov 01 05:47:24.576195 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.576168    4864 cpu_manager.go:414] "containerMap: removing container" podUID="28fa7c01-92f6-475d-9362-c0d37edaf50e" containerName="maas-api"
Nov 01 05:47:24.576195 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.576173    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28fa7c01-92f6-475d-9362-c0d37edaf50e" containerName="maas-api"
Nov 01 05:47:24.576293 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.576260    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="2308831a-1717-48ae-b6df-f819710d4425" containerName="test-curl-7440-108a1d"
Nov 01 05:47:24.576293 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.576271    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="28fa7c01-92f6-475d-9362-c0d37edaf50e" containerName="maas-api"
Nov 01 05:47:24.579486 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.579472    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7440-1d3127"
Nov 01 05:47:24.634842 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.634819    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7440-1d3127"]
Nov 01 05:47:24.673173 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.673090    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwrht\" (UniqueName: \"kubernetes.io/projected/5f0cd2bb-459e-403c-9634-89026e8504bb-kube-api-access-vwrht\") pod \"test-curl-7440-1d3127\" (UID: \"5f0cd2bb-459e-403c-9634-89026e8504bb\") " pod="opendatahub/test-curl-7440-1d3127"
Nov 01 05:47:24.774087 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.774053    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwrht\" (UniqueName: \"kubernetes.io/projected/5f0cd2bb-459e-403c-9634-89026e8504bb-kube-api-access-vwrht\") pod \"test-curl-7440-1d3127\" (UID: \"5f0cd2bb-459e-403c-9634-89026e8504bb\") " pod="opendatahub/test-curl-7440-1d3127"
Nov 01 05:47:24.777905 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.777875    4864 generic.go:356] "Generic (PLEG): container finished" podID="5f871f72-e78e-4efb-8a1f-e2a54967d474" containerID="2c5917f62107e30e7464a8403d5b3fb764f8d16aa20a4a38fe9ca2f9ca4c17d3" exitCode=137
Nov 01 05:47:24.778051 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.777921    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7440-3ab145"
Nov 01 05:47:24.778051 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.777973    4864 scope.go:122] "RemoveContainer" containerID="2c5917f62107e30e7464a8403d5b3fb764f8d16aa20a4a38fe9ca2f9ca4c17d3"
Nov 01 05:47:24.778261 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.778222    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7440-108a1d" podUID="2308831a-1717-48ae-b6df-f819710d4425" containerName="test-curl-7440-108a1d" containerID="cri-o://0c407828efdddf498d8502c7930315d93cd2b8d68fdb3671c324b39b30ca3019" gracePeriod=2
Nov 01 05:47:24.790979 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.790952    4864 scope.go:122] "RemoveContainer" containerID="2c5917f62107e30e7464a8403d5b3fb764f8d16aa20a4a38fe9ca2f9ca4c17d3"
Nov 01 05:47:24.791288 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:24.791267    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c5917f62107e30e7464a8403d5b3fb764f8d16aa20a4a38fe9ca2f9ca4c17d3\": container with ID starting with 2c5917f62107e30e7464a8403d5b3fb764f8d16aa20a4a38fe9ca2f9ca4c17d3 not found: ID does not exist" containerID="2c5917f62107e30e7464a8403d5b3fb764f8d16aa20a4a38fe9ca2f9ca4c17d3"
Nov 01 05:47:24.791379 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.791296    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c5917f62107e30e7464a8403d5b3fb764f8d16aa20a4a38fe9ca2f9ca4c17d3"} err="failed to get container status \"2c5917f62107e30e7464a8403d5b3fb764f8d16aa20a4a38fe9ca2f9ca4c17d3\": rpc error: code = NotFound desc = could not find container \"2c5917f62107e30e7464a8403d5b3fb764f8d16aa20a4a38fe9ca2f9ca4c17d3\": container with ID starting with 2c5917f62107e30e7464a8403d5b3fb764f8d16aa20a4a38fe9ca2f9ca4c17d3 not found: ID does not exist"
Nov 01 05:47:24.828814 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:24.828759    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7440-108a1d\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="2308831a-1717-48ae-b6df-f819710d4425" pod="opendatahub/test-curl-7440-108a1d"
Nov 01 05:47:24.834614 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.834592    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwrht\" (UniqueName: \"kubernetes.io/projected/5f0cd2bb-459e-403c-9634-89026e8504bb-kube-api-access-vwrht\") pod \"test-curl-7440-1d3127\" (UID: \"5f0cd2bb-459e-403c-9634-89026e8504bb\") " pod="opendatahub/test-curl-7440-1d3127"
Nov 01 05:47:24.879421 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:24.879388    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7440-108a1d\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="2308831a-1717-48ae-b6df-f819710d4425" pod="opendatahub/test-curl-7440-108a1d"
Nov 01 05:47:24.896722 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.896698    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7440-1d3127"
Nov 01 05:47:24.914273 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.912271    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"]
Nov 01 05:47:24.914273 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:24.912626    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698" podUID="0e9c0e6d-f282-45d2-a1e3-8753edbbc355" containerName="istio-proxy" containerID="cri-o://94ed3ee3b267224c00a199e7258782b232d7dba8ca2ed541a7fafa480c8e4895" gracePeriod=30
Nov 01 05:47:24.930222 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:24.930005    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7440-3ab145\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="5f871f72-e78e-4efb-8a1f-e2a54967d474" pod="opendatahub/test-curl-7440-3ab145"
Nov 01 05:47:24.980855 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:24.980762    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7440-3ab145\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="5f871f72-e78e-4efb-8a1f-e2a54967d474" pod="opendatahub/test-curl-7440-3ab145"
Nov 01 05:47:25.203525 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:25.203437    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7440-108a1d\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="2308831a-1717-48ae-b6df-f819710d4425" pod="opendatahub/test-curl-7440-108a1d"
Nov 01 05:47:25.247884 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:25.247853    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28fa7c01-92f6-475d-9362-c0d37edaf50e" path="/var/lib/kubelet/pods/28fa7c01-92f6-475d-9362-c0d37edaf50e/volumes"
Nov 01 05:47:25.248224 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:25.248212    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5f871f72-e78e-4efb-8a1f-e2a54967d474" path="/var/lib/kubelet/pods/5f871f72-e78e-4efb-8a1f-e2a54967d474/volumes"
Nov 01 05:47:25.303706 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:25.303679    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7440-1d3127"]
Nov 01 05:47:25.305481 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:47:25.305450    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f0cd2bb_459e_403c_9634_89026e8504bb.slice/crio-8049fe1fbd4edf8115a602ae28967892085852e38f11f1e84d9d593ff90a7751 WatchSource:0}: Error finding container 8049fe1fbd4edf8115a602ae28967892085852e38f11f1e84d9d593ff90a7751: Status 404 returned error can't find the container with id 8049fe1fbd4edf8115a602ae28967892085852e38f11f1e84d9d593ff90a7751
Nov 01 05:47:25.786850 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:25.786810    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7440-1d3127" event={"ID":"5f0cd2bb-459e-403c-9634-89026e8504bb","Type":"ContainerStarted","Data":"f8bd9f7c7c7cbc843c0c7896971c3d2dcdff16bfd4cd3d774ab5d92645deb225"}
Nov 01 05:47:25.787300 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:25.786861    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7440-1d3127" event={"ID":"5f0cd2bb-459e-403c-9634-89026e8504bb","Type":"ContainerStarted","Data":"8049fe1fbd4edf8115a602ae28967892085852e38f11f1e84d9d593ff90a7751"}
Nov 01 05:47:25.900932 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:25.900883    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7440-1d3127" podStartSLOduration=1.900866321 podStartE2EDuration="1.900866321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:47:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:47:25.898935871 +0000 UTC m=+2662.789311850" watchObservedRunningTime="2026-11-01 05:47:25.900866321 +0000 UTC m=+2662.791242296"
Nov 01 05:47:27.050336 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:27.050274    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7440-108a1d"
Nov 01 05:47:27.103462 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:27.103425    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2308831a-1717-48ae-b6df-f819710d4425-kube-api-access-kbbjn\" (UniqueName: \"kubernetes.io/projected/2308831a-1717-48ae-b6df-f819710d4425-kube-api-access-kbbjn\") pod \"2308831a-1717-48ae-b6df-f819710d4425\" (UID: \"2308831a-1717-48ae-b6df-f819710d4425\") "
Nov 01 05:47:27.106846 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:27.106809    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2308831a-1717-48ae-b6df-f819710d4425-kube-api-access-kbbjn" pod "2308831a-1717-48ae-b6df-f819710d4425" (UID: "2308831a-1717-48ae-b6df-f819710d4425"). InnerVolumeSpecName "kube-api-access-kbbjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:47:27.205756 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:27.205723    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbbjn\" (UniqueName: \"kubernetes.io/projected/2308831a-1717-48ae-b6df-f819710d4425-kube-api-access-kbbjn\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:47:27.248201 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:27.248175    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2308831a-1717-48ae-b6df-f819710d4425" path="/var/lib/kubelet/pods/2308831a-1717-48ae-b6df-f819710d4425/volumes"
Nov 01 05:47:27.395207 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:27.395062    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7440-1d3127"]
Nov 01 05:47:27.397596 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:27.397541    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7440-1d3127"]
Nov 01 05:47:27.510922 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:27.510888    4864 patch_prober.go:29] interesting pod/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 05:47:27.511090 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:27.510950    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698" podUID="0e9c0e6d-f282-45d2-a1e3-8753edbbc355" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 05:47:27.806933 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:27.806891    4864 generic.go:356] "Generic (PLEG): container finished" podID="2308831a-1717-48ae-b6df-f819710d4425" containerID="0c407828efdddf498d8502c7930315d93cd2b8d68fdb3671c324b39b30ca3019" exitCode=137
Nov 01 05:47:27.807098 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:27.806956    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7440-108a1d"
Nov 01 05:47:27.807098 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:27.806983    4864 scope.go:122] "RemoveContainer" containerID="0c407828efdddf498d8502c7930315d93cd2b8d68fdb3671c324b39b30ca3019"
Nov 01 05:47:27.807234 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:27.807130    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7440-1d3127" podUID="5f0cd2bb-459e-403c-9634-89026e8504bb" containerName="test-curl-7440-1d3127" containerID="cri-o://f8bd9f7c7c7cbc843c0c7896971c3d2dcdff16bfd4cd3d774ab5d92645deb225" gracePeriod=2
Nov 01 05:47:27.816718 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:27.816698    4864 scope.go:122] "RemoveContainer" containerID="0c407828efdddf498d8502c7930315d93cd2b8d68fdb3671c324b39b30ca3019"
Nov 01 05:47:27.816992 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:27.816970    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c407828efdddf498d8502c7930315d93cd2b8d68fdb3671c324b39b30ca3019\": container with ID starting with 0c407828efdddf498d8502c7930315d93cd2b8d68fdb3671c324b39b30ca3019 not found: ID does not exist" containerID="0c407828efdddf498d8502c7930315d93cd2b8d68fdb3671c324b39b30ca3019"
Nov 01 05:47:27.817068 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:27.817003    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c407828efdddf498d8502c7930315d93cd2b8d68fdb3671c324b39b30ca3019"} err="failed to get container status \"0c407828efdddf498d8502c7930315d93cd2b8d68fdb3671c324b39b30ca3019\": rpc error: code = NotFound desc = could not find container \"0c407828efdddf498d8502c7930315d93cd2b8d68fdb3671c324b39b30ca3019\": container with ID starting with 0c407828efdddf498d8502c7930315d93cd2b8d68fdb3671c324b39b30ca3019 not found: ID does not exist"
Nov 01 05:47:27.857874 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:27.857836    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7440-1d3127\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="5f0cd2bb-459e-403c-9634-89026e8504bb" pod="opendatahub/test-curl-7440-1d3127"
Nov 01 05:47:27.908577 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:27.908544    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7440-1d3127\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="5f0cd2bb-459e-403c-9634-89026e8504bb" pod="opendatahub/test-curl-7440-1d3127"
Nov 01 05:47:30.070009 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.069776    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7440-1d3127"
Nov 01 05:47:30.136268 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.135320    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f0cd2bb-459e-403c-9634-89026e8504bb-kube-api-access-vwrht\" (UniqueName: \"kubernetes.io/projected/5f0cd2bb-459e-403c-9634-89026e8504bb-kube-api-access-vwrht\") pod \"5f0cd2bb-459e-403c-9634-89026e8504bb\" (UID: \"5f0cd2bb-459e-403c-9634-89026e8504bb\") "
Nov 01 05:47:30.138866 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.138819    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f0cd2bb-459e-403c-9634-89026e8504bb-kube-api-access-vwrht" pod "5f0cd2bb-459e-403c-9634-89026e8504bb" (UID: "5f0cd2bb-459e-403c-9634-89026e8504bb"). InnerVolumeSpecName "kube-api-access-vwrht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:47:30.188412 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.188389    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:47:30.237168 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.237137    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istiod-ca-cert\") pod \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") "
Nov 01 05:47:30.237321 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.237176    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-kube-api-access-7v2wk\" (UniqueName: \"kubernetes.io/projected/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-kube-api-access-7v2wk\") pod \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") "
Nov 01 05:47:30.237321 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.237211    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-credential-socket\") pod \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") "
Nov 01 05:47:30.237321 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.237283    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-envoy\") pod \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") "
Nov 01 05:47:30.237448 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.237345    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-workload-socket\") pod \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") "
Nov 01 05:47:30.237448 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.237375    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-podinfo\") pod \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") "
Nov 01 05:47:30.237448 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.237408    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-workload-certs\") pod \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") "
Nov 01 05:47:30.237602 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.237491    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-data\") pod \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") "
Nov 01 05:47:30.237602 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.237529    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-token\" (UniqueName: \"kubernetes.io/projected/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-token\") pod \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\" (UID: \"0e9c0e6d-f282-45d2-a1e3-8753edbbc355\") "
Nov 01 05:47:30.237709 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.237600    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-credential-socket" pod "0e9c0e6d-f282-45d2-a1e3-8753edbbc355" (UID: "0e9c0e6d-f282-45d2-a1e3-8753edbbc355"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:47:30.237709 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.237644    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istiod-ca-cert" pod "0e9c0e6d-f282-45d2-a1e3-8753edbbc355" (UID: "0e9c0e6d-f282-45d2-a1e3-8753edbbc355"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:47:30.237709 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.237668    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-workload-socket" pod "0e9c0e6d-f282-45d2-a1e3-8753edbbc355" (UID: "0e9c0e6d-f282-45d2-a1e3-8753edbbc355"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:47:30.237972 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.237942    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-data" pod "0e9c0e6d-f282-45d2-a1e3-8753edbbc355" (UID: "0e9c0e6d-f282-45d2-a1e3-8753edbbc355"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:47:30.237972 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.237958    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-workload-certs" pod "0e9c0e6d-f282-45d2-a1e3-8753edbbc355" (UID: "0e9c0e6d-f282-45d2-a1e3-8753edbbc355"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:47:30.238120 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.238008    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:47:30.238120 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.238028    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwrht\" (UniqueName: \"kubernetes.io/projected/5f0cd2bb-459e-403c-9634-89026e8504bb-kube-api-access-vwrht\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:47:30.238120 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.238043    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:47:30.238120 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.238056    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:47:30.240202 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.240169    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-podinfo" pod "0e9c0e6d-f282-45d2-a1e3-8753edbbc355" (UID: "0e9c0e6d-f282-45d2-a1e3-8753edbbc355"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 05:47:30.240314 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.240219    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-kube-api-access-7v2wk" pod "0e9c0e6d-f282-45d2-a1e3-8753edbbc355" (UID: "0e9c0e6d-f282-45d2-a1e3-8753edbbc355"). InnerVolumeSpecName "kube-api-access-7v2wk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:47:30.240390 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.240374    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-token" pod "0e9c0e6d-f282-45d2-a1e3-8753edbbc355" (UID: "0e9c0e6d-f282-45d2-a1e3-8753edbbc355"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:47:30.240432 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.240417    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-envoy" pod "0e9c0e6d-f282-45d2-a1e3-8753edbbc355" (UID: "0e9c0e6d-f282-45d2-a1e3-8753edbbc355"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:47:30.338975 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.338912    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:47:30.338975 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.338935    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:47:30.338975 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.338945    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:47:30.338975 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.338956    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:47:30.338975 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.338965    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:47:30.338975 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.338973    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7v2wk\" (UniqueName: \"kubernetes.io/projected/0e9c0e6d-f282-45d2-a1e3-8753edbbc355-kube-api-access-7v2wk\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:47:30.837054 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.837023    4864 generic.go:356] "Generic (PLEG): container finished" podID="0e9c0e6d-f282-45d2-a1e3-8753edbbc355" containerID="94ed3ee3b267224c00a199e7258782b232d7dba8ca2ed541a7fafa480c8e4895" exitCode=0
Nov 01 05:47:30.837185 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.837069    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"
Nov 01 05:47:30.837185 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.837095    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698" event={"ID":"0e9c0e6d-f282-45d2-a1e3-8753edbbc355","Type":"ContainerDied","Data":"94ed3ee3b267224c00a199e7258782b232d7dba8ca2ed541a7fafa480c8e4895"}
Nov 01 05:47:30.837185 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.837151    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698" event={"ID":"0e9c0e6d-f282-45d2-a1e3-8753edbbc355","Type":"ContainerDied","Data":"19fcac466997b21ec2c6c6718ef333f5a9e69a717872adc7c06d91b734418b3d"}
Nov 01 05:47:30.837185 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.837171    4864 scope.go:122] "RemoveContainer" containerID="94ed3ee3b267224c00a199e7258782b232d7dba8ca2ed541a7fafa480c8e4895"
Nov 01 05:47:30.838659 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.838633    4864 generic.go:356] "Generic (PLEG): container finished" podID="5f0cd2bb-459e-403c-9634-89026e8504bb" containerID="f8bd9f7c7c7cbc843c0c7896971c3d2dcdff16bfd4cd3d774ab5d92645deb225" exitCode=137
Nov 01 05:47:30.838749 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.838674    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7440-1d3127"
Nov 01 05:47:30.848450 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.848434    4864 scope.go:122] "RemoveContainer" containerID="8c4858fcd84c6b12145d47343f54e18f3d2b4a8049cad2edb4a7bd1f8d3005f9"
Nov 01 05:47:30.861029 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.860963    4864 scope.go:122] "RemoveContainer" containerID="94ed3ee3b267224c00a199e7258782b232d7dba8ca2ed541a7fafa480c8e4895"
Nov 01 05:47:30.861318 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:30.861295    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94ed3ee3b267224c00a199e7258782b232d7dba8ca2ed541a7fafa480c8e4895\": container with ID starting with 94ed3ee3b267224c00a199e7258782b232d7dba8ca2ed541a7fafa480c8e4895 not found: ID does not exist" containerID="94ed3ee3b267224c00a199e7258782b232d7dba8ca2ed541a7fafa480c8e4895"
Nov 01 05:47:30.861402 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.861331    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94ed3ee3b267224c00a199e7258782b232d7dba8ca2ed541a7fafa480c8e4895"} err="failed to get container status \"94ed3ee3b267224c00a199e7258782b232d7dba8ca2ed541a7fafa480c8e4895\": rpc error: code = NotFound desc = could not find container \"94ed3ee3b267224c00a199e7258782b232d7dba8ca2ed541a7fafa480c8e4895\": container with ID starting with 94ed3ee3b267224c00a199e7258782b232d7dba8ca2ed541a7fafa480c8e4895 not found: ID does not exist"
Nov 01 05:47:30.861402 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.861358    4864 scope.go:122] "RemoveContainer" containerID="8c4858fcd84c6b12145d47343f54e18f3d2b4a8049cad2edb4a7bd1f8d3005f9"
Nov 01 05:47:30.862458 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:30.861721    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c4858fcd84c6b12145d47343f54e18f3d2b4a8049cad2edb4a7bd1f8d3005f9\": container with ID starting with 8c4858fcd84c6b12145d47343f54e18f3d2b4a8049cad2edb4a7bd1f8d3005f9 not found: ID does not exist" containerID="8c4858fcd84c6b12145d47343f54e18f3d2b4a8049cad2edb4a7bd1f8d3005f9"
Nov 01 05:47:30.862458 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.861761    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c4858fcd84c6b12145d47343f54e18f3d2b4a8049cad2edb4a7bd1f8d3005f9"} err="failed to get container status \"8c4858fcd84c6b12145d47343f54e18f3d2b4a8049cad2edb4a7bd1f8d3005f9\": rpc error: code = NotFound desc = could not find container \"8c4858fcd84c6b12145d47343f54e18f3d2b4a8049cad2edb4a7bd1f8d3005f9\": container with ID starting with 8c4858fcd84c6b12145d47343f54e18f3d2b4a8049cad2edb4a7bd1f8d3005f9 not found: ID does not exist"
Nov 01 05:47:30.862458 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.861874    4864 scope.go:122] "RemoveContainer" containerID="f8bd9f7c7c7cbc843c0c7896971c3d2dcdff16bfd4cd3d774ab5d92645deb225"
Nov 01 05:47:30.874617 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.874595    4864 scope.go:122] "RemoveContainer" containerID="f8bd9f7c7c7cbc843c0c7896971c3d2dcdff16bfd4cd3d774ab5d92645deb225"
Nov 01 05:47:30.875000 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:30.874976    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8bd9f7c7c7cbc843c0c7896971c3d2dcdff16bfd4cd3d774ab5d92645deb225\": container with ID starting with f8bd9f7c7c7cbc843c0c7896971c3d2dcdff16bfd4cd3d774ab5d92645deb225 not found: ID does not exist" containerID="f8bd9f7c7c7cbc843c0c7896971c3d2dcdff16bfd4cd3d774ab5d92645deb225"
Nov 01 05:47:30.875082 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:30.875009    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8bd9f7c7c7cbc843c0c7896971c3d2dcdff16bfd4cd3d774ab5d92645deb225"} err="failed to get container status \"f8bd9f7c7c7cbc843c0c7896971c3d2dcdff16bfd4cd3d774ab5d92645deb225\": rpc error: code = NotFound desc = could not find container \"f8bd9f7c7c7cbc843c0c7896971c3d2dcdff16bfd4cd3d774ab5d92645deb225\": container with ID starting with f8bd9f7c7c7cbc843c0c7896971c3d2dcdff16bfd4cd3d774ab5d92645deb225 not found: ID does not exist"
Nov 01 05:47:31.002170 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:31.002138    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"]
Nov 01 05:47:31.051765 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:31.051728    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698"]
Nov 01 05:47:31.056051 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:31.056022    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7440-1d3127\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="5f0cd2bb-459e-403c-9634-89026e8504bb" pod="opendatahub/test-curl-7440-1d3127"
Nov 01 05:47:31.248817 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:31.248700    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e9c0e6d-f282-45d2-a1e3-8753edbbc355" path="/var/lib/kubelet/pods/0e9c0e6d-f282-45d2-a1e3-8753edbbc355/volumes"
Nov 01 05:47:31.249182 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:31.249155    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5f0cd2bb-459e-403c-9634-89026e8504bb" path="/var/lib/kubelet/pods/5f0cd2bb-459e-403c-9634-89026e8504bb/volumes"
Nov 01 05:47:34.211440 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:34.211399    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7446-a58b91"]
Nov 01 05:47:34.212037 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:34.212019    4864 cpu_manager.go:414] "containerMap: removing container" podUID="5f0cd2bb-459e-403c-9634-89026e8504bb" containerName="test-curl-7440-1d3127"
Nov 01 05:47:34.212108 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:34.212039    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f0cd2bb-459e-403c-9634-89026e8504bb" containerName="test-curl-7440-1d3127"
Nov 01 05:47:34.212108 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:34.212052    4864 cpu_manager.go:414] "containerMap: removing container" podUID="0e9c0e6d-f282-45d2-a1e3-8753edbbc355" containerName="istio-proxy"
Nov 01 05:47:34.212108 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:34.212057    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e9c0e6d-f282-45d2-a1e3-8753edbbc355" containerName="istio-proxy"
Nov 01 05:47:34.212202 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:34.212157    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e9c0e6d-f282-45d2-a1e3-8753edbbc355" containerName="istio-proxy"
Nov 01 05:47:34.212202 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:34.212166    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f0cd2bb-459e-403c-9634-89026e8504bb" containerName="test-curl-7440-1d3127"
Nov 01 05:47:34.212202 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:34.212172    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e9c0e6d-f282-45d2-a1e3-8753edbbc355" containerName="istio-proxy"
Nov 01 05:47:34.217228 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:34.217202    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7446-a58b91"
Nov 01 05:47:34.269192 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:34.269156    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7446-a58b91"]
Nov 01 05:47:34.284250 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:34.284217    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4rz4\" (UniqueName: \"kubernetes.io/projected/c03dc71f-0d5d-4713-8e95-c8befe9e28b0-kube-api-access-f4rz4\") pod \"test-curl-7446-a58b91\" (UID: \"c03dc71f-0d5d-4713-8e95-c8befe9e28b0\") " pod="opendatahub/test-curl-7446-a58b91"
Nov 01 05:47:34.315259 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:34.315230    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-wfzxk\""
Nov 01 05:47:34.385237 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:34.385203    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4rz4\" (UniqueName: \"kubernetes.io/projected/c03dc71f-0d5d-4713-8e95-c8befe9e28b0-kube-api-access-f4rz4\") pod \"test-curl-7446-a58b91\" (UID: \"c03dc71f-0d5d-4713-8e95-c8befe9e28b0\") " pod="opendatahub/test-curl-7446-a58b91"
Nov 01 05:47:34.446288 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:34.446256    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4rz4\" (UniqueName: \"kubernetes.io/projected/c03dc71f-0d5d-4713-8e95-c8befe9e28b0-kube-api-access-f4rz4\") pod \"test-curl-7446-a58b91\" (UID: \"c03dc71f-0d5d-4713-8e95-c8befe9e28b0\") " pod="opendatahub/test-curl-7446-a58b91"
Nov 01 05:47:34.531312 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:34.531229    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7446-a58b91"
Nov 01 05:47:34.906690 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:34.906662    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7446-a58b91"]
Nov 01 05:47:34.908185 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:47:34.908158    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc03dc71f_0d5d_4713_8e95_c8befe9e28b0.slice/crio-170c13f7bd1b6e0fa54a44417a39407c38e0e56d54dedc830315c021be531256 WatchSource:0}: Error finding container 170c13f7bd1b6e0fa54a44417a39407c38e0e56d54dedc830315c021be531256: Status 404 returned error can't find the container with id 170c13f7bd1b6e0fa54a44417a39407c38e0e56d54dedc830315c021be531256
Nov 01 05:47:35.877424 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:35.877388    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7446-a58b91" event={"ID":"c03dc71f-0d5d-4713-8e95-c8befe9e28b0","Type":"ContainerStarted","Data":"190b1e5535b7dd72f6fe14cd23044a31669e7bfafd62c6f2cdd7f2dbc5b641d2"}
Nov 01 05:47:35.877908 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:35.877436    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7446-a58b91" event={"ID":"c03dc71f-0d5d-4713-8e95-c8befe9e28b0","Type":"ContainerStarted","Data":"170c13f7bd1b6e0fa54a44417a39407c38e0e56d54dedc830315c021be531256"}
Nov 01 05:47:35.991963 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:35.991912    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7446-a58b91" podStartSLOduration=1.9918982330000001 podStartE2EDuration="1.991898233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:47:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:47:35.990219774 +0000 UTC m=+2672.880595750" watchObservedRunningTime="2026-11-01 05:47:35.991898233 +0000 UTC m=+2672.882274209"
Nov 01 05:47:37.426457 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:37.426425    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7446-a58b91"]
Nov 01 05:47:37.433051 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:37.433022    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7446-a58b91"]
Nov 01 05:47:37.890982 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:37.890918    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7446-a58b91" podUID="c03dc71f-0d5d-4713-8e95-c8befe9e28b0" containerName="test-curl-7446-a58b91" containerID="cri-o://190b1e5535b7dd72f6fe14cd23044a31669e7bfafd62c6f2cdd7f2dbc5b641d2" gracePeriod=2
Nov 01 05:47:37.941946 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:37.941910    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7446-a58b91\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="c03dc71f-0d5d-4713-8e95-c8befe9e28b0" pod="opendatahub/test-curl-7446-a58b91"
Nov 01 05:47:40.121060 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:40.121036    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7446-a58b91"
Nov 01 05:47:40.249114 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:40.249032    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c03dc71f-0d5d-4713-8e95-c8befe9e28b0-kube-api-access-f4rz4\" (UniqueName: \"kubernetes.io/projected/c03dc71f-0d5d-4713-8e95-c8befe9e28b0-kube-api-access-f4rz4\") pod \"c03dc71f-0d5d-4713-8e95-c8befe9e28b0\" (UID: \"c03dc71f-0d5d-4713-8e95-c8befe9e28b0\") "
Nov 01 05:47:40.251272 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:40.251250    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c03dc71f-0d5d-4713-8e95-c8befe9e28b0-kube-api-access-f4rz4" pod "c03dc71f-0d5d-4713-8e95-c8befe9e28b0" (UID: "c03dc71f-0d5d-4713-8e95-c8befe9e28b0"). InnerVolumeSpecName "kube-api-access-f4rz4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:47:40.350663 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:40.350626    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4rz4\" (UniqueName: \"kubernetes.io/projected/c03dc71f-0d5d-4713-8e95-c8befe9e28b0-kube-api-access-f4rz4\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:47:40.914755 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:40.914718    4864 generic.go:356] "Generic (PLEG): container finished" podID="c03dc71f-0d5d-4713-8e95-c8befe9e28b0" containerID="190b1e5535b7dd72f6fe14cd23044a31669e7bfafd62c6f2cdd7f2dbc5b641d2" exitCode=137
Nov 01 05:47:40.914952 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:40.914765    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7446-a58b91"
Nov 01 05:47:40.914952 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:40.914846    4864 scope.go:122] "RemoveContainer" containerID="190b1e5535b7dd72f6fe14cd23044a31669e7bfafd62c6f2cdd7f2dbc5b641d2"
Nov 01 05:47:40.925370 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:40.925356    4864 scope.go:122] "RemoveContainer" containerID="190b1e5535b7dd72f6fe14cd23044a31669e7bfafd62c6f2cdd7f2dbc5b641d2"
Nov 01 05:47:40.925641 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:40.925621    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"190b1e5535b7dd72f6fe14cd23044a31669e7bfafd62c6f2cdd7f2dbc5b641d2\": container with ID starting with 190b1e5535b7dd72f6fe14cd23044a31669e7bfafd62c6f2cdd7f2dbc5b641d2 not found: ID does not exist" containerID="190b1e5535b7dd72f6fe14cd23044a31669e7bfafd62c6f2cdd7f2dbc5b641d2"
Nov 01 05:47:40.925710 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:40.925653    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"190b1e5535b7dd72f6fe14cd23044a31669e7bfafd62c6f2cdd7f2dbc5b641d2"} err="failed to get container status \"190b1e5535b7dd72f6fe14cd23044a31669e7bfafd62c6f2cdd7f2dbc5b641d2\": rpc error: code = NotFound desc = could not find container \"190b1e5535b7dd72f6fe14cd23044a31669e7bfafd62c6f2cdd7f2dbc5b641d2\": container with ID starting with 190b1e5535b7dd72f6fe14cd23044a31669e7bfafd62c6f2cdd7f2dbc5b641d2 not found: ID does not exist"
Nov 01 05:47:40.974537 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:40.974503    4864 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7446-a58b91\" is forbidden: User \"system:node:ip-10-0-134-253.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-253.us-west-1.compute.internal' and this object" podUID="c03dc71f-0d5d-4713-8e95-c8befe9e28b0" pod="opendatahub/test-curl-7446-a58b91"
Nov 01 05:47:41.247671 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:41.247595    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c03dc71f-0d5d-4713-8e95-c8befe9e28b0" path="/var/lib/kubelet/pods/c03dc71f-0d5d-4713-8e95-c8befe9e28b0/volumes"
Nov 01 05:47:50.608567 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:50.608529    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"]
Nov 01 05:47:50.609238 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:50.609217    4864 cpu_manager.go:414] "containerMap: removing container" podUID="c03dc71f-0d5d-4713-8e95-c8befe9e28b0" containerName="test-curl-7446-a58b91"
Nov 01 05:47:50.609346 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:50.609240    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c03dc71f-0d5d-4713-8e95-c8befe9e28b0" containerName="test-curl-7446-a58b91"
Nov 01 05:47:50.609346 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:50.609258    4864 cpu_manager.go:414] "containerMap: removing container" podUID="0e9c0e6d-f282-45d2-a1e3-8753edbbc355" containerName="istio-proxy"
Nov 01 05:47:50.609346 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:50.609266    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e9c0e6d-f282-45d2-a1e3-8753edbbc355" containerName="istio-proxy"
Nov 01 05:47:50.609492 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:50.609407    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="c03dc71f-0d5d-4713-8e95-c8befe9e28b0" containerName="test-curl-7446-a58b91"
Nov 01 05:47:50.613779 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:50.613758    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"
Nov 01 05:47:50.695326 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:50.695288    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-openshift-service-ca\") pod \"maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t\" (UID: \"1073ee9b-bccc-4fb1-8f1e-653281dbdcae\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"
Nov 01 05:47:50.695481 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:50.695364    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-maas-api-tls\") pod \"maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t\" (UID: \"1073ee9b-bccc-4fb1-8f1e-653281dbdcae\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"
Nov 01 05:47:50.695481 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:50.695431    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kpbj\" (UniqueName: \"kubernetes.io/projected/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-kube-api-access-6kpbj\") pod \"maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t\" (UID: \"1073ee9b-bccc-4fb1-8f1e-653281dbdcae\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"
Nov 01 05:47:50.695481 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:50.695474    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-metrics-certs\") pod \"maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t\" (UID: \"1073ee9b-bccc-4fb1-8f1e-653281dbdcae\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"
Nov 01 05:47:50.717211 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:50.717184    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-ad05c6-serving-cert\""
Nov 01 05:47:50.720176 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:50.720158    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"]
Nov 01 05:47:50.796618 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:50.796588    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-openshift-service-ca\") pod \"maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t\" (UID: \"1073ee9b-bccc-4fb1-8f1e-653281dbdcae\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"
Nov 01 05:47:50.796798 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:50.796675    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-maas-api-tls\") pod \"maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t\" (UID: \"1073ee9b-bccc-4fb1-8f1e-653281dbdcae\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"
Nov 01 05:47:50.796798 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:50.796708    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kpbj\" (UniqueName: \"kubernetes.io/projected/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-kube-api-access-6kpbj\") pod \"maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t\" (UID: \"1073ee9b-bccc-4fb1-8f1e-653281dbdcae\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"
Nov 01 05:47:50.796798 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:50.796744    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-metrics-certs\") pod \"maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t\" (UID: \"1073ee9b-bccc-4fb1-8f1e-653281dbdcae\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"
Nov 01 05:47:50.796989 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:50.796859    4864 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-serving-cert: secret "maas-api-e2e-autores-ad05c6-serving-cert" not found
Nov 01 05:47:50.796989 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:50.796943    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-maas-api-tls podName:1073ee9b-bccc-4fb1-8f1e-653281dbdcae nodeName:}" failed. No retries permitted until 2026-11-01 05:47:51.296921265 +0000 UTC m=+2688.187297219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-maas-api-tls") pod "maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t" (UID: "1073ee9b-bccc-4fb1-8f1e-653281dbdcae") : secret "maas-api-e2e-autores-ad05c6-serving-cert" not found
Nov 01 05:47:50.797399 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:50.797371    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-openshift-service-ca\") pod \"maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t\" (UID: \"1073ee9b-bccc-4fb1-8f1e-653281dbdcae\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"
Nov 01 05:47:50.799743 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:50.799723    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-metrics-certs\") pod \"maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t\" (UID: \"1073ee9b-bccc-4fb1-8f1e-653281dbdcae\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"
Nov 01 05:47:50.860842 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:50.860743    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kpbj\" (UniqueName: \"kubernetes.io/projected/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-kube-api-access-6kpbj\") pod \"maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t\" (UID: \"1073ee9b-bccc-4fb1-8f1e-653281dbdcae\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"
Nov 01 05:47:51.302612 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:51.302504    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-maas-api-tls\") pod \"maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t\" (UID: \"1073ee9b-bccc-4fb1-8f1e-653281dbdcae\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"
Nov 01 05:47:51.326574 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:51.321817    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-maas-api-tls\") pod \"maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t\" (UID: \"1073ee9b-bccc-4fb1-8f1e-653281dbdcae\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"
Nov 01 05:47:51.527017 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:51.526978    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"
Nov 01 05:47:51.913676 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:51.913648    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"]
Nov 01 05:47:51.914527 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:47:51.914499    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1073ee9b_bccc_4fb1_8f1e_653281dbdcae.slice/crio-1a8db81c09deb5ef754b80a1b6c173ef33e8b1aee7ccea98c2a30b9368c136c7 WatchSource:0}: Error finding container 1a8db81c09deb5ef754b80a1b6c173ef33e8b1aee7ccea98c2a30b9368c136c7: Status 404 returned error can't find the container with id 1a8db81c09deb5ef754b80a1b6c173ef33e8b1aee7ccea98c2a30b9368c136c7
Nov 01 05:47:51.915925 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:51.915905    4864 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 05:47:52.011994 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:52.011953    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t" event={"ID":"1073ee9b-bccc-4fb1-8f1e-653281dbdcae","Type":"ContainerStarted","Data":"1a8db81c09deb5ef754b80a1b6c173ef33e8b1aee7ccea98c2a30b9368c136c7"}
Nov 01 05:47:52.525414 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:47:52.525371    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:47:53.023903 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:53.023868    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t" event={"ID":"1073ee9b-bccc-4fb1-8f1e-653281dbdcae","Type":"ContainerStarted","Data":"947dd1d080188818e439040f6a9c475fc9127a118a40ec9a3c88b279a044f6c6"}
Nov 01 05:47:53.024400 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:53.024011    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"
Nov 01 05:47:53.142101 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:53.142048    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t" podStartSLOduration=2.307447001 podStartE2EDuration="3.142036058s" totalImagesPullingTime="834.589057ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:47:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:47:53.140212445 +0000 UTC m=+2690.030588423" watchObservedRunningTime="2026-11-01 05:47:53.142036058 +0000 UTC m=+2690.032412054"
Nov 01 05:47:59.042277 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:47:59.041288    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"
Nov 01 05:48:03.965925 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:03.965894    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:48:03.969187 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:03.969164    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:48:03.974701 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:03.974679    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:48:03.977364 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:03.977343    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:48:22.523661 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:48:22.523627    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:48:34.731072 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:34.731040    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"]
Nov 01 05:48:34.735125 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:34.735105    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"
Nov 01 05:48:34.787593 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:34.787567    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-c73135-serving-cert\""
Nov 01 05:48:34.840514 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:34.840489    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"]
Nov 01 05:48:34.869129 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:34.869097    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fmtg\" (UniqueName: \"kubernetes.io/projected/e3588066-acd0-433f-96c3-f4461690728f-kube-api-access-7fmtg\") pod \"maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk\" (UID: \"e3588066-acd0-433f-96c3-f4461690728f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"
Nov 01 05:48:34.869423 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:34.869392    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3588066-acd0-433f-96c3-f4461690728f-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk\" (UID: \"e3588066-acd0-433f-96c3-f4461690728f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"
Nov 01 05:48:34.869555 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:34.869487    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3588066-acd0-433f-96c3-f4461690728f-metrics-certs\") pod \"maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk\" (UID: \"e3588066-acd0-433f-96c3-f4461690728f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"
Nov 01 05:48:34.869555 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:34.869534    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3588066-acd0-433f-96c3-f4461690728f-maas-api-tls\") pod \"maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk\" (UID: \"e3588066-acd0-433f-96c3-f4461690728f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"
Nov 01 05:48:34.970442 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:34.970408    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3588066-acd0-433f-96c3-f4461690728f-metrics-certs\") pod \"maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk\" (UID: \"e3588066-acd0-433f-96c3-f4461690728f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"
Nov 01 05:48:34.970639 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:34.970451    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3588066-acd0-433f-96c3-f4461690728f-maas-api-tls\") pod \"maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk\" (UID: \"e3588066-acd0-433f-96c3-f4461690728f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"
Nov 01 05:48:34.970639 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:34.970518    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fmtg\" (UniqueName: \"kubernetes.io/projected/e3588066-acd0-433f-96c3-f4461690728f-kube-api-access-7fmtg\") pod \"maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk\" (UID: \"e3588066-acd0-433f-96c3-f4461690728f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"
Nov 01 05:48:34.970639 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:34.970608    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3588066-acd0-433f-96c3-f4461690728f-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk\" (UID: \"e3588066-acd0-433f-96c3-f4461690728f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"
Nov 01 05:48:34.971565 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:34.971535    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3588066-acd0-433f-96c3-f4461690728f-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk\" (UID: \"e3588066-acd0-433f-96c3-f4461690728f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"
Nov 01 05:48:34.973277 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:34.973250    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3588066-acd0-433f-96c3-f4461690728f-metrics-certs\") pod \"maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk\" (UID: \"e3588066-acd0-433f-96c3-f4461690728f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"
Nov 01 05:48:34.973372 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:34.973250    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3588066-acd0-433f-96c3-f4461690728f-maas-api-tls\") pod \"maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk\" (UID: \"e3588066-acd0-433f-96c3-f4461690728f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"
Nov 01 05:48:35.031834 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:35.031762    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fmtg\" (UniqueName: \"kubernetes.io/projected/e3588066-acd0-433f-96c3-f4461690728f-kube-api-access-7fmtg\") pod \"maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk\" (UID: \"e3588066-acd0-433f-96c3-f4461690728f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"
Nov 01 05:48:35.048546 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:35.048523    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"
Nov 01 05:48:35.437028 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:35.436998    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"]
Nov 01 05:48:35.441935 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:48:35.441909    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3588066_acd0_433f_96c3_f4461690728f.slice/crio-c5a0c0763a66d0d007361352e5ad1e9eb53825440fca94867489857459598f89 WatchSource:0}: Error finding container c5a0c0763a66d0d007361352e5ad1e9eb53825440fca94867489857459598f89: Status 404 returned error can't find the container with id c5a0c0763a66d0d007361352e5ad1e9eb53825440fca94867489857459598f89
Nov 01 05:48:36.421328 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:36.421287    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk" event={"ID":"e3588066-acd0-433f-96c3-f4461690728f","Type":"ContainerStarted","Data":"49cf6439743e5d40c4956e00e420bd0f6cdfbe27e1fb93c05b679a23d18a4fc9"}
Nov 01 05:48:36.421710 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:36.421350    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk" event={"ID":"e3588066-acd0-433f-96c3-f4461690728f","Type":"ContainerStarted","Data":"c5a0c0763a66d0d007361352e5ad1e9eb53825440fca94867489857459598f89"}
Nov 01 05:48:36.421710 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:36.421396    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"
Nov 01 05:48:36.541283 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:36.541193    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk" podStartSLOduration=1.7333017179999999 podStartE2EDuration="2.54117856s" totalImagesPullingTime="807.876842ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:48:36.538290127 +0000 UTC m=+2733.428666106" watchObservedRunningTime="2026-11-01 05:48:36.54117856 +0000 UTC m=+2733.431554535"
Nov 01 05:48:42.433374 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:48:42.433288    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"
Nov 01 05:48:52.613078 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:48:52.613023    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:49:02.832855 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:02.832811    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"]
Nov 01 05:49:02.837537 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:02.837516    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:02.903758 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:02.903715    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-ad05c6\"/\"openshift-service-ca.crt\""
Nov 01 05:49:02.908980 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:02.908947    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"]
Nov 01 05:49:02.912464 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:02.912440    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-dshm\") pod \"autores-model-ad05c6-kserve-65bf575d9-q5ljh\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") " pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:02.912601 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:02.912495    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-tmp-dir\") pod \"autores-model-ad05c6-kserve-65bf575d9-q5ljh\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") " pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:02.912675 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:02.912639    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2613874a-0440-43c5-bc69-84afdedc1c47-tls-certs\") pod \"autores-model-ad05c6-kserve-65bf575d9-q5ljh\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") " pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:02.912721 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:02.912673    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-model-cache\") pod \"autores-model-ad05c6-kserve-65bf575d9-q5ljh\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") " pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:02.912811 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:02.912768    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-home\") pod \"autores-model-ad05c6-kserve-65bf575d9-q5ljh\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") " pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:02.912880 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:02.912851    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cdrh\" (UniqueName: \"kubernetes.io/projected/2613874a-0440-43c5-bc69-84afdedc1c47-kube-api-access-9cdrh\") pod \"autores-model-ad05c6-kserve-65bf575d9-q5ljh\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") " pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:02.940212 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:02.940181    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-ad05c6\"/\"kube-root-ca.crt\""
Nov 01 05:49:02.940673 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:02.940653    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-ad05c6\"/\"default-dockercfg-ts464\""
Nov 01 05:49:02.940767 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:02.940654    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-ad05c6\"/\"autores-model-ad05c6-kserve-self-signed-certs\""
Nov 01 05:49:03.014329 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:03.014297    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-dshm\") pod \"autores-model-ad05c6-kserve-65bf575d9-q5ljh\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") " pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:03.014576 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:03.014344    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-tmp-dir\") pod \"autores-model-ad05c6-kserve-65bf575d9-q5ljh\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") " pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:03.014576 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:03.014379    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2613874a-0440-43c5-bc69-84afdedc1c47-tls-certs\") pod \"autores-model-ad05c6-kserve-65bf575d9-q5ljh\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") " pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:03.014576 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:03.014399    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-model-cache\") pod \"autores-model-ad05c6-kserve-65bf575d9-q5ljh\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") " pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:03.014576 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:03.014422    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-home\") pod \"autores-model-ad05c6-kserve-65bf575d9-q5ljh\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") " pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:03.014576 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:03.014445    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cdrh\" (UniqueName: \"kubernetes.io/projected/2613874a-0440-43c5-bc69-84afdedc1c47-kube-api-access-9cdrh\") pod \"autores-model-ad05c6-kserve-65bf575d9-q5ljh\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") " pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:03.015111 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:03.015083    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-tmp-dir\") pod \"autores-model-ad05c6-kserve-65bf575d9-q5ljh\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") " pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:03.015111 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:03.015100    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-home\") pod \"autores-model-ad05c6-kserve-65bf575d9-q5ljh\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") " pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:03.015331 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:03.015311    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-model-cache\") pod \"autores-model-ad05c6-kserve-65bf575d9-q5ljh\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") " pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:03.017340 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:03.017313    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2613874a-0440-43c5-bc69-84afdedc1c47-tls-certs\") pod \"autores-model-ad05c6-kserve-65bf575d9-q5ljh\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") " pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:03.018099 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:03.018079    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-dshm\") pod \"autores-model-ad05c6-kserve-65bf575d9-q5ljh\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") " pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:03.076430 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:03.076391    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cdrh\" (UniqueName: \"kubernetes.io/projected/2613874a-0440-43c5-bc69-84afdedc1c47-kube-api-access-9cdrh\") pod \"autores-model-ad05c6-kserve-65bf575d9-q5ljh\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") " pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:03.149938 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:03.149847    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:03.550288 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:03.550261    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"]
Nov 01 05:49:03.551745 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:49:03.551718    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2613874a_0440_43c5_bc69_84afdedc1c47.slice/crio-56d44b08ce43763998f5f370fd37f94fa941f4d3b2b10c3c348a4f8ce46c2aed WatchSource:0}: Error finding container 56d44b08ce43763998f5f370fd37f94fa941f4d3b2b10c3c348a4f8ce46c2aed: Status 404 returned error can't find the container with id 56d44b08ce43763998f5f370fd37f94fa941f4d3b2b10c3c348a4f8ce46c2aed
Nov 01 05:49:03.642116 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:03.642077    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh" event={"ID":"2613874a-0440-43c5-bc69-84afdedc1c47","Type":"ContainerStarted","Data":"7461297545de732256d2450721152cc79566bba036af94a1bd2dab97b7ad43b1"}
Nov 01 05:49:03.642470 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:03.642137    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh" event={"ID":"2613874a-0440-43c5-bc69-84afdedc1c47","Type":"ContainerStarted","Data":"56d44b08ce43763998f5f370fd37f94fa941f4d3b2b10c3c348a4f8ce46c2aed"}
Nov 01 05:49:03.760836 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:03.759024    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh" podStartSLOduration=1.759005019 podStartE2EDuration="1.759005019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:49:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:49:03.757825281 +0000 UTC m=+2760.648201251" watchObservedRunningTime="2026-11-01 05:49:03.759005019 +0000 UTC m=+2760.649380996"
Nov 01 05:49:04.150002 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:04.149969    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:13.150762 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:13.150725    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:13.163687 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:13.163660    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:13.730720 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:13.730679    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:49:22.526232 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:49:22.526187    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:49:47.914253 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:47.914217    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"]
Nov 01 05:49:47.914825 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:47.914446    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx" podUID="a148b0c7-891c-4eba-8f2e-f5dfeaaec96b" containerName="maas-api" containerID="cri-o://b48c2517cb57dde5d2f0817d80d07278b67de73a62d2fb036a01756c62b82c77" gracePeriod=30
Nov 01 05:49:48.181711 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:48.181644    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"
Nov 01 05:49:48.230958 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:48.230919    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-openshift-service-ca\") pod \"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b\" (UID: \"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b\") "
Nov 01 05:49:48.231740 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:48.231594    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-openshift-service-ca" pod "a148b0c7-891c-4eba-8f2e-f5dfeaaec96b" (UID: "a148b0c7-891c-4eba-8f2e-f5dfeaaec96b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:49:48.232923 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:48.232053    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-maas-api-tls\") pod \"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b\" (UID: \"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b\") "
Nov 01 05:49:48.232923 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:48.232109    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-metrics-certs\") pod \"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b\" (UID: \"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b\") "
Nov 01 05:49:48.232923 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:48.232158    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-kube-api-access-87mqb\" (UniqueName: \"kubernetes.io/projected/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-kube-api-access-87mqb\") pod \"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b\" (UID: \"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b\") "
Nov 01 05:49:48.232923 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:48.232641    4864 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-openshift-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:49:48.237028 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:48.236999    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-kube-api-access-87mqb" pod "a148b0c7-891c-4eba-8f2e-f5dfeaaec96b" (UID: "a148b0c7-891c-4eba-8f2e-f5dfeaaec96b"). InnerVolumeSpecName "kube-api-access-87mqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:49:48.238270 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:48.238229    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-metrics-certs" pod "a148b0c7-891c-4eba-8f2e-f5dfeaaec96b" (UID: "a148b0c7-891c-4eba-8f2e-f5dfeaaec96b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:49:48.249943 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:48.249899    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-maas-api-tls" pod "a148b0c7-891c-4eba-8f2e-f5dfeaaec96b" (UID: "a148b0c7-891c-4eba-8f2e-f5dfeaaec96b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:49:48.333330 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:48.333301    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-maas-api-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:49:48.333330 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:48.333329    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:49:48.333330 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:48.333344    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87mqb\" (UniqueName: \"kubernetes.io/projected/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b-kube-api-access-87mqb\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:49:48.989121 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:48.989082    4864 generic.go:356] "Generic (PLEG): container finished" podID="a148b0c7-891c-4eba-8f2e-f5dfeaaec96b" containerID="b48c2517cb57dde5d2f0817d80d07278b67de73a62d2fb036a01756c62b82c77" exitCode=0
Nov 01 05:49:48.989611 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:48.989144    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"
Nov 01 05:49:48.989611 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:48.989166    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx" event={"ID":"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b","Type":"ContainerDied","Data":"b48c2517cb57dde5d2f0817d80d07278b67de73a62d2fb036a01756c62b82c77"}
Nov 01 05:49:48.989611 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:48.989222    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx" event={"ID":"a148b0c7-891c-4eba-8f2e-f5dfeaaec96b","Type":"ContainerDied","Data":"d3a290e5d8e93245bc9fec51dbcc5f964961a243eaaf23ba0ba2e069fbab8546"}
Nov 01 05:49:48.989611 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:48.989247    4864 scope.go:122] "RemoveContainer" containerID="b48c2517cb57dde5d2f0817d80d07278b67de73a62d2fb036a01756c62b82c77"
Nov 01 05:49:49.000479 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:49.000464    4864 scope.go:122] "RemoveContainer" containerID="b48c2517cb57dde5d2f0817d80d07278b67de73a62d2fb036a01756c62b82c77"
Nov 01 05:49:49.000761 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:49:49.000734    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b48c2517cb57dde5d2f0817d80d07278b67de73a62d2fb036a01756c62b82c77\": container with ID starting with b48c2517cb57dde5d2f0817d80d07278b67de73a62d2fb036a01756c62b82c77 not found: ID does not exist" containerID="b48c2517cb57dde5d2f0817d80d07278b67de73a62d2fb036a01756c62b82c77"
Nov 01 05:49:49.000850 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:49.000773    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b48c2517cb57dde5d2f0817d80d07278b67de73a62d2fb036a01756c62b82c77"} err="failed to get container status \"b48c2517cb57dde5d2f0817d80d07278b67de73a62d2fb036a01756c62b82c77\": rpc error: code = NotFound desc = could not find container \"b48c2517cb57dde5d2f0817d80d07278b67de73a62d2fb036a01756c62b82c77\": container with ID starting with b48c2517cb57dde5d2f0817d80d07278b67de73a62d2fb036a01756c62b82c77 not found: ID does not exist"
Nov 01 05:49:49.153798 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:49.153761    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"]
Nov 01 05:49:49.202611 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:49.202582    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-6mnzx"]
Nov 01 05:49:49.249047 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:49.248983    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a148b0c7-891c-4eba-8f2e-f5dfeaaec96b" path="/var/lib/kubelet/pods/a148b0c7-891c-4eba-8f2e-f5dfeaaec96b/volumes"
Nov 01 05:49:52.541412 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:49:52.541335    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:49:59.797778 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:49:59.797746    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"]
Nov 01 05:50:14.818735 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:14.818640    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh" podUID="2613874a-0440-43c5-bc69-84afdedc1c47" containerName="main" containerID="cri-o://7461297545de732256d2450721152cc79566bba036af94a1bd2dab97b7ad43b1" gracePeriod=45
Nov 01 05:50:15.086909 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.086844    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:50:15.175861 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.175828    4864 generic.go:356] "Generic (PLEG): container finished" podID="2613874a-0440-43c5-bc69-84afdedc1c47" containerID="7461297545de732256d2450721152cc79566bba036af94a1bd2dab97b7ad43b1" exitCode=0
Nov 01 05:50:15.176043 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.175959    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"
Nov 01 05:50:15.176043 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.175990    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh" event={"ID":"2613874a-0440-43c5-bc69-84afdedc1c47","Type":"ContainerDied","Data":"7461297545de732256d2450721152cc79566bba036af94a1bd2dab97b7ad43b1"}
Nov 01 05:50:15.176043 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.176037    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh" event={"ID":"2613874a-0440-43c5-bc69-84afdedc1c47","Type":"ContainerDied","Data":"56d44b08ce43763998f5f370fd37f94fa941f4d3b2b10c3c348a4f8ce46c2aed"}
Nov 01 05:50:15.176157 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.176056    4864 scope.go:122] "RemoveContainer" containerID="7461297545de732256d2450721152cc79566bba036af94a1bd2dab97b7ad43b1"
Nov 01 05:50:15.185821 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.185766    4864 scope.go:122] "RemoveContainer" containerID="7461297545de732256d2450721152cc79566bba036af94a1bd2dab97b7ad43b1"
Nov 01 05:50:15.186131 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:50:15.186112    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7461297545de732256d2450721152cc79566bba036af94a1bd2dab97b7ad43b1\": container with ID starting with 7461297545de732256d2450721152cc79566bba036af94a1bd2dab97b7ad43b1 not found: ID does not exist" containerID="7461297545de732256d2450721152cc79566bba036af94a1bd2dab97b7ad43b1"
Nov 01 05:50:15.186232 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.186140    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7461297545de732256d2450721152cc79566bba036af94a1bd2dab97b7ad43b1"} err="failed to get container status \"7461297545de732256d2450721152cc79566bba036af94a1bd2dab97b7ad43b1\": rpc error: code = NotFound desc = could not find container \"7461297545de732256d2450721152cc79566bba036af94a1bd2dab97b7ad43b1\": container with ID starting with 7461297545de732256d2450721152cc79566bba036af94a1bd2dab97b7ad43b1 not found: ID does not exist"
Nov 01 05:50:15.230888 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.230849    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-home\" (UniqueName: \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-home\") pod \"2613874a-0440-43c5-bc69-84afdedc1c47\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") "
Nov 01 05:50:15.230888 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.230892    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-dshm\") pod \"2613874a-0440-43c5-bc69-84afdedc1c47\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") "
Nov 01 05:50:15.231146 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.231035    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2613874a-0440-43c5-bc69-84afdedc1c47-tls-certs\" (UniqueName: \"kubernetes.io/secret/2613874a-0440-43c5-bc69-84afdedc1c47-tls-certs\") pod \"2613874a-0440-43c5-bc69-84afdedc1c47\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") "
Nov 01 05:50:15.231146 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.231055    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-model-cache\") pod \"2613874a-0440-43c5-bc69-84afdedc1c47\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") "
Nov 01 05:50:15.231146 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.231077    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-tmp-dir\") pod \"2613874a-0440-43c5-bc69-84afdedc1c47\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") "
Nov 01 05:50:15.231146 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.231129    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2613874a-0440-43c5-bc69-84afdedc1c47-kube-api-access-9cdrh\" (UniqueName: \"kubernetes.io/projected/2613874a-0440-43c5-bc69-84afdedc1c47-kube-api-access-9cdrh\") pod \"2613874a-0440-43c5-bc69-84afdedc1c47\" (UID: \"2613874a-0440-43c5-bc69-84afdedc1c47\") "
Nov 01 05:50:15.231614 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.231522    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-model-cache" pod "2613874a-0440-43c5-bc69-84afdedc1c47" (UID: "2613874a-0440-43c5-bc69-84afdedc1c47"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:50:15.231614 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.231572    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-tmp-dir" pod "2613874a-0440-43c5-bc69-84afdedc1c47" (UID: "2613874a-0440-43c5-bc69-84afdedc1c47"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:50:15.231614 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.231584    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-home" pod "2613874a-0440-43c5-bc69-84afdedc1c47" (UID: "2613874a-0440-43c5-bc69-84afdedc1c47"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:50:15.233769 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.233743    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2613874a-0440-43c5-bc69-84afdedc1c47-kube-api-access-9cdrh" pod "2613874a-0440-43c5-bc69-84afdedc1c47" (UID: "2613874a-0440-43c5-bc69-84afdedc1c47"). InnerVolumeSpecName "kube-api-access-9cdrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:50:15.233985 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.233915    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-dshm" pod "2613874a-0440-43c5-bc69-84afdedc1c47" (UID: "2613874a-0440-43c5-bc69-84afdedc1c47"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:50:15.233985 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.233922    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2613874a-0440-43c5-bc69-84afdedc1c47-tls-certs" pod "2613874a-0440-43c5-bc69-84afdedc1c47" (UID: "2613874a-0440-43c5-bc69-84afdedc1c47"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:50:15.333026 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.332994    4864 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2613874a-0440-43c5-bc69-84afdedc1c47-tls-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:50:15.333026 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.333020    4864 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-model-cache\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:50:15.333026 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.333030    4864 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-tmp-dir\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:50:15.333317 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.333039    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cdrh\" (UniqueName: \"kubernetes.io/projected/2613874a-0440-43c5-bc69-84afdedc1c47-kube-api-access-9cdrh\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:50:15.333317 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.333050    4864 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-home\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:50:15.333317 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.333059    4864 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2613874a-0440-43c5-bc69-84afdedc1c47-dshm\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:50:15.592232 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.592198    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"]
Nov 01 05:50:15.598257 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:15.598222    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-ad05c6/autores-model-ad05c6-kserve-65bf575d9-q5ljh"]
Nov 01 05:50:17.247773 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:17.247737    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2613874a-0440-43c5-bc69-84afdedc1c47" path="/var/lib/kubelet/pods/2613874a-0440-43c5-bc69-84afdedc1c47/volumes"
Nov 01 05:50:22.524150 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:50:22.524112    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:50:48.235145 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:48.235107    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"]
Nov 01 05:50:48.237537 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:48.235585    4864 cpu_manager.go:414] "containerMap: removing container" podUID="a148b0c7-891c-4eba-8f2e-f5dfeaaec96b" containerName="maas-api"
Nov 01 05:50:48.237537 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:48.235596    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a148b0c7-891c-4eba-8f2e-f5dfeaaec96b" containerName="maas-api"
Nov 01 05:50:48.237537 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:48.235611    4864 cpu_manager.go:414] "containerMap: removing container" podUID="2613874a-0440-43c5-bc69-84afdedc1c47" containerName="main"
Nov 01 05:50:48.237537 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:48.235616    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2613874a-0440-43c5-bc69-84afdedc1c47" containerName="main"
Nov 01 05:50:48.237537 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:48.235710    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="2613874a-0440-43c5-bc69-84afdedc1c47" containerName="main"
Nov 01 05:50:48.237537 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:48.235724    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="a148b0c7-891c-4eba-8f2e-f5dfeaaec96b" containerName="maas-api"
Nov 01 05:50:48.238879 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:48.238859    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"
Nov 01 05:50:48.290746 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:48.290713    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-7a360a-serving-cert\""
Nov 01 05:50:48.343353 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:48.343319    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"]
Nov 01 05:50:48.393590 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:48.393552    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv\" (UID: \"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"
Nov 01 05:50:48.393829 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:48.393608    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-maas-api-tls\") pod \"maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv\" (UID: \"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"
Nov 01 05:50:48.393829 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:48.393635    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-metrics-certs\") pod \"maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv\" (UID: \"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"
Nov 01 05:50:48.393829 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:48.393766    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqkrc\" (UniqueName: \"kubernetes.io/projected/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-kube-api-access-hqkrc\") pod \"maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv\" (UID: \"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"
Nov 01 05:50:48.494548 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:48.494453    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqkrc\" (UniqueName: \"kubernetes.io/projected/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-kube-api-access-hqkrc\") pod \"maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv\" (UID: \"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"
Nov 01 05:50:48.494548 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:48.494527    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv\" (UID: \"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"
Nov 01 05:50:48.494868 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:48.494563    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-maas-api-tls\") pod \"maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv\" (UID: \"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"
Nov 01 05:50:48.494868 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:48.494603    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-metrics-certs\") pod \"maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv\" (UID: \"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"
Nov 01 05:50:48.495002 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:50:48.494887    4864 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-serving-cert: secret "maas-api-e2e-extmod-b-7a360a-serving-cert" not found
Nov 01 05:50:48.495002 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:50:48.494967    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-maas-api-tls podName:cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22 nodeName:}" failed. No retries permitted until 2026-11-01 05:50:48.994931804 +0000 UTC m=+2865.885307774 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-maas-api-tls") pod "maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv" (UID: "cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22") : secret "maas-api-e2e-extmod-b-7a360a-serving-cert" not found
Nov 01 05:50:48.495461 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:48.495439    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv\" (UID: \"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"
Nov 01 05:50:48.497607 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:48.497584    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-metrics-certs\") pod \"maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv\" (UID: \"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"
Nov 01 05:50:48.558269 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:48.558236    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqkrc\" (UniqueName: \"kubernetes.io/projected/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-kube-api-access-hqkrc\") pod \"maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv\" (UID: \"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"
Nov 01 05:50:49.000476 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:49.000435    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-maas-api-tls\") pod \"maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv\" (UID: \"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"
Nov 01 05:50:49.003231 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:49.003207    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-maas-api-tls\") pod \"maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv\" (UID: \"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"
Nov 01 05:50:49.153155 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:49.153112    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"
Nov 01 05:50:49.531112 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:49.531082    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"]
Nov 01 05:50:49.532693 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:50:49.532667    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf5e70e7_fe8f_46d9_b46a_cffe0c54ee22.slice/crio-de405e3228c74ca0fe30297b0b2b1e78a4609b6c9e29a6103978848442e2d9c3 WatchSource:0}: Error finding container de405e3228c74ca0fe30297b0b2b1e78a4609b6c9e29a6103978848442e2d9c3: Status 404 returned error can't find the container with id de405e3228c74ca0fe30297b0b2b1e78a4609b6c9e29a6103978848442e2d9c3
Nov 01 05:50:50.415195 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:50.415161    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv" event={"ID":"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22","Type":"ContainerStarted","Data":"de405e3228c74ca0fe30297b0b2b1e78a4609b6c9e29a6103978848442e2d9c3"}
Nov 01 05:50:51.423821 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:51.423745    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv" event={"ID":"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22","Type":"ContainerStarted","Data":"12a553e1f7c8deecd9f6082be33a9272b262b686059a9adcbedd09b4a8496838"}
Nov 01 05:50:51.424241 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:51.423841    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"
Nov 01 05:50:51.546071 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:51.546003    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv" podStartSLOduration=2.661152595 podStartE2EDuration="3.545985229s" totalImagesPullingTime="884.832634ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:50:51.541599673 +0000 UTC m=+2868.431975642" watchObservedRunningTime="2026-11-01 05:50:51.545985229 +0000 UTC m=+2868.436361206"
Nov 01 05:50:52.524602 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:50:52.524561    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:50:53.119860 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:53.119817    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"]
Nov 01 05:50:53.123828 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:53.123778    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"
Nov 01 05:50:53.175562 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:53.175528    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-25565930-serving-cert\""
Nov 01 05:50:53.228936 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:53.228895    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"]
Nov 01 05:50:53.250420 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:53.250384    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8022115-881a-40ff-bab0-cfead6161302-maas-api-tls\") pod \"maas-api-e2e-mt-25565930-847c47b46f-tnl66\" (UID: \"d8022115-881a-40ff-bab0-cfead6161302\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"
Nov 01 05:50:53.250610 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:53.250524    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8022115-881a-40ff-bab0-cfead6161302-openshift-service-ca\") pod \"maas-api-e2e-mt-25565930-847c47b46f-tnl66\" (UID: \"d8022115-881a-40ff-bab0-cfead6161302\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"
Nov 01 05:50:53.250610 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:53.250551    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h85wp\" (UniqueName: \"kubernetes.io/projected/d8022115-881a-40ff-bab0-cfead6161302-kube-api-access-h85wp\") pod \"maas-api-e2e-mt-25565930-847c47b46f-tnl66\" (UID: \"d8022115-881a-40ff-bab0-cfead6161302\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"
Nov 01 05:50:53.250610 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:53.250580    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8022115-881a-40ff-bab0-cfead6161302-metrics-certs\") pod \"maas-api-e2e-mt-25565930-847c47b46f-tnl66\" (UID: \"d8022115-881a-40ff-bab0-cfead6161302\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"
Nov 01 05:50:53.351754 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:53.351714    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8022115-881a-40ff-bab0-cfead6161302-maas-api-tls\") pod \"maas-api-e2e-mt-25565930-847c47b46f-tnl66\" (UID: \"d8022115-881a-40ff-bab0-cfead6161302\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"
Nov 01 05:50:53.351986 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:53.351849    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8022115-881a-40ff-bab0-cfead6161302-openshift-service-ca\") pod \"maas-api-e2e-mt-25565930-847c47b46f-tnl66\" (UID: \"d8022115-881a-40ff-bab0-cfead6161302\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"
Nov 01 05:50:53.351986 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:53.351870    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h85wp\" (UniqueName: \"kubernetes.io/projected/d8022115-881a-40ff-bab0-cfead6161302-kube-api-access-h85wp\") pod \"maas-api-e2e-mt-25565930-847c47b46f-tnl66\" (UID: \"d8022115-881a-40ff-bab0-cfead6161302\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"
Nov 01 05:50:53.351986 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:53.351910    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8022115-881a-40ff-bab0-cfead6161302-metrics-certs\") pod \"maas-api-e2e-mt-25565930-847c47b46f-tnl66\" (UID: \"d8022115-881a-40ff-bab0-cfead6161302\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"
Nov 01 05:50:53.351986 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:50:53.351870    4864 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-25565930-serving-cert: secret "maas-api-e2e-mt-25565930-serving-cert" not found
Nov 01 05:50:53.352229 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:50:53.352003    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8022115-881a-40ff-bab0-cfead6161302-maas-api-tls podName:d8022115-881a-40ff-bab0-cfead6161302 nodeName:}" failed. No retries permitted until 2026-11-01 05:50:53.851979908 +0000 UTC m=+2870.742355864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d8022115-881a-40ff-bab0-cfead6161302-maas-api-tls") pod "maas-api-e2e-mt-25565930-847c47b46f-tnl66" (UID: "d8022115-881a-40ff-bab0-cfead6161302") : secret "maas-api-e2e-mt-25565930-serving-cert" not found
Nov 01 05:50:53.352546 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:53.352526    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8022115-881a-40ff-bab0-cfead6161302-openshift-service-ca\") pod \"maas-api-e2e-mt-25565930-847c47b46f-tnl66\" (UID: \"d8022115-881a-40ff-bab0-cfead6161302\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"
Nov 01 05:50:53.354741 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:53.354720    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8022115-881a-40ff-bab0-cfead6161302-metrics-certs\") pod \"maas-api-e2e-mt-25565930-847c47b46f-tnl66\" (UID: \"d8022115-881a-40ff-bab0-cfead6161302\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"
Nov 01 05:50:53.413147 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:53.413077    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h85wp\" (UniqueName: \"kubernetes.io/projected/d8022115-881a-40ff-bab0-cfead6161302-kube-api-access-h85wp\") pod \"maas-api-e2e-mt-25565930-847c47b46f-tnl66\" (UID: \"d8022115-881a-40ff-bab0-cfead6161302\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"
Nov 01 05:50:53.857970 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:53.857933    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8022115-881a-40ff-bab0-cfead6161302-maas-api-tls\") pod \"maas-api-e2e-mt-25565930-847c47b46f-tnl66\" (UID: \"d8022115-881a-40ff-bab0-cfead6161302\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"
Nov 01 05:50:53.860624 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:53.860597    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8022115-881a-40ff-bab0-cfead6161302-maas-api-tls\") pod \"maas-api-e2e-mt-25565930-847c47b46f-tnl66\" (UID: \"d8022115-881a-40ff-bab0-cfead6161302\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"
Nov 01 05:50:54.036654 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:54.036613    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"
Nov 01 05:50:54.425132 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:54.425103    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"]
Nov 01 05:50:54.426967 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:50:54.426936    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8022115_881a_40ff_bab0_cfead6161302.slice/crio-c3aee56f6c987f8a3128f65ca539053e2506ca29cada62a419a723428e394900 WatchSource:0}: Error finding container c3aee56f6c987f8a3128f65ca539053e2506ca29cada62a419a723428e394900: Status 404 returned error can't find the container with id c3aee56f6c987f8a3128f65ca539053e2506ca29cada62a419a723428e394900
Nov 01 05:50:54.444262 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:54.444229    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66" event={"ID":"d8022115-881a-40ff-bab0-cfead6161302","Type":"ContainerStarted","Data":"c3aee56f6c987f8a3128f65ca539053e2506ca29cada62a419a723428e394900"}
Nov 01 05:50:55.451754 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:55.451719    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66" event={"ID":"d8022115-881a-40ff-bab0-cfead6161302","Type":"ContainerStarted","Data":"92ed49407e179380b9c9bed5b5cb82e8a96b68de08c0d64e9d51f4e77cfa5230"}
Nov 01 05:50:55.452275 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:55.451837    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"
Nov 01 05:50:55.569501 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:55.569445    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66" podStartSLOduration=1.742631477 podStartE2EDuration="2.569431378s" totalImagesPullingTime="826.799901ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:50:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:50:55.567307988 +0000 UTC m=+2872.457683965" watchObservedRunningTime="2026-11-01 05:50:55.569431378 +0000 UTC m=+2872.459807353"
Nov 01 05:50:57.435944 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:57.435909    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"
Nov 01 05:50:57.540763 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:57.540724    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"]
Nov 01 05:50:57.540950 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:57.540924    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t" podUID="1073ee9b-bccc-4fb1-8f1e-653281dbdcae" containerName="maas-api" containerID="cri-o://947dd1d080188818e439040f6a9c475fc9127a118a40ec9a3c88b279a044f6c6" gracePeriod=30
Nov 01 05:50:57.842933 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:57.842906    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"
Nov 01 05:50:57.897827 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:57.897778    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-maas-api-tls\") pod \"1073ee9b-bccc-4fb1-8f1e-653281dbdcae\" (UID: \"1073ee9b-bccc-4fb1-8f1e-653281dbdcae\") "
Nov 01 05:50:57.898312 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:57.897843    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-openshift-service-ca\") pod \"1073ee9b-bccc-4fb1-8f1e-653281dbdcae\" (UID: \"1073ee9b-bccc-4fb1-8f1e-653281dbdcae\") "
Nov 01 05:50:57.898312 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:57.897889    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-kube-api-access-6kpbj\" (UniqueName: \"kubernetes.io/projected/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-kube-api-access-6kpbj\") pod \"1073ee9b-bccc-4fb1-8f1e-653281dbdcae\" (UID: \"1073ee9b-bccc-4fb1-8f1e-653281dbdcae\") "
Nov 01 05:50:57.898312 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:57.897923    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-metrics-certs\") pod \"1073ee9b-bccc-4fb1-8f1e-653281dbdcae\" (UID: \"1073ee9b-bccc-4fb1-8f1e-653281dbdcae\") "
Nov 01 05:50:57.899386 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:57.899353    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-openshift-service-ca" pod "1073ee9b-bccc-4fb1-8f1e-653281dbdcae" (UID: "1073ee9b-bccc-4fb1-8f1e-653281dbdcae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:50:57.901000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:57.900970    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-kube-api-access-6kpbj" pod "1073ee9b-bccc-4fb1-8f1e-653281dbdcae" (UID: "1073ee9b-bccc-4fb1-8f1e-653281dbdcae"). InnerVolumeSpecName "kube-api-access-6kpbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:50:57.902473 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:57.902441    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-metrics-certs" pod "1073ee9b-bccc-4fb1-8f1e-653281dbdcae" (UID: "1073ee9b-bccc-4fb1-8f1e-653281dbdcae"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:50:57.902565 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:57.902463    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-maas-api-tls" pod "1073ee9b-bccc-4fb1-8f1e-653281dbdcae" (UID: "1073ee9b-bccc-4fb1-8f1e-653281dbdcae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:50:57.999720 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:57.999639    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-maas-api-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:50:57.999720 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:57.999669    4864 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-openshift-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:50:57.999720 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:57.999680    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kpbj\" (UniqueName: \"kubernetes.io/projected/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-kube-api-access-6kpbj\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:50:57.999720 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:57.999689    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1073ee9b-bccc-4fb1-8f1e-653281dbdcae-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:50:58.495905 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:58.495856    4864 generic.go:356] "Generic (PLEG): container finished" podID="1073ee9b-bccc-4fb1-8f1e-653281dbdcae" containerID="947dd1d080188818e439040f6a9c475fc9127a118a40ec9a3c88b279a044f6c6" exitCode=0
Nov 01 05:50:58.496676 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:58.495920    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t" event={"ID":"1073ee9b-bccc-4fb1-8f1e-653281dbdcae","Type":"ContainerDied","Data":"947dd1d080188818e439040f6a9c475fc9127a118a40ec9a3c88b279a044f6c6"}
Nov 01 05:50:58.496676 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:58.495993    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t" event={"ID":"1073ee9b-bccc-4fb1-8f1e-653281dbdcae","Type":"ContainerDied","Data":"1a8db81c09deb5ef754b80a1b6c173ef33e8b1aee7ccea98c2a30b9368c136c7"}
Nov 01 05:50:58.496676 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:58.496028    4864 scope.go:122] "RemoveContainer" containerID="947dd1d080188818e439040f6a9c475fc9127a118a40ec9a3c88b279a044f6c6"
Nov 01 05:50:58.497352 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:58.497316    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"
Nov 01 05:50:58.508672 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:58.508650    4864 scope.go:122] "RemoveContainer" containerID="947dd1d080188818e439040f6a9c475fc9127a118a40ec9a3c88b279a044f6c6"
Nov 01 05:50:58.509048 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:50:58.509027    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"947dd1d080188818e439040f6a9c475fc9127a118a40ec9a3c88b279a044f6c6\": container with ID starting with 947dd1d080188818e439040f6a9c475fc9127a118a40ec9a3c88b279a044f6c6 not found: ID does not exist" containerID="947dd1d080188818e439040f6a9c475fc9127a118a40ec9a3c88b279a044f6c6"
Nov 01 05:50:58.509120 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:58.509059    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"947dd1d080188818e439040f6a9c475fc9127a118a40ec9a3c88b279a044f6c6"} err="failed to get container status \"947dd1d080188818e439040f6a9c475fc9127a118a40ec9a3c88b279a044f6c6\": rpc error: code = NotFound desc = could not find container \"947dd1d080188818e439040f6a9c475fc9127a118a40ec9a3c88b279a044f6c6\": container with ID starting with 947dd1d080188818e439040f6a9c475fc9127a118a40ec9a3c88b279a044f6c6 not found: ID does not exist"
Nov 01 05:50:58.665423 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:58.665385    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"]
Nov 01 05:50:58.716240 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:58.716197    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-ad05c6-7cb7f86f9c-pqf2t"]
Nov 01 05:50:59.248448 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:50:59.248415    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1073ee9b-bccc-4fb1-8f1e-653281dbdcae" path="/var/lib/kubelet/pods/1073ee9b-bccc-4fb1-8f1e-653281dbdcae/volumes"
Nov 01 05:51:01.464509 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:01.464475    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"
Nov 01 05:51:06.339134 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.339099    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"]
Nov 01 05:51:06.339655 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.339637    4864 cpu_manager.go:414] "containerMap: removing container" podUID="1073ee9b-bccc-4fb1-8f1e-653281dbdcae" containerName="maas-api"
Nov 01 05:51:06.339655 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.339650    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1073ee9b-bccc-4fb1-8f1e-653281dbdcae" containerName="maas-api"
Nov 01 05:51:06.339746 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.339735    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="1073ee9b-bccc-4fb1-8f1e-653281dbdcae" containerName="maas-api"
Nov 01 05:51:06.344135 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.344113    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.399960 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.398393    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7238a45-e742-44f5-a321-1fd4d5daed4d-istiod-ca-cert\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.399960 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.398541    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-workload-socket\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.399960 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.398707    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-credential-socket\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.399960 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.399202    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-podinfo\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.399960 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.399338    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-5d585e-openshift-default-dockercfg-ksg55\""
Nov 01 05:51:06.399960 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.399569    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-data\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.399960 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.399608    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-token\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.411005 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.399998    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-workload-certs\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.411481 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.411348    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-envoy\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.411749 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.411642    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfnmj\" (UniqueName: \"kubernetes.io/projected/d7238a45-e742-44f5-a321-1fd4d5daed4d-kube-api-access-dfnmj\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.452610 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.452559    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"]
Nov 01 05:51:06.513254 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.513211    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-workload-certs\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.513464 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.513349    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-envoy\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.513464 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.513410    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfnmj\" (UniqueName: \"kubernetes.io/projected/d7238a45-e742-44f5-a321-1fd4d5daed4d-kube-api-access-dfnmj\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.513464 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.513462    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7238a45-e742-44f5-a321-1fd4d5daed4d-istiod-ca-cert\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.513646 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.513495    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-workload-socket\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.513646 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.513570    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-credential-socket\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.513646 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.513633    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-podinfo\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.513838 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.513722    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-data\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.513838 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.513759    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-token\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.514333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.514303    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-workload-socket\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.514491 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.514396    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-workload-certs\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.514609 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.514442    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-credential-socket\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.515092 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.515066    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-data\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.515388 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.515367    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7238a45-e742-44f5-a321-1fd4d5daed4d-istiod-ca-cert\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.518059 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.517914    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-envoy\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.523877 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.523154    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-podinfo\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.578122 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.578085    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-token\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.579893 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.579861    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfnmj\" (UniqueName: \"kubernetes.io/projected/d7238a45-e742-44f5-a321-1fd4d5daed4d-kube-api-access-dfnmj\") pod \"e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") " pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:06.658465 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:06.658374    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:07.112333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:07.112309    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"]
Nov 01 05:51:07.113940 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:51:07.113908    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7238a45_e742_44f5_a321_1fd4d5daed4d.slice/crio-55d7da79c963023b4e3684db582acec65d6fe66400874d8e7e3c4b0883365fba WatchSource:0}: Error finding container 55d7da79c963023b4e3684db582acec65d6fe66400874d8e7e3c4b0883365fba: Status 404 returned error can't find the container with id 55d7da79c963023b4e3684db582acec65d6fe66400874d8e7e3c4b0883365fba
Nov 01 05:51:07.120276 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:07.120238    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:51:07.120402 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:07.120311    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:51:07.120402 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:07.120341    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:51:07.560752 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:07.560714    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr" event={"ID":"d7238a45-e742-44f5-a321-1fd4d5daed4d","Type":"ContainerStarted","Data":"e84ca85f13aa56800ed9c17dbf626f5b26ad1ffe58bb0d5ac6aff202d3ce9afb"}
Nov 01 05:51:07.561159 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:07.560800    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr" event={"ID":"d7238a45-e742-44f5-a321-1fd4d5daed4d","Type":"ContainerStarted","Data":"55d7da79c963023b4e3684db582acec65d6fe66400874d8e7e3c4b0883365fba"}
Nov 01 05:51:07.658551 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:07.658505    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:07.682133 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:07.682080    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr" podStartSLOduration=1.682066164 podStartE2EDuration="1.682066164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:51:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:51:07.679915919 +0000 UTC m=+2884.570291897" watchObservedRunningTime="2026-11-01 05:51:07.682066164 +0000 UTC m=+2884.572442201"
Nov 01 05:51:08.666420 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:08.666387    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:09.576845 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:09.576814    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:09.577947 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:09.577922    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:51:10.857542 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:10.857503    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"]
Nov 01 05:51:10.862552 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:10.862529    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:10.958269 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:10.958232    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"]
Nov 01 05:51:10.965832 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:10.965231    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7584cc14-1a27-4b58-b661-804a446c1382-istio-podinfo\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:10.965832 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:10.965298    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-workload-certs\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:10.965832 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:10.965368    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-credential-socket\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:10.965832 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:10.965394    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7584cc14-1a27-4b58-b661-804a446c1382-istio-token\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:10.965832 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:10.965426    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7584cc14-1a27-4b58-b661-804a446c1382-istiod-ca-cert\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:10.965832 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:10.965459    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-workload-socket\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:10.965832 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:10.965585    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-istio-envoy\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:10.965832 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:10.965650    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8h2h\" (UniqueName: \"kubernetes.io/projected/7584cc14-1a27-4b58-b661-804a446c1382-kube-api-access-z8h2h\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:10.965832 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:10.965701    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-istio-data\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:11.003316 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.003279    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-25565930-openshift-default-dockercfg-f64wx\""
Nov 01 05:51:11.066595 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.066552    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-istio-envoy\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:11.066826 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.066632    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8h2h\" (UniqueName: \"kubernetes.io/projected/7584cc14-1a27-4b58-b661-804a446c1382-kube-api-access-z8h2h\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:11.066826 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.066680    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-istio-data\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:11.066952 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.066830    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7584cc14-1a27-4b58-b661-804a446c1382-istio-podinfo\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:11.066952 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.066901    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-workload-certs\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:11.067065 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.066975    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-credential-socket\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:11.067065 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.067000    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7584cc14-1a27-4b58-b661-804a446c1382-istio-token\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:11.067065 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.067034    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7584cc14-1a27-4b58-b661-804a446c1382-istiod-ca-cert\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:11.067227 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.067065    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-workload-socket\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:11.067227 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.067092    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-istio-data\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:11.067601 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.067574    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-workload-socket\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:11.067906 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.067842    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-credential-socket\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:11.067906 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.067951    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-workload-certs\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:11.068200 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.068062    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7584cc14-1a27-4b58-b661-804a446c1382-istiod-ca-cert\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:11.069475 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.069455    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-istio-envoy\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:11.069927 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.069904    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7584cc14-1a27-4b58-b661-804a446c1382-istio-podinfo\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:11.127867 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.127771    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7584cc14-1a27-4b58-b661-804a446c1382-istio-token\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:11.127867 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.127806    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8h2h\" (UniqueName: \"kubernetes.io/projected/7584cc14-1a27-4b58-b661-804a446c1382-kube-api-access-z8h2h\") pod \"e2e-mt-25565930-openshift-default-64f5666649-wjfgh\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:11.177440 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.177402    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:11.610171 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.610143    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"]
Nov 01 05:51:11.611216 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:51:11.611184    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7584cc14_1a27_4b58_b661_804a446c1382.slice/crio-fe6c5bb47c7497f5021c1b96e4b635ac99dc708a4d6fea57be3a7a094d080bac WatchSource:0}: Error finding container fe6c5bb47c7497f5021c1b96e4b635ac99dc708a4d6fea57be3a7a094d080bac: Status 404 returned error can't find the container with id fe6c5bb47c7497f5021c1b96e4b635ac99dc708a4d6fea57be3a7a094d080bac
Nov 01 05:51:11.613676 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.613645    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:51:11.613759 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.613712    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:51:11.613759 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:11.613740    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:51:12.610756 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:12.610714    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh" event={"ID":"7584cc14-1a27-4b58-b661-804a446c1382","Type":"ContainerStarted","Data":"717d9166d9877d8dcc228cc407ba024f5b162095c491e44d5baa2c86ee5db8b3"}
Nov 01 05:51:12.611197 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:12.610806    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh" event={"ID":"7584cc14-1a27-4b58-b661-804a446c1382","Type":"ContainerStarted","Data":"fe6c5bb47c7497f5021c1b96e4b635ac99dc708a4d6fea57be3a7a094d080bac"}
Nov 01 05:51:12.790698 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:12.790625    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh" podStartSLOduration=2.790603308 podStartE2EDuration="2.790603308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:51:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:51:12.787339544 +0000 UTC m=+2889.677715520" watchObservedRunningTime="2026-11-01 05:51:12.790603308 +0000 UTC m=+2889.680979284"
Nov 01 05:51:13.181955 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:13.181846    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:13.182897 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:13.182767    4864 patch_prober.go:29] interesting pod/e2e-mt-25565930-openshift-default-64f5666649-wjfgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Nov 01 05:51:13.183051 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:13.182928    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh" podUID="7584cc14-1a27-4b58-b661-804a446c1382" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Nov 01 05:51:14.182347 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.182307    4864 patch_prober.go:29] interesting pod/e2e-mt-25565930-openshift-default-64f5666649-wjfgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Nov 01 05:51:14.183230 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.183187    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh" podUID="7584cc14-1a27-4b58-b661-804a446c1382" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Nov 01 05:51:14.305552 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.305503    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"]
Nov 01 05:51:14.310857 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.310763    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.349756 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.349718    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"]
Nov 01 05:51:14.401742 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.401169    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-7a360a-openshift-default-dockercfg-ggnz9\""
Nov 01 05:51:14.439330 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.439242    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-credential-socket\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.439330 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.439277    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-envoy\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.439330 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.439316    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-workload-certs\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.439330 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.439333    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-workload-socket\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.439666 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.439354    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9mrl\" (UniqueName: \"kubernetes.io/projected/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-kube-api-access-z9mrl\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.439666 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.439379    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-token\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.439666 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.439420    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-podinfo\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.439666 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.439464    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-data\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.439666 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.439521    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istiod-ca-cert\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.541124 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.541084    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-credential-socket\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.541306 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.541127    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-envoy\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.541306 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.541173    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-workload-certs\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.541306 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.541191    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-workload-socket\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.541306 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.541218    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9mrl\" (UniqueName: \"kubernetes.io/projected/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-kube-api-access-z9mrl\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.541306 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.541248    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-token\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.541306 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.541305    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-podinfo\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.541492 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.541369    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-data\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.541492 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.541454    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istiod-ca-cert\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.541603 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.541579    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-workload-certs\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.542056 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.542034    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-credential-socket\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.544950 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.542558    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-workload-socket\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.544950 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.542813    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-data\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.544950 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.543318    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istiod-ca-cert\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.545432 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.545402    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-podinfo\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.552882 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.552858    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-envoy\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.602289 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.602248    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-token\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.602492 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.602466    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9mrl\" (UniqueName: \"kubernetes.io/projected/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-kube-api-access-z9mrl\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:14.636483 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:14.636215    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:15.101742 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:15.101706    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"]
Nov 01 05:51:15.141882 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:51:15.141848    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fe9c2f7_8c01_4c99_9b80_0b3442562b56.slice/crio-e658705ccb82b5046e4f1859cd3854b039d7a62d3a213d1d735c3ed4631eca3e WatchSource:0}: Error finding container e658705ccb82b5046e4f1859cd3854b039d7a62d3a213d1d735c3ed4631eca3e: Status 404 returned error can't find the container with id e658705ccb82b5046e4f1859cd3854b039d7a62d3a213d1d735c3ed4631eca3e
Nov 01 05:51:15.144086 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:15.144053    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:51:15.144194 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:15.144124    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:51:15.144194 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:15.144153    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:51:15.179811 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:15.179221    4864 patch_prober.go:29] interesting pod/e2e-mt-25565930-openshift-default-64f5666649-wjfgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Nov 01 05:51:15.179811 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:15.179284    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh" podUID="7584cc14-1a27-4b58-b661-804a446c1382" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Nov 01 05:51:15.631341 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:15.631302    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt" event={"ID":"0fe9c2f7-8c01-4c99-9b80-0b3442562b56","Type":"ContainerStarted","Data":"92cd4241647a7d0880575e1e6a9e10df526be282837e36708e9fb334e73a52de"}
Nov 01 05:51:15.631866 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:15.631368    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt" event={"ID":"0fe9c2f7-8c01-4c99-9b80-0b3442562b56","Type":"ContainerStarted","Data":"e658705ccb82b5046e4f1859cd3854b039d7a62d3a213d1d735c3ed4631eca3e"}
Nov 01 05:51:15.636643 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:15.636617    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:15.771812 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:15.763298    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt" podStartSLOduration=1.7632775870000001 podStartE2EDuration="1.763277587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:51:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:51:15.759941396 +0000 UTC m=+2892.650317373" watchObservedRunningTime="2026-11-01 05:51:15.763277587 +0000 UTC m=+2892.653653563"
Nov 01 05:51:16.178275 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:16.178233    4864 patch_prober.go:29] interesting pod/e2e-mt-25565930-openshift-default-64f5666649-wjfgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Nov 01 05:51:16.178549 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:16.178521    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh" podUID="7584cc14-1a27-4b58-b661-804a446c1382" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Nov 01 05:51:16.638716 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:16.638685    4864 patch_prober.go:29] interesting pod/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 01 05:51:16.639134 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:16.638729    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt" podUID="0fe9c2f7-8c01-4c99-9b80-0b3442562b56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 01 05:51:17.178280 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:17.178238    4864 patch_prober.go:29] interesting pod/e2e-mt-25565930-openshift-default-64f5666649-wjfgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Nov 01 05:51:17.179046 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:17.178592    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh" podUID="7584cc14-1a27-4b58-b661-804a446c1382" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Nov 01 05:51:17.636761 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:17.636730    4864 patch_prober.go:29] interesting pod/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 01 05:51:17.637039 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:17.637015    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt" podUID="0fe9c2f7-8c01-4c99-9b80-0b3442562b56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 01 05:51:18.183811 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:18.179198    4864 patch_prober.go:29] interesting pod/e2e-mt-25565930-openshift-default-64f5666649-wjfgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Nov 01 05:51:18.183811 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:18.179267    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh" podUID="7584cc14-1a27-4b58-b661-804a446c1382" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Nov 01 05:51:18.637000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:18.636962    4864 patch_prober.go:29] interesting pod/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 01 05:51:18.637195 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:18.637014    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt" podUID="0fe9c2f7-8c01-4c99-9b80-0b3442562b56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 01 05:51:19.190066 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:19.190035    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:19.190500 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:19.190418    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:19.191374 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:19.191355    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:51:19.637196 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:19.637161    4864 patch_prober.go:29] interesting pod/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 01 05:51:19.637395 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:19.637220    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt" podUID="0fe9c2f7-8c01-4c99-9b80-0b3442562b56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 01 05:51:20.636848 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:20.636815    4864 patch_prober.go:29] interesting pod/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 01 05:51:20.637276 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:20.636870    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt" podUID="0fe9c2f7-8c01-4c99-9b80-0b3442562b56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 01 05:51:21.641363 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:21.641332    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:21.641833 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:21.641800    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:21.642527 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:21.642507    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 05:51:22.528747 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:51:22.528708    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:51:25.955118 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:25.955076    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"]
Nov 01 05:51:25.960640 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:25.960615    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.053772 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.053735    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-istio-data\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.053994 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.053856    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxhmt\" (UniqueName: \"kubernetes.io/projected/402a4249-e41e-47db-ab46-977480475a50-kube-api-access-vxhmt\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.053994 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.053900    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/402a4249-e41e-47db-ab46-977480475a50-istio-podinfo\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.054111 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.054052    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/402a4249-e41e-47db-ab46-977480475a50-istio-token\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.054111 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.054094    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-workload-certs\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.054184 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.054168    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-workload-socket\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.054234 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.054216    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-istio-envoy\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.054287 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.054276    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/402a4249-e41e-47db-ab46-977480475a50-istiod-ca-cert\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.054322 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.054297    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-credential-socket\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.066074 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.066043    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"]
Nov 01 05:51:26.155815 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.155759    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-workload-socket\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.156089 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.155831    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-istio-envoy\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.156089 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.155852    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/402a4249-e41e-47db-ab46-977480475a50-istiod-ca-cert\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.156089 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.155868    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-credential-socket\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.156089 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.155952    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-istio-data\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.156089 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.156004    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxhmt\" (UniqueName: \"kubernetes.io/projected/402a4249-e41e-47db-ab46-977480475a50-kube-api-access-vxhmt\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.156089 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.156056    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/402a4249-e41e-47db-ab46-977480475a50-istio-podinfo\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.156419 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.156132    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/402a4249-e41e-47db-ab46-977480475a50-istio-token\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.156419 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.156157    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-workload-certs\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.156419 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.156287    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-workload-socket\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.156419 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.156360    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-credential-socket\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.156677 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.156587    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-istio-data\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.156677 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.156601    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-workload-certs\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.156925 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.156904    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/402a4249-e41e-47db-ab46-977480475a50-istiod-ca-cert\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.158622 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.158601    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-istio-envoy\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.158977 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.158956    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/402a4249-e41e-47db-ab46-977480475a50-istio-podinfo\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.217691 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.217606    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/402a4249-e41e-47db-ab46-977480475a50-istio-token\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.218257 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.218233    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxhmt\" (UniqueName: \"kubernetes.io/projected/402a4249-e41e-47db-ab46-977480475a50-kube-api-access-vxhmt\") pod \"e2e-mt-25565930-openshift-default-64f5666649-v8m4h\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") " pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.274964 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.274924    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:26.708739 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:51:26.708693    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod402a4249_e41e_47db_ab46_977480475a50.slice/crio-6f27c8a4bf2ff2f70b172f7bf93306bc53adc8822c95cbd9d0b9f26bb5eed168 WatchSource:0}: Error finding container 6f27c8a4bf2ff2f70b172f7bf93306bc53adc8822c95cbd9d0b9f26bb5eed168: Status 404 returned error can't find the container with id 6f27c8a4bf2ff2f70b172f7bf93306bc53adc8822c95cbd9d0b9f26bb5eed168
Nov 01 05:51:26.708739 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.708714    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"]
Nov 01 05:51:26.711592 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.711560    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:51:26.711698 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.711625    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:51:26.711698 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.711654    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:51:26.727550 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:26.727517    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h" event={"ID":"402a4249-e41e-47db-ab46-977480475a50","Type":"ContainerStarted","Data":"6f27c8a4bf2ff2f70b172f7bf93306bc53adc8822c95cbd9d0b9f26bb5eed168"}
Nov 01 05:51:27.735634 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:27.735596    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h" event={"ID":"402a4249-e41e-47db-ab46-977480475a50","Type":"ContainerStarted","Data":"ec6ff5039a608a041d4adf50fb3e89a1d601e463900ec15afce09d0939ed809e"}
Nov 01 05:51:27.907344 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:27.907290    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h" podStartSLOduration=2.907276037 podStartE2EDuration="2.907276037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:51:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:51:27.903768452 +0000 UTC m=+2904.794144428" watchObservedRunningTime="2026-11-01 05:51:27.907276037 +0000 UTC m=+2904.797652006"
Nov 01 05:51:28.283339 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:28.283302    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:28.283753 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:28.283730    4864 patch_prober.go:29] interesting pod/e2e-mt-25565930-openshift-default-64f5666649-v8m4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 01 05:51:28.283989 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:28.283964    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h" podUID="402a4249-e41e-47db-ab46-977480475a50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 01 05:51:29.278809 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:29.278743    4864 patch_prober.go:29] interesting pod/e2e-mt-25565930-openshift-default-64f5666649-v8m4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 01 05:51:29.280333 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:29.280296    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h" podUID="402a4249-e41e-47db-ab46-977480475a50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 01 05:51:30.283810 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:30.283170    4864 patch_prober.go:29] interesting pod/e2e-mt-25565930-openshift-default-64f5666649-v8m4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 01 05:51:30.283810 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:30.283234    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h" podUID="402a4249-e41e-47db-ab46-977480475a50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 01 05:51:31.291816 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:31.276153    4864 patch_prober.go:29] interesting pod/e2e-mt-25565930-openshift-default-64f5666649-v8m4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 01 05:51:31.291816 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:31.276213    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h" podUID="402a4249-e41e-47db-ab46-977480475a50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 01 05:51:32.277258 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:32.277215    4864 patch_prober.go:29] interesting pod/e2e-mt-25565930-openshift-default-64f5666649-v8m4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 01 05:51:32.277541 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:32.277509    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h" podUID="402a4249-e41e-47db-ab46-977480475a50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 01 05:51:33.276036 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:33.275993    4864 patch_prober.go:29] interesting pod/e2e-mt-25565930-openshift-default-64f5666649-v8m4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 01 05:51:33.276454 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:33.276056    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h" podUID="402a4249-e41e-47db-ab46-977480475a50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 01 05:51:34.279920 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:34.279886    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:34.280319 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:34.280285    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:34.280906 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:34.280887    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:51:52.524674 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:51:52.524587    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:51:59.633272 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.633234    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"]
Nov 01 05:51:59.638486 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.638465    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.668034 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.668007    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-credential-socket\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.668181 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.668064    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-workload-certs\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.668250 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.668182    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-data\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.668250 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.668212    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0e25f67-566f-4d28-a0b5-4df45774fd39-istiod-ca-cert\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.668368 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.668349    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-token\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.668426 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.668383    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q25f\" (UniqueName: \"kubernetes.io/projected/b0e25f67-566f-4d28-a0b5-4df45774fd39-kube-api-access-2q25f\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.668508 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.668494    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-workload-socket\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.668565 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.668542    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-podinfo\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.668654 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.668636    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-envoy\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.743330 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.743295    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"]
Nov 01 05:51:59.770182 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.770145    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-data\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.770182 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.770177    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0e25f67-566f-4d28-a0b5-4df45774fd39-istiod-ca-cert\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.770431 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.770330    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-token\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.770431 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.770368    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q25f\" (UniqueName: \"kubernetes.io/projected/b0e25f67-566f-4d28-a0b5-4df45774fd39-kube-api-access-2q25f\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.770431 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.770428    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-workload-socket\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.770617 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.770487    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-podinfo\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.770617 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.770518    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-envoy\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.770617 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.770584    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-credential-socket\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.770777 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.770633    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-workload-certs\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.770777 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.770682    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-data\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.771000 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.770974    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0e25f67-566f-4d28-a0b5-4df45774fd39-istiod-ca-cert\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.771166 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.771007    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-workload-socket\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.771281 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.771170    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-workload-certs\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.771349 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.771288    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-credential-socket\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.773822 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.773797    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-envoy\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.773929 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.773805    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-podinfo\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.831014 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.830981    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-token\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.831195 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.831178    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q25f\" (UniqueName: \"kubernetes.io/projected/b0e25f67-566f-4d28-a0b5-4df45774fd39-kube-api-access-2q25f\") pod \"e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") " pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:51:59.953462 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:51:59.953377    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:52:00.386751 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:00.386718    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"]
Nov 01 05:52:00.424180 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:52:00.424144    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0e25f67_566f_4d28_a0b5_4df45774fd39.slice/crio-07f9808d3dcecab7b96330a959a693684fd91ab69fb6655966815a1574144601 WatchSource:0}: Error finding container 07f9808d3dcecab7b96330a959a693684fd91ab69fb6655966815a1574144601: Status 404 returned error can't find the container with id 07f9808d3dcecab7b96330a959a693684fd91ab69fb6655966815a1574144601
Nov 01 05:52:00.427007 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:00.426969    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:52:00.427098 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:00.427060    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:52:00.427142 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:00.427100    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:52:00.978691 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:00.978653    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr" event={"ID":"b0e25f67-566f-4d28-a0b5-4df45774fd39","Type":"ContainerStarted","Data":"ec36ee7ba0e2fa333243fcc17344bf76db2bd049860229d119cf49cf50acb08e"}
Nov 01 05:52:00.979361 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:00.978720    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr" event={"ID":"b0e25f67-566f-4d28-a0b5-4df45774fd39","Type":"ContainerStarted","Data":"07f9808d3dcecab7b96330a959a693684fd91ab69fb6655966815a1574144601"}
Nov 01 05:52:01.100756 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:01.100699    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr" podStartSLOduration=2.100682467 podStartE2EDuration="2.100682467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:51:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:52:01.097441879 +0000 UTC m=+2937.987817866" watchObservedRunningTime="2026-11-01 05:52:01.100682467 +0000 UTC m=+2937.991058442"
Nov 01 05:52:01.954001 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:01.953963    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:52:01.955986 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:01.955954    4864 patch_prober.go:29] interesting pod/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 01 05:52:01.956122 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:01.956008    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr" podUID="b0e25f67-566f-4d28-a0b5-4df45774fd39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 01 05:52:02.966669 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:02.966263    4864 patch_prober.go:29] interesting pod/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 01 05:52:02.966669 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:02.966325    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr" podUID="b0e25f67-566f-4d28-a0b5-4df45774fd39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 01 05:52:03.960814 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:03.959187    4864 patch_prober.go:29] interesting pod/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 01 05:52:03.960814 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:03.959257    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr" podUID="b0e25f67-566f-4d28-a0b5-4df45774fd39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 01 05:52:04.954771 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:04.954736    4864 patch_prober.go:29] interesting pod/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 01 05:52:04.955244 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:04.954823    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr" podUID="b0e25f67-566f-4d28-a0b5-4df45774fd39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 01 05:52:05.953858 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:05.953824    4864 patch_prober.go:29] interesting pod/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 01 05:52:05.954038 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:05.953875    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr" podUID="b0e25f67-566f-4d28-a0b5-4df45774fd39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 01 05:52:06.959035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:06.958998    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:52:06.959488 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:06.959425    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:52:06.960119 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:06.960098    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:52:13.493485 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:13.493447    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"]
Nov 01 05:52:13.498831 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:13.498808    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"
Nov 01 05:52:13.552321 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:13.552298    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-5d585e-serving-cert\""
Nov 01 05:52:13.577351 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:13.577318    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8626eef5-3aa4-487c-aeb1-488f32b4712c-metrics-certs\") pod \"maas-api-e2e-explicit-5d585e-64bc54f899-jf489\" (UID: \"8626eef5-3aa4-487c-aeb1-488f32b4712c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"
Nov 01 05:52:13.577518 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:13.577389    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8626eef5-3aa4-487c-aeb1-488f32b4712c-openshift-service-ca\") pod \"maas-api-e2e-explicit-5d585e-64bc54f899-jf489\" (UID: \"8626eef5-3aa4-487c-aeb1-488f32b4712c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"
Nov 01 05:52:13.577518 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:13.577504    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8626eef5-3aa4-487c-aeb1-488f32b4712c-maas-api-tls\") pod \"maas-api-e2e-explicit-5d585e-64bc54f899-jf489\" (UID: \"8626eef5-3aa4-487c-aeb1-488f32b4712c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"
Nov 01 05:52:13.577819 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:13.577524    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfqt6\" (UniqueName: \"kubernetes.io/projected/8626eef5-3aa4-487c-aeb1-488f32b4712c-kube-api-access-rfqt6\") pod \"maas-api-e2e-explicit-5d585e-64bc54f899-jf489\" (UID: \"8626eef5-3aa4-487c-aeb1-488f32b4712c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"
Nov 01 05:52:13.605442 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:13.605413    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"]
Nov 01 05:52:13.678968 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:13.678937    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8626eef5-3aa4-487c-aeb1-488f32b4712c-openshift-service-ca\") pod \"maas-api-e2e-explicit-5d585e-64bc54f899-jf489\" (UID: \"8626eef5-3aa4-487c-aeb1-488f32b4712c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"
Nov 01 05:52:13.679113 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:13.679042    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8626eef5-3aa4-487c-aeb1-488f32b4712c-maas-api-tls\") pod \"maas-api-e2e-explicit-5d585e-64bc54f899-jf489\" (UID: \"8626eef5-3aa4-487c-aeb1-488f32b4712c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"
Nov 01 05:52:13.679113 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:13.679099    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfqt6\" (UniqueName: \"kubernetes.io/projected/8626eef5-3aa4-487c-aeb1-488f32b4712c-kube-api-access-rfqt6\") pod \"maas-api-e2e-explicit-5d585e-64bc54f899-jf489\" (UID: \"8626eef5-3aa4-487c-aeb1-488f32b4712c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"
Nov 01 05:52:13.679223 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:13.679206    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8626eef5-3aa4-487c-aeb1-488f32b4712c-metrics-certs\") pod \"maas-api-e2e-explicit-5d585e-64bc54f899-jf489\" (UID: \"8626eef5-3aa4-487c-aeb1-488f32b4712c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"
Nov 01 05:52:13.679311 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:52:13.679212    4864 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-serving-cert: secret "maas-api-e2e-explicit-5d585e-serving-cert" not found
Nov 01 05:52:13.679388 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:52:13.679358    4864 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8626eef5-3aa4-487c-aeb1-488f32b4712c-maas-api-tls podName:8626eef5-3aa4-487c-aeb1-488f32b4712c nodeName:}" failed. No retries permitted until 2026-11-01 05:52:14.179338331 +0000 UTC m=+2951.069714299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8626eef5-3aa4-487c-aeb1-488f32b4712c-maas-api-tls") pod "maas-api-e2e-explicit-5d585e-64bc54f899-jf489" (UID: "8626eef5-3aa4-487c-aeb1-488f32b4712c") : secret "maas-api-e2e-explicit-5d585e-serving-cert" not found
Nov 01 05:52:13.679696 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:13.679668    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8626eef5-3aa4-487c-aeb1-488f32b4712c-openshift-service-ca\") pod \"maas-api-e2e-explicit-5d585e-64bc54f899-jf489\" (UID: \"8626eef5-3aa4-487c-aeb1-488f32b4712c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"
Nov 01 05:52:13.681969 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:13.681947    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8626eef5-3aa4-487c-aeb1-488f32b4712c-metrics-certs\") pod \"maas-api-e2e-explicit-5d585e-64bc54f899-jf489\" (UID: \"8626eef5-3aa4-487c-aeb1-488f32b4712c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"
Nov 01 05:52:13.744987 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:13.744910    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfqt6\" (UniqueName: \"kubernetes.io/projected/8626eef5-3aa4-487c-aeb1-488f32b4712c-kube-api-access-rfqt6\") pod \"maas-api-e2e-explicit-5d585e-64bc54f899-jf489\" (UID: \"8626eef5-3aa4-487c-aeb1-488f32b4712c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"
Nov 01 05:52:14.185223 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:14.185190    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8626eef5-3aa4-487c-aeb1-488f32b4712c-maas-api-tls\") pod \"maas-api-e2e-explicit-5d585e-64bc54f899-jf489\" (UID: \"8626eef5-3aa4-487c-aeb1-488f32b4712c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"
Nov 01 05:52:14.188032 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:14.188008    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8626eef5-3aa4-487c-aeb1-488f32b4712c-maas-api-tls\") pod \"maas-api-e2e-explicit-5d585e-64bc54f899-jf489\" (UID: \"8626eef5-3aa4-487c-aeb1-488f32b4712c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"
Nov 01 05:52:14.415642 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:14.415606    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"
Nov 01 05:52:14.798971 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:14.798944    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"]
Nov 01 05:52:14.800193 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:52:14.800165    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8626eef5_3aa4_487c_aeb1_488f32b4712c.slice/crio-89cb0902b0e3edbf43643ce978747bf8b7c160bf994a7ed06e62d090f3056ee0 WatchSource:0}: Error finding container 89cb0902b0e3edbf43643ce978747bf8b7c160bf994a7ed06e62d090f3056ee0: Status 404 returned error can't find the container with id 89cb0902b0e3edbf43643ce978747bf8b7c160bf994a7ed06e62d090f3056ee0
Nov 01 05:52:15.184302 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:15.184268    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489" event={"ID":"8626eef5-3aa4-487c-aeb1-488f32b4712c","Type":"ContainerStarted","Data":"89cb0902b0e3edbf43643ce978747bf8b7c160bf994a7ed06e62d090f3056ee0"}
Nov 01 05:52:16.192848 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:16.192813    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489" event={"ID":"8626eef5-3aa4-487c-aeb1-488f32b4712c","Type":"ContainerStarted","Data":"063352f4f46b4680e6d263fcc220c33d68b53236bc294410c1f0a06c8e92e067"}
Nov 01 05:52:16.193336 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:16.192881    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"
Nov 01 05:52:16.310844 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:16.310763    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489" podStartSLOduration=2.476571907 podStartE2EDuration="3.310744328s" totalImagesPullingTime="834.172421ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:52:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:52:16.308503192 +0000 UTC m=+2953.198879167" watchObservedRunningTime="2026-11-01 05:52:16.310744328 +0000 UTC m=+2953.201120305"
Nov 01 05:52:22.206662 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:22.206632    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"
Nov 01 05:52:22.523854 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:52:22.523745    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:52:46.404429 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:46.404393    4864 generic.go:356] "Generic (PLEG): container finished" podID="d7238a45-e742-44f5-a321-1fd4d5daed4d" containerID="e84ca85f13aa56800ed9c17dbf626f5b26ad1ffe58bb0d5ac6aff202d3ce9afb" exitCode=0
Nov 01 05:52:46.404429 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:46.404427    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr" event={"ID":"d7238a45-e742-44f5-a321-1fd4d5daed4d","Type":"ContainerDied","Data":"e84ca85f13aa56800ed9c17dbf626f5b26ad1ffe58bb0d5ac6aff202d3ce9afb"}
Nov 01 05:52:46.405132 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:46.404882    4864 scope.go:122] "RemoveContainer" containerID="e84ca85f13aa56800ed9c17dbf626f5b26ad1ffe58bb0d5ac6aff202d3ce9afb"
Nov 01 05:52:46.406566 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:46.406534    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:52:46.406654 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:46.406624    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:52:46.406691 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:46.406661    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:52:47.421330 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:47.421296    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr" event={"ID":"d7238a45-e742-44f5-a321-1fd4d5daed4d","Type":"ContainerStarted","Data":"2a816faada5fcdc5c156bc114d479f6305c31cc3e6fdc64b0228a34f9abf934a"}
Nov 01 05:52:47.669353 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:47.669319    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:52:47.669717 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:47.669692    4864 patch_prober.go:29] interesting pod/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Nov 01 05:52:47.669903 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:47.669881    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr" podUID="d7238a45-e742-44f5-a321-1fd4d5daed4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Nov 01 05:52:48.667811 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:48.661876    4864 patch_prober.go:29] interesting pod/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Nov 01 05:52:48.667811 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:48.661941    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr" podUID="d7238a45-e742-44f5-a321-1fd4d5daed4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Nov 01 05:52:49.662230 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:49.662198    4864 patch_prober.go:29] interesting pod/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Nov 01 05:52:49.662517 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:49.662487    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr" podUID="d7238a45-e742-44f5-a321-1fd4d5daed4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Nov 01 05:52:50.663807 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:50.662072    4864 patch_prober.go:29] interesting pod/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Nov 01 05:52:50.663807 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:50.662131    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr" podUID="d7238a45-e742-44f5-a321-1fd4d5daed4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Nov 01 05:52:51.671848 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:51.669065    4864 patch_prober.go:29] interesting pod/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Nov 01 05:52:51.671848 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:51.669120    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr" podUID="d7238a45-e742-44f5-a321-1fd4d5daed4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Nov 01 05:52:52.530004 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:52:52.529962    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:52:52.664278 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:52.664245    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:52:52.664669 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:52.664641    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:52:52.665547 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:52:52.665527    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:53:04.015111 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:04.015081    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:53:04.024048 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:04.024024    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:53:04.024580 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:04.024558    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:53:04.033183 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:04.033159    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:53:06.050108 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.050071    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"]
Nov 01 05:53:06.050625 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.050295    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66" podUID="d8022115-881a-40ff-bab0-cfead6161302" containerName="maas-api" containerID="cri-o://92ed49407e179380b9c9bed5b5cb82e8a96b68de08c0d64e9d51f4e77cfa5230" gracePeriod=30
Nov 01 05:53:06.369005 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.368980    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"
Nov 01 05:53:06.550241 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.549231    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d8022115-881a-40ff-bab0-cfead6161302-kube-api-access-h85wp\" (UniqueName: \"kubernetes.io/projected/d8022115-881a-40ff-bab0-cfead6161302-kube-api-access-h85wp\") pod \"d8022115-881a-40ff-bab0-cfead6161302\" (UID: \"d8022115-881a-40ff-bab0-cfead6161302\") "
Nov 01 05:53:06.550241 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.549325    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d8022115-881a-40ff-bab0-cfead6161302-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8022115-881a-40ff-bab0-cfead6161302-metrics-certs\") pod \"d8022115-881a-40ff-bab0-cfead6161302\" (UID: \"d8022115-881a-40ff-bab0-cfead6161302\") "
Nov 01 05:53:06.550241 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.549454    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d8022115-881a-40ff-bab0-cfead6161302-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8022115-881a-40ff-bab0-cfead6161302-maas-api-tls\") pod \"d8022115-881a-40ff-bab0-cfead6161302\" (UID: \"d8022115-881a-40ff-bab0-cfead6161302\") "
Nov 01 05:53:06.550241 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.549543    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d8022115-881a-40ff-bab0-cfead6161302-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8022115-881a-40ff-bab0-cfead6161302-openshift-service-ca\") pod \"d8022115-881a-40ff-bab0-cfead6161302\" (UID: \"d8022115-881a-40ff-bab0-cfead6161302\") "
Nov 01 05:53:06.552189 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.551842    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8022115-881a-40ff-bab0-cfead6161302-openshift-service-ca" pod "d8022115-881a-40ff-bab0-cfead6161302" (UID: "d8022115-881a-40ff-bab0-cfead6161302"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:53:06.556159 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.556132    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8022115-881a-40ff-bab0-cfead6161302-metrics-certs" pod "d8022115-881a-40ff-bab0-cfead6161302" (UID: "d8022115-881a-40ff-bab0-cfead6161302"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:53:06.557943 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.557891    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8022115-881a-40ff-bab0-cfead6161302-maas-api-tls" pod "d8022115-881a-40ff-bab0-cfead6161302" (UID: "d8022115-881a-40ff-bab0-cfead6161302"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:53:06.558058 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.557971    4864 generic.go:356] "Generic (PLEG): container finished" podID="d8022115-881a-40ff-bab0-cfead6161302" containerID="92ed49407e179380b9c9bed5b5cb82e8a96b68de08c0d64e9d51f4e77cfa5230" exitCode=0
Nov 01 05:53:06.558159 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.558138    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66" event={"ID":"d8022115-881a-40ff-bab0-cfead6161302","Type":"ContainerDied","Data":"92ed49407e179380b9c9bed5b5cb82e8a96b68de08c0d64e9d51f4e77cfa5230"}
Nov 01 05:53:06.558220 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.558196    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66" event={"ID":"d8022115-881a-40ff-bab0-cfead6161302","Type":"ContainerDied","Data":"c3aee56f6c987f8a3128f65ca539053e2506ca29cada62a419a723428e394900"}
Nov 01 05:53:06.558281 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.558226    4864 scope.go:122] "RemoveContainer" containerID="92ed49407e179380b9c9bed5b5cb82e8a96b68de08c0d64e9d51f4e77cfa5230"
Nov 01 05:53:06.559822 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.558831    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"
Nov 01 05:53:06.567655 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.567378    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8022115-881a-40ff-bab0-cfead6161302-kube-api-access-h85wp" pod "d8022115-881a-40ff-bab0-cfead6161302" (UID: "d8022115-881a-40ff-bab0-cfead6161302"). InnerVolumeSpecName "kube-api-access-h85wp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:53:06.602138 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.602094    4864 scope.go:122] "RemoveContainer" containerID="92ed49407e179380b9c9bed5b5cb82e8a96b68de08c0d64e9d51f4e77cfa5230"
Nov 01 05:53:06.602928 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:53:06.602877    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92ed49407e179380b9c9bed5b5cb82e8a96b68de08c0d64e9d51f4e77cfa5230\": container with ID starting with 92ed49407e179380b9c9bed5b5cb82e8a96b68de08c0d64e9d51f4e77cfa5230 not found: ID does not exist" containerID="92ed49407e179380b9c9bed5b5cb82e8a96b68de08c0d64e9d51f4e77cfa5230"
Nov 01 05:53:06.603052 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.602920    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92ed49407e179380b9c9bed5b5cb82e8a96b68de08c0d64e9d51f4e77cfa5230"} err="failed to get container status \"92ed49407e179380b9c9bed5b5cb82e8a96b68de08c0d64e9d51f4e77cfa5230\": rpc error: code = NotFound desc = could not find container \"92ed49407e179380b9c9bed5b5cb82e8a96b68de08c0d64e9d51f4e77cfa5230\": container with ID starting with 92ed49407e179380b9c9bed5b5cb82e8a96b68de08c0d64e9d51f4e77cfa5230 not found: ID does not exist"
Nov 01 05:53:06.650933 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.650903    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h85wp\" (UniqueName: \"kubernetes.io/projected/d8022115-881a-40ff-bab0-cfead6161302-kube-api-access-h85wp\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:06.650933 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.650931    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8022115-881a-40ff-bab0-cfead6161302-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:06.650933 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.650941    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8022115-881a-40ff-bab0-cfead6161302-maas-api-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:06.651176 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.650951    4864 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8022115-881a-40ff-bab0-cfead6161302-openshift-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:06.981373 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.981344    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"]
Nov 01 05:53:06.985892 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:06.985865    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25565930-847c47b46f-tnl66"]
Nov 01 05:53:07.248168 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:07.248083    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d8022115-881a-40ff-bab0-cfead6161302" path="/var/lib/kubelet/pods/d8022115-881a-40ff-bab0-cfead6161302/volumes"
Nov 01 05:53:10.625964 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:10.624458    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"]
Nov 01 05:53:10.625964 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:10.624730    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh" podUID="7584cc14-1a27-4b58-b661-804a446c1382" containerName="istio-proxy" containerID="cri-o://717d9166d9877d8dcc228cc407ba024f5b162095c491e44d5baa2c86ee5db8b3" gracePeriod=30
Nov 01 05:53:10.642259 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:10.642230    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"]
Nov 01 05:53:10.642432 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:10.642407    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h" podUID="402a4249-e41e-47db-ab46-977480475a50" containerName="istio-proxy" containerID="cri-o://ec6ff5039a608a041d4adf50fb3e89a1d601e463900ec15afce09d0939ed809e" gracePeriod=30
Nov 01 05:53:15.941875 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.941849    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:53:15.967515 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.967481    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:53:15.975530 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.975510    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-workload-socket\") pod \"7584cc14-1a27-4b58-b661-804a446c1382\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") "
Nov 01 05:53:15.975618 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.975564    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7584cc14-1a27-4b58-b661-804a446c1382-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7584cc14-1a27-4b58-b661-804a446c1382-istio-podinfo\") pod \"7584cc14-1a27-4b58-b661-804a446c1382\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") "
Nov 01 05:53:15.975618 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.975592    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-istio-envoy\") pod \"7584cc14-1a27-4b58-b661-804a446c1382\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") "
Nov 01 05:53:15.975618 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.975612    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-istio-data\") pod \"7584cc14-1a27-4b58-b661-804a446c1382\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") "
Nov 01 05:53:15.975768 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.975648    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7584cc14-1a27-4b58-b661-804a446c1382-kube-api-access-z8h2h\" (UniqueName: \"kubernetes.io/projected/7584cc14-1a27-4b58-b661-804a446c1382-kube-api-access-z8h2h\") pod \"7584cc14-1a27-4b58-b661-804a446c1382\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") "
Nov 01 05:53:15.975768 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.975687    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7584cc14-1a27-4b58-b661-804a446c1382-istio-token\" (UniqueName: \"kubernetes.io/projected/7584cc14-1a27-4b58-b661-804a446c1382-istio-token\") pod \"7584cc14-1a27-4b58-b661-804a446c1382\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") "
Nov 01 05:53:15.975900 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.975815    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-workload-socket" pod "7584cc14-1a27-4b58-b661-804a446c1382" (UID: "7584cc14-1a27-4b58-b661-804a446c1382"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:53:15.975900 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.975858    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-workload-certs\") pod \"7584cc14-1a27-4b58-b661-804a446c1382\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") "
Nov 01 05:53:15.976004 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.975905    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-credential-socket\") pod \"7584cc14-1a27-4b58-b661-804a446c1382\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") "
Nov 01 05:53:15.976004 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.975955    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7584cc14-1a27-4b58-b661-804a446c1382-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7584cc14-1a27-4b58-b661-804a446c1382-istiod-ca-cert\") pod \"7584cc14-1a27-4b58-b661-804a446c1382\" (UID: \"7584cc14-1a27-4b58-b661-804a446c1382\") "
Nov 01 05:53:15.976004 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.975985    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-istio-data" pod "7584cc14-1a27-4b58-b661-804a446c1382" (UID: "7584cc14-1a27-4b58-b661-804a446c1382"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:53:15.976494 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.976226    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-workload-certs" pod "7584cc14-1a27-4b58-b661-804a446c1382" (UID: "7584cc14-1a27-4b58-b661-804a446c1382"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:53:15.976494 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.976406    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:15.976494 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.976427    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:15.976494 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.976443    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:15.976819 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.976632    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-credential-socket" pod "7584cc14-1a27-4b58-b661-804a446c1382" (UID: "7584cc14-1a27-4b58-b661-804a446c1382"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:53:15.976819 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.976710    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7584cc14-1a27-4b58-b661-804a446c1382-istiod-ca-cert" pod "7584cc14-1a27-4b58-b661-804a446c1382" (UID: "7584cc14-1a27-4b58-b661-804a446c1382"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:53:15.978345 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.978318    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-istio-envoy" pod "7584cc14-1a27-4b58-b661-804a446c1382" (UID: "7584cc14-1a27-4b58-b661-804a446c1382"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:53:15.978569 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.978538    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7584cc14-1a27-4b58-b661-804a446c1382-istio-podinfo" pod "7584cc14-1a27-4b58-b661-804a446c1382" (UID: "7584cc14-1a27-4b58-b661-804a446c1382"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 05:53:15.978956 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.978929    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7584cc14-1a27-4b58-b661-804a446c1382-istio-token" pod "7584cc14-1a27-4b58-b661-804a446c1382" (UID: "7584cc14-1a27-4b58-b661-804a446c1382"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:53:15.979256 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:15.979234    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7584cc14-1a27-4b58-b661-804a446c1382-kube-api-access-z8h2h" pod "7584cc14-1a27-4b58-b661-804a446c1382" (UID: "7584cc14-1a27-4b58-b661-804a446c1382"). InnerVolumeSpecName "kube-api-access-z8h2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:53:16.076798 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.076762    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/402a4249-e41e-47db-ab46-977480475a50-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/402a4249-e41e-47db-ab46-977480475a50-istiod-ca-cert\") pod \"402a4249-e41e-47db-ab46-977480475a50\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") "
Nov 01 05:53:16.076962 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.076882    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-workload-socket\") pod \"402a4249-e41e-47db-ab46-977480475a50\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") "
Nov 01 05:53:16.076962 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.076937    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/402a4249-e41e-47db-ab46-977480475a50-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/402a4249-e41e-47db-ab46-977480475a50-istio-podinfo\") pod \"402a4249-e41e-47db-ab46-977480475a50\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") "
Nov 01 05:53:16.077038 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.076969    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-istio-envoy\") pod \"402a4249-e41e-47db-ab46-977480475a50\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") "
Nov 01 05:53:16.077072 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.077054    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-istio-data\") pod \"402a4249-e41e-47db-ab46-977480475a50\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") "
Nov 01 05:53:16.077107 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.077086    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/402a4249-e41e-47db-ab46-977480475a50-istio-token\" (UniqueName: \"kubernetes.io/projected/402a4249-e41e-47db-ab46-977480475a50-istio-token\") pod \"402a4249-e41e-47db-ab46-977480475a50\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") "
Nov 01 05:53:16.077156 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.077139    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/402a4249-e41e-47db-ab46-977480475a50-kube-api-access-vxhmt\" (UniqueName: \"kubernetes.io/projected/402a4249-e41e-47db-ab46-977480475a50-kube-api-access-vxhmt\") pod \"402a4249-e41e-47db-ab46-977480475a50\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") "
Nov 01 05:53:16.077227 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.077211    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-workload-certs\") pod \"402a4249-e41e-47db-ab46-977480475a50\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") "
Nov 01 05:53:16.077227 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.077215    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/402a4249-e41e-47db-ab46-977480475a50-istiod-ca-cert" pod "402a4249-e41e-47db-ab46-977480475a50" (UID: "402a4249-e41e-47db-ab46-977480475a50"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:53:16.077344 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.077250    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-credential-socket\") pod \"402a4249-e41e-47db-ab46-977480475a50\" (UID: \"402a4249-e41e-47db-ab46-977480475a50\") "
Nov 01 05:53:16.077427 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.077400    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-istio-data" pod "402a4249-e41e-47db-ab46-977480475a50" (UID: "402a4249-e41e-47db-ab46-977480475a50"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:53:16.077522 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.077496    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-workload-socket" pod "402a4249-e41e-47db-ab46-977480475a50" (UID: "402a4249-e41e-47db-ab46-977480475a50"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:53:16.077772 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.077721    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-workload-certs" pod "402a4249-e41e-47db-ab46-977480475a50" (UID: "402a4249-e41e-47db-ab46-977480475a50"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:53:16.077898 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.077777    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8h2h\" (UniqueName: \"kubernetes.io/projected/7584cc14-1a27-4b58-b661-804a446c1382-kube-api-access-z8h2h\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:16.077898 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.077835    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7584cc14-1a27-4b58-b661-804a446c1382-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:16.077898 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.077848    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/402a4249-e41e-47db-ab46-977480475a50-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:16.077898 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.077862    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:16.077898 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.077875    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:16.077898 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.077888    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7584cc14-1a27-4b58-b661-804a446c1382-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:16.078375 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.077903    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:16.078375 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.077915    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-credential-socket" pod "402a4249-e41e-47db-ab46-977480475a50" (UID: "402a4249-e41e-47db-ab46-977480475a50"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:53:16.078375 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.077945    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7584cc14-1a27-4b58-b661-804a446c1382-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:16.078375 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.078039    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7584cc14-1a27-4b58-b661-804a446c1382-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:16.079986 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.079960    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/402a4249-e41e-47db-ab46-977480475a50-kube-api-access-vxhmt" pod "402a4249-e41e-47db-ab46-977480475a50" (UID: "402a4249-e41e-47db-ab46-977480475a50"). InnerVolumeSpecName "kube-api-access-vxhmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:53:16.080147 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.080117    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/402a4249-e41e-47db-ab46-977480475a50-istio-token" pod "402a4249-e41e-47db-ab46-977480475a50" (UID: "402a4249-e41e-47db-ab46-977480475a50"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:53:16.080227 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.080198    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/402a4249-e41e-47db-ab46-977480475a50-istio-podinfo" pod "402a4249-e41e-47db-ab46-977480475a50" (UID: "402a4249-e41e-47db-ab46-977480475a50"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 05:53:16.080831 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.080811    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-istio-envoy" pod "402a4249-e41e-47db-ab46-977480475a50" (UID: "402a4249-e41e-47db-ab46-977480475a50"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:53:16.179321 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.179199    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxhmt\" (UniqueName: \"kubernetes.io/projected/402a4249-e41e-47db-ab46-977480475a50-kube-api-access-vxhmt\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:16.179321 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.179240    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:16.179321 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.179255    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:16.179321 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.179270    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/402a4249-e41e-47db-ab46-977480475a50-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:16.179321 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.179283    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/402a4249-e41e-47db-ab46-977480475a50-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:16.179321 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.179297    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/402a4249-e41e-47db-ab46-977480475a50-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:53:16.683872 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.683107    4864 generic.go:356] "Generic (PLEG): container finished" podID="402a4249-e41e-47db-ab46-977480475a50" containerID="ec6ff5039a608a041d4adf50fb3e89a1d601e463900ec15afce09d0939ed809e" exitCode=0
Nov 01 05:53:16.683872 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.683300    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h" event={"ID":"402a4249-e41e-47db-ab46-977480475a50","Type":"ContainerDied","Data":"ec6ff5039a608a041d4adf50fb3e89a1d601e463900ec15afce09d0939ed809e"}
Nov 01 05:53:16.683872 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.683395    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h" event={"ID":"402a4249-e41e-47db-ab46-977480475a50","Type":"ContainerDied","Data":"6f27c8a4bf2ff2f70b172f7bf93306bc53adc8822c95cbd9d0b9f26bb5eed168"}
Nov 01 05:53:16.683872 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.683432    4864 scope.go:122] "RemoveContainer" containerID="ec6ff5039a608a041d4adf50fb3e89a1d601e463900ec15afce09d0939ed809e"
Nov 01 05:53:16.683872 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.683626    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"
Nov 01 05:53:16.686459 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.686426    4864 generic.go:356] "Generic (PLEG): container finished" podID="7584cc14-1a27-4b58-b661-804a446c1382" containerID="717d9166d9877d8dcc228cc407ba024f5b162095c491e44d5baa2c86ee5db8b3" exitCode=0
Nov 01 05:53:16.686607 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.686586    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"
Nov 01 05:53:16.686669 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.686625    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh" event={"ID":"7584cc14-1a27-4b58-b661-804a446c1382","Type":"ContainerDied","Data":"717d9166d9877d8dcc228cc407ba024f5b162095c491e44d5baa2c86ee5db8b3"}
Nov 01 05:53:16.686717 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.686672    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh" event={"ID":"7584cc14-1a27-4b58-b661-804a446c1382","Type":"ContainerDied","Data":"fe6c5bb47c7497f5021c1b96e4b635ac99dc708a4d6fea57be3a7a094d080bac"}
Nov 01 05:53:16.700397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.700366    4864 scope.go:122] "RemoveContainer" containerID="ec6ff5039a608a041d4adf50fb3e89a1d601e463900ec15afce09d0939ed809e"
Nov 01 05:53:16.701116 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:53:16.700804    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec6ff5039a608a041d4adf50fb3e89a1d601e463900ec15afce09d0939ed809e\": container with ID starting with ec6ff5039a608a041d4adf50fb3e89a1d601e463900ec15afce09d0939ed809e not found: ID does not exist" containerID="ec6ff5039a608a041d4adf50fb3e89a1d601e463900ec15afce09d0939ed809e"
Nov 01 05:53:16.701116 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.700843    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec6ff5039a608a041d4adf50fb3e89a1d601e463900ec15afce09d0939ed809e"} err="failed to get container status \"ec6ff5039a608a041d4adf50fb3e89a1d601e463900ec15afce09d0939ed809e\": rpc error: code = NotFound desc = could not find container \"ec6ff5039a608a041d4adf50fb3e89a1d601e463900ec15afce09d0939ed809e\": container with ID starting with ec6ff5039a608a041d4adf50fb3e89a1d601e463900ec15afce09d0939ed809e not found: ID does not exist"
Nov 01 05:53:16.701116 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.700867    4864 scope.go:122] "RemoveContainer" containerID="717d9166d9877d8dcc228cc407ba024f5b162095c491e44d5baa2c86ee5db8b3"
Nov 01 05:53:16.714283 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.714256    4864 scope.go:122] "RemoveContainer" containerID="717d9166d9877d8dcc228cc407ba024f5b162095c491e44d5baa2c86ee5db8b3"
Nov 01 05:53:16.714623 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:53:16.714600    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"717d9166d9877d8dcc228cc407ba024f5b162095c491e44d5baa2c86ee5db8b3\": container with ID starting with 717d9166d9877d8dcc228cc407ba024f5b162095c491e44d5baa2c86ee5db8b3 not found: ID does not exist" containerID="717d9166d9877d8dcc228cc407ba024f5b162095c491e44d5baa2c86ee5db8b3"
Nov 01 05:53:16.714734 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.714628    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"717d9166d9877d8dcc228cc407ba024f5b162095c491e44d5baa2c86ee5db8b3"} err="failed to get container status \"717d9166d9877d8dcc228cc407ba024f5b162095c491e44d5baa2c86ee5db8b3\": rpc error: code = NotFound desc = could not find container \"717d9166d9877d8dcc228cc407ba024f5b162095c491e44d5baa2c86ee5db8b3\": container with ID starting with 717d9166d9877d8dcc228cc407ba024f5b162095c491e44d5baa2c86ee5db8b3 not found: ID does not exist"
Nov 01 05:53:16.847887 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.847857    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"]
Nov 01 05:53:16.898748 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.898717    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh"]
Nov 01 05:53:16.959244 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.959164    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"]
Nov 01 05:53:16.965522 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:16.965496    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h"]
Nov 01 05:53:17.247587 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:17.247506    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="402a4249-e41e-47db-ab46-977480475a50" path="/var/lib/kubelet/pods/402a4249-e41e-47db-ab46-977480475a50/volumes"
Nov 01 05:53:17.247985 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:53:17.247970    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7584cc14-1a27-4b58-b661-804a446c1382" path="/var/lib/kubelet/pods/7584cc14-1a27-4b58-b661-804a446c1382/volumes"
Nov 01 05:53:22.564010 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:53:22.563959    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:53:52.526016 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:53:52.525981    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:54:22.525041 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:54:22.525004    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:54:52.524600 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:54:52.524525    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:55:22.525023 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:55:22.524978    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:55:27.700762 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:27.700725    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"]
Nov 01 05:55:27.701358 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:27.700960    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489" podUID="8626eef5-3aa4-487c-aeb1-488f32b4712c" containerName="maas-api" containerID="cri-o://063352f4f46b4680e6d263fcc220c33d68b53236bc294410c1f0a06c8e92e067" gracePeriod=30
Nov 01 05:55:27.993527 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:27.993482    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"
Nov 01 05:55:28.022558 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:28.022511    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8626eef5-3aa4-487c-aeb1-488f32b4712c-kube-api-access-rfqt6\" (UniqueName: \"kubernetes.io/projected/8626eef5-3aa4-487c-aeb1-488f32b4712c-kube-api-access-rfqt6\") pod \"8626eef5-3aa4-487c-aeb1-488f32b4712c\" (UID: \"8626eef5-3aa4-487c-aeb1-488f32b4712c\") "
Nov 01 05:55:28.022819 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:28.022628    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8626eef5-3aa4-487c-aeb1-488f32b4712c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8626eef5-3aa4-487c-aeb1-488f32b4712c-metrics-certs\") pod \"8626eef5-3aa4-487c-aeb1-488f32b4712c\" (UID: \"8626eef5-3aa4-487c-aeb1-488f32b4712c\") "
Nov 01 05:55:28.022819 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:28.022734    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8626eef5-3aa4-487c-aeb1-488f32b4712c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8626eef5-3aa4-487c-aeb1-488f32b4712c-maas-api-tls\") pod \"8626eef5-3aa4-487c-aeb1-488f32b4712c\" (UID: \"8626eef5-3aa4-487c-aeb1-488f32b4712c\") "
Nov 01 05:55:28.022965 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:28.022888    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8626eef5-3aa4-487c-aeb1-488f32b4712c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8626eef5-3aa4-487c-aeb1-488f32b4712c-openshift-service-ca\") pod \"8626eef5-3aa4-487c-aeb1-488f32b4712c\" (UID: \"8626eef5-3aa4-487c-aeb1-488f32b4712c\") "
Nov 01 05:55:28.023484 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:28.023428    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8626eef5-3aa4-487c-aeb1-488f32b4712c-openshift-service-ca" pod "8626eef5-3aa4-487c-aeb1-488f32b4712c" (UID: "8626eef5-3aa4-487c-aeb1-488f32b4712c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:55:28.025646 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:28.025609    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8626eef5-3aa4-487c-aeb1-488f32b4712c-maas-api-tls" pod "8626eef5-3aa4-487c-aeb1-488f32b4712c" (UID: "8626eef5-3aa4-487c-aeb1-488f32b4712c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:55:28.025769 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:28.025653    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8626eef5-3aa4-487c-aeb1-488f32b4712c-metrics-certs" pod "8626eef5-3aa4-487c-aeb1-488f32b4712c" (UID: "8626eef5-3aa4-487c-aeb1-488f32b4712c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:55:28.026232 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:28.026206    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8626eef5-3aa4-487c-aeb1-488f32b4712c-kube-api-access-rfqt6" pod "8626eef5-3aa4-487c-aeb1-488f32b4712c" (UID: "8626eef5-3aa4-487c-aeb1-488f32b4712c"). InnerVolumeSpecName "kube-api-access-rfqt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:55:28.124327 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:28.124240    4864 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8626eef5-3aa4-487c-aeb1-488f32b4712c-openshift-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:55:28.124327 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:28.124276    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfqt6\" (UniqueName: \"kubernetes.io/projected/8626eef5-3aa4-487c-aeb1-488f32b4712c-kube-api-access-rfqt6\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:55:28.124327 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:28.124291    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8626eef5-3aa4-487c-aeb1-488f32b4712c-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:55:28.124327 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:28.124301    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8626eef5-3aa4-487c-aeb1-488f32b4712c-maas-api-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:55:28.764240 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:28.764202    4864 generic.go:356] "Generic (PLEG): container finished" podID="8626eef5-3aa4-487c-aeb1-488f32b4712c" containerID="063352f4f46b4680e6d263fcc220c33d68b53236bc294410c1f0a06c8e92e067" exitCode=0
Nov 01 05:55:28.764701 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:28.764248    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"
Nov 01 05:55:28.764701 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:28.764293    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489" event={"ID":"8626eef5-3aa4-487c-aeb1-488f32b4712c","Type":"ContainerDied","Data":"063352f4f46b4680e6d263fcc220c33d68b53236bc294410c1f0a06c8e92e067"}
Nov 01 05:55:28.764701 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:28.764351    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489" event={"ID":"8626eef5-3aa4-487c-aeb1-488f32b4712c","Type":"ContainerDied","Data":"89cb0902b0e3edbf43643ce978747bf8b7c160bf994a7ed06e62d090f3056ee0"}
Nov 01 05:55:28.764701 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:28.764380    4864 scope.go:122] "RemoveContainer" containerID="063352f4f46b4680e6d263fcc220c33d68b53236bc294410c1f0a06c8e92e067"
Nov 01 05:55:28.777499 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:28.777475    4864 scope.go:122] "RemoveContainer" containerID="063352f4f46b4680e6d263fcc220c33d68b53236bc294410c1f0a06c8e92e067"
Nov 01 05:55:28.777872 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:55:28.777840    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"063352f4f46b4680e6d263fcc220c33d68b53236bc294410c1f0a06c8e92e067\": container with ID starting with 063352f4f46b4680e6d263fcc220c33d68b53236bc294410c1f0a06c8e92e067 not found: ID does not exist" containerID="063352f4f46b4680e6d263fcc220c33d68b53236bc294410c1f0a06c8e92e067"
Nov 01 05:55:28.777967 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:28.777883    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"063352f4f46b4680e6d263fcc220c33d68b53236bc294410c1f0a06c8e92e067"} err="failed to get container status \"063352f4f46b4680e6d263fcc220c33d68b53236bc294410c1f0a06c8e92e067\": rpc error: code = NotFound desc = could not find container \"063352f4f46b4680e6d263fcc220c33d68b53236bc294410c1f0a06c8e92e067\": container with ID starting with 063352f4f46b4680e6d263fcc220c33d68b53236bc294410c1f0a06c8e92e067 not found: ID does not exist"
Nov 01 05:55:28.928219 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:28.928180    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"]
Nov 01 05:55:28.932308 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:28.932280    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-5d585e-64bc54f899-jf489"]
Nov 01 05:55:29.247370 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:29.247336    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8626eef5-3aa4-487c-aeb1-488f32b4712c" path="/var/lib/kubelet/pods/8626eef5-3aa4-487c-aeb1-488f32b4712c/volumes"
Nov 01 05:55:35.963585 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:35.963551    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"]
Nov 01 05:55:35.964180 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:35.963829    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr" podUID="d7238a45-e742-44f5-a321-1fd4d5daed4d" containerName="istio-proxy" containerID="cri-o://2a816faada5fcdc5c156bc114d479f6305c31cc3e6fdc64b0228a34f9abf934a" gracePeriod=30
Nov 01 05:55:37.667017 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:37.666983    4864 patch_prober.go:29] interesting pod/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 05:55:37.667446 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:37.667051    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr" podUID="d7238a45-e742-44f5-a321-1fd4d5daed4d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 05:55:41.233357 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.233301    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:55:41.317176 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.317144    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-workload-socket\") pod \"d7238a45-e742-44f5-a321-1fd4d5daed4d\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") "
Nov 01 05:55:41.317372 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.317210    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-workload-certs\") pod \"d7238a45-e742-44f5-a321-1fd4d5daed4d\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") "
Nov 01 05:55:41.317372 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.317245    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-token\" (UniqueName: \"kubernetes.io/projected/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-token\") pod \"d7238a45-e742-44f5-a321-1fd4d5daed4d\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") "
Nov 01 05:55:41.317372 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.317320    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-data\") pod \"d7238a45-e742-44f5-a321-1fd4d5daed4d\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") "
Nov 01 05:55:41.317372 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.317348    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7238a45-e742-44f5-a321-1fd4d5daed4d-kube-api-access-dfnmj\" (UniqueName: \"kubernetes.io/projected/d7238a45-e742-44f5-a321-1fd4d5daed4d-kube-api-access-dfnmj\") pod \"d7238a45-e742-44f5-a321-1fd4d5daed4d\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") "
Nov 01 05:55:41.317584 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.317377    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7238a45-e742-44f5-a321-1fd4d5daed4d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7238a45-e742-44f5-a321-1fd4d5daed4d-istiod-ca-cert\") pod \"d7238a45-e742-44f5-a321-1fd4d5daed4d\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") "
Nov 01 05:55:41.317584 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.317457    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-workload-socket" pod "d7238a45-e742-44f5-a321-1fd4d5daed4d" (UID: "d7238a45-e742-44f5-a321-1fd4d5daed4d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:55:41.317584 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.317560    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-credential-socket\") pod \"d7238a45-e742-44f5-a321-1fd4d5daed4d\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") "
Nov 01 05:55:41.317750 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.317585    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-workload-certs" pod "d7238a45-e742-44f5-a321-1fd4d5daed4d" (UID: "d7238a45-e742-44f5-a321-1fd4d5daed4d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:55:41.317750 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.317713    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-envoy\") pod \"d7238a45-e742-44f5-a321-1fd4d5daed4d\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") "
Nov 01 05:55:41.317947 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.317763    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-podinfo\") pod \"d7238a45-e742-44f5-a321-1fd4d5daed4d\" (UID: \"d7238a45-e742-44f5-a321-1fd4d5daed4d\") "
Nov 01 05:55:41.317947 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.317763    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-credential-socket" pod "d7238a45-e742-44f5-a321-1fd4d5daed4d" (UID: "d7238a45-e742-44f5-a321-1fd4d5daed4d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:55:41.318057 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.317842    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7238a45-e742-44f5-a321-1fd4d5daed4d-istiod-ca-cert" pod "d7238a45-e742-44f5-a321-1fd4d5daed4d" (UID: "d7238a45-e742-44f5-a321-1fd4d5daed4d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:55:41.318725 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.318310    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-data" pod "d7238a45-e742-44f5-a321-1fd4d5daed4d" (UID: "d7238a45-e742-44f5-a321-1fd4d5daed4d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:55:41.319069 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.319036    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:55:41.319172 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.319069    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:55:41.319172 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.319090    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:55:41.319172 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.319107    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:55:41.319172 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.319124    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7238a45-e742-44f5-a321-1fd4d5daed4d-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:55:41.320113 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.320075    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-token" pod "d7238a45-e742-44f5-a321-1fd4d5daed4d" (UID: "d7238a45-e742-44f5-a321-1fd4d5daed4d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:55:41.320368 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.320345    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7238a45-e742-44f5-a321-1fd4d5daed4d-kube-api-access-dfnmj" pod "d7238a45-e742-44f5-a321-1fd4d5daed4d" (UID: "d7238a45-e742-44f5-a321-1fd4d5daed4d"). InnerVolumeSpecName "kube-api-access-dfnmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:55:41.320478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.320457    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-envoy" pod "d7238a45-e742-44f5-a321-1fd4d5daed4d" (UID: "d7238a45-e742-44f5-a321-1fd4d5daed4d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:55:41.320718 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.320511    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-podinfo" pod "d7238a45-e742-44f5-a321-1fd4d5daed4d" (UID: "d7238a45-e742-44f5-a321-1fd4d5daed4d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 05:55:41.420154 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.420122    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:55:41.420154 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.420150    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:55:41.420154 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.420160    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7238a45-e742-44f5-a321-1fd4d5daed4d-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:55:41.420484 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.420170    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfnmj\" (UniqueName: \"kubernetes.io/projected/d7238a45-e742-44f5-a321-1fd4d5daed4d-kube-api-access-dfnmj\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:55:41.855280 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.855237    4864 generic.go:356] "Generic (PLEG): container finished" podID="d7238a45-e742-44f5-a321-1fd4d5daed4d" containerID="2a816faada5fcdc5c156bc114d479f6305c31cc3e6fdc64b0228a34f9abf934a" exitCode=0
Nov 01 05:55:41.855495 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.855294    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"
Nov 01 05:55:41.855495 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.855304    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr" event={"ID":"d7238a45-e742-44f5-a321-1fd4d5daed4d","Type":"ContainerDied","Data":"2a816faada5fcdc5c156bc114d479f6305c31cc3e6fdc64b0228a34f9abf934a"}
Nov 01 05:55:41.855495 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.855358    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr" event={"ID":"d7238a45-e742-44f5-a321-1fd4d5daed4d","Type":"ContainerDied","Data":"55d7da79c963023b4e3684db582acec65d6fe66400874d8e7e3c4b0883365fba"}
Nov 01 05:55:41.855495 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.855381    4864 scope.go:122] "RemoveContainer" containerID="2a816faada5fcdc5c156bc114d479f6305c31cc3e6fdc64b0228a34f9abf934a"
Nov 01 05:55:41.869262 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.869236    4864 scope.go:122] "RemoveContainer" containerID="e84ca85f13aa56800ed9c17dbf626f5b26ad1ffe58bb0d5ac6aff202d3ce9afb"
Nov 01 05:55:41.881053 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.881029    4864 scope.go:122] "RemoveContainer" containerID="2a816faada5fcdc5c156bc114d479f6305c31cc3e6fdc64b0228a34f9abf934a"
Nov 01 05:55:41.881374 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:55:41.881346    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a816faada5fcdc5c156bc114d479f6305c31cc3e6fdc64b0228a34f9abf934a\": container with ID starting with 2a816faada5fcdc5c156bc114d479f6305c31cc3e6fdc64b0228a34f9abf934a not found: ID does not exist" containerID="2a816faada5fcdc5c156bc114d479f6305c31cc3e6fdc64b0228a34f9abf934a"
Nov 01 05:55:41.881451 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.881391    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a816faada5fcdc5c156bc114d479f6305c31cc3e6fdc64b0228a34f9abf934a"} err="failed to get container status \"2a816faada5fcdc5c156bc114d479f6305c31cc3e6fdc64b0228a34f9abf934a\": rpc error: code = NotFound desc = could not find container \"2a816faada5fcdc5c156bc114d479f6305c31cc3e6fdc64b0228a34f9abf934a\": container with ID starting with 2a816faada5fcdc5c156bc114d479f6305c31cc3e6fdc64b0228a34f9abf934a not found: ID does not exist"
Nov 01 05:55:41.881451 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.881427    4864 scope.go:122] "RemoveContainer" containerID="e84ca85f13aa56800ed9c17dbf626f5b26ad1ffe58bb0d5ac6aff202d3ce9afb"
Nov 01 05:55:41.881704 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:55:41.881683    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e84ca85f13aa56800ed9c17dbf626f5b26ad1ffe58bb0d5ac6aff202d3ce9afb\": container with ID starting with e84ca85f13aa56800ed9c17dbf626f5b26ad1ffe58bb0d5ac6aff202d3ce9afb not found: ID does not exist" containerID="e84ca85f13aa56800ed9c17dbf626f5b26ad1ffe58bb0d5ac6aff202d3ce9afb"
Nov 01 05:55:41.881768 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:41.881709    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e84ca85f13aa56800ed9c17dbf626f5b26ad1ffe58bb0d5ac6aff202d3ce9afb"} err="failed to get container status \"e84ca85f13aa56800ed9c17dbf626f5b26ad1ffe58bb0d5ac6aff202d3ce9afb\": rpc error: code = NotFound desc = could not find container \"e84ca85f13aa56800ed9c17dbf626f5b26ad1ffe58bb0d5ac6aff202d3ce9afb\": container with ID starting with e84ca85f13aa56800ed9c17dbf626f5b26ad1ffe58bb0d5ac6aff202d3ce9afb not found: ID does not exist"
Nov 01 05:55:42.022306 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:42.022271    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"]
Nov 01 05:55:42.072051 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:42.072015    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr"]
Nov 01 05:55:43.248998 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:43.248964    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7238a45-e742-44f5-a321-1fd4d5daed4d" path="/var/lib/kubelet/pods/d7238a45-e742-44f5-a321-1fd4d5daed4d/volumes"
Nov 01 05:55:44.165778 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.165742    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"]
Nov 01 05:55:44.166285 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.166270    4864 cpu_manager.go:414] "containerMap: removing container" podUID="d8022115-881a-40ff-bab0-cfead6161302" containerName="maas-api"
Nov 01 05:55:44.166330 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.166287    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d8022115-881a-40ff-bab0-cfead6161302" containerName="maas-api"
Nov 01 05:55:44.166330 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.166305    4864 cpu_manager.go:414] "containerMap: removing container" podUID="7584cc14-1a27-4b58-b661-804a446c1382" containerName="istio-proxy"
Nov 01 05:55:44.166330 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.166310    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7584cc14-1a27-4b58-b661-804a446c1382" containerName="istio-proxy"
Nov 01 05:55:44.166330 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.166325    4864 cpu_manager.go:414] "containerMap: removing container" podUID="d7238a45-e742-44f5-a321-1fd4d5daed4d" containerName="istio-proxy"
Nov 01 05:55:44.166330 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.166330    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7238a45-e742-44f5-a321-1fd4d5daed4d" containerName="istio-proxy"
Nov 01 05:55:44.166487 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.166345    4864 cpu_manager.go:414] "containerMap: removing container" podUID="402a4249-e41e-47db-ab46-977480475a50" containerName="istio-proxy"
Nov 01 05:55:44.166487 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.166350    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="402a4249-e41e-47db-ab46-977480475a50" containerName="istio-proxy"
Nov 01 05:55:44.166487 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.166366    4864 cpu_manager.go:414] "containerMap: removing container" podUID="8626eef5-3aa4-487c-aeb1-488f32b4712c" containerName="maas-api"
Nov 01 05:55:44.166487 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.166370    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8626eef5-3aa4-487c-aeb1-488f32b4712c" containerName="maas-api"
Nov 01 05:55:44.166487 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.166448    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="7584cc14-1a27-4b58-b661-804a446c1382" containerName="istio-proxy"
Nov 01 05:55:44.166487 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.166459    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7238a45-e742-44f5-a321-1fd4d5daed4d" containerName="istio-proxy"
Nov 01 05:55:44.166487 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.166466    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7238a45-e742-44f5-a321-1fd4d5daed4d" containerName="istio-proxy"
Nov 01 05:55:44.166487 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.166472    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="402a4249-e41e-47db-ab46-977480475a50" containerName="istio-proxy"
Nov 01 05:55:44.166487 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.166480    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="8626eef5-3aa4-487c-aeb1-488f32b4712c" containerName="maas-api"
Nov 01 05:55:44.166487 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.166485    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="d8022115-881a-40ff-bab0-cfead6161302" containerName="maas-api"
Nov 01 05:55:44.171282 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.171257    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:44.250347 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.250313    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wgxt\" (UniqueName: \"kubernetes.io/projected/f8d428c3-a558-4339-8dde-4ba23ab874a2-kube-api-access-4wgxt\") pod \"noten-model-40af1b-kserve-76546979c7-rx4pr\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") " pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:44.250742 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.250366    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-tmp-dir\") pod \"noten-model-40af1b-kserve-76546979c7-rx4pr\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") " pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:44.250742 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.250424    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8d428c3-a558-4339-8dde-4ba23ab874a2-tls-certs\") pod \"noten-model-40af1b-kserve-76546979c7-rx4pr\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") " pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:44.250742 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.250497    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-dshm\") pod \"noten-model-40af1b-kserve-76546979c7-rx4pr\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") " pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:44.250742 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.250553    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-home\") pod \"noten-model-40af1b-kserve-76546979c7-rx4pr\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") " pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:44.250742 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.250696    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-model-cache\") pod \"noten-model-40af1b-kserve-76546979c7-rx4pr\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") " pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:44.273766 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.273729    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-40af1b\"/\"openshift-service-ca.crt\""
Nov 01 05:55:44.273766 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.273742    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-40af1b\"/\"kube-root-ca.crt\""
Nov 01 05:55:44.274067 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.274044    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-40af1b\"/\"noten-model-40af1b-kserve-self-signed-certs\""
Nov 01 05:55:44.274513 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.274491    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-40af1b\"/\"default-dockercfg-vqhsm\""
Nov 01 05:55:44.277736 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.277713    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"]
Nov 01 05:55:44.351833 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.351772    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-tmp-dir\") pod \"noten-model-40af1b-kserve-76546979c7-rx4pr\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") " pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:44.351833 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.351824    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8d428c3-a558-4339-8dde-4ba23ab874a2-tls-certs\") pod \"noten-model-40af1b-kserve-76546979c7-rx4pr\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") " pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:44.352048 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.351851    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-dshm\") pod \"noten-model-40af1b-kserve-76546979c7-rx4pr\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") " pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:44.352048 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.351912    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-home\") pod \"noten-model-40af1b-kserve-76546979c7-rx4pr\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") " pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:44.352150 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.352050    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-model-cache\") pod \"noten-model-40af1b-kserve-76546979c7-rx4pr\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") " pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:44.352150 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.352122    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wgxt\" (UniqueName: \"kubernetes.io/projected/f8d428c3-a558-4339-8dde-4ba23ab874a2-kube-api-access-4wgxt\") pod \"noten-model-40af1b-kserve-76546979c7-rx4pr\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") " pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:44.352276 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.352258    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-tmp-dir\") pod \"noten-model-40af1b-kserve-76546979c7-rx4pr\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") " pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:44.352356 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.352337    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-home\") pod \"noten-model-40af1b-kserve-76546979c7-rx4pr\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") " pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:44.352420 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.352410    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-model-cache\") pod \"noten-model-40af1b-kserve-76546979c7-rx4pr\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") " pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:44.354478 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.354449    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-dshm\") pod \"noten-model-40af1b-kserve-76546979c7-rx4pr\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") " pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:44.354707 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.354690    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8d428c3-a558-4339-8dde-4ba23ab874a2-tls-certs\") pod \"noten-model-40af1b-kserve-76546979c7-rx4pr\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") " pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:44.415717 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.415683    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wgxt\" (UniqueName: \"kubernetes.io/projected/f8d428c3-a558-4339-8dde-4ba23ab874a2-kube-api-access-4wgxt\") pod \"noten-model-40af1b-kserve-76546979c7-rx4pr\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") " pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:44.486327 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.486231    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:44.882549 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:55:44.882513    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8d428c3_a558_4339_8dde_4ba23ab874a2.slice/crio-3c99ec05bcb5b69a1f08eae810d4c810f5888dae0e4424985db344e9909388e5 WatchSource:0}: Error finding container 3c99ec05bcb5b69a1f08eae810d4c810f5888dae0e4424985db344e9909388e5: Status 404 returned error can't find the container with id 3c99ec05bcb5b69a1f08eae810d4c810f5888dae0e4424985db344e9909388e5
Nov 01 05:55:44.882701 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:44.882568    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"]
Nov 01 05:55:45.892373 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:45.892327    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr" event={"ID":"f8d428c3-a558-4339-8dde-4ba23ab874a2","Type":"ContainerStarted","Data":"0e89be52ad2f5ca06c431bb172e78b55ea90b5b79a857e78afbc6526c4e320e8"}
Nov 01 05:55:45.892840 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:45.892387    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr" event={"ID":"f8d428c3-a558-4339-8dde-4ba23ab874a2","Type":"ContainerStarted","Data":"3c99ec05bcb5b69a1f08eae810d4c810f5888dae0e4424985db344e9909388e5"}
Nov 01 05:55:46.008097 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:46.008034    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr" podStartSLOduration=2.0080132 podStartE2EDuration="2.0080132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:55:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:55:46.007708202 +0000 UTC m=+3162.898084193" watchObservedRunningTime="2026-11-01 05:55:46.0080132 +0000 UTC m=+3162.898389178"
Nov 01 05:55:46.486802 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:46.486757    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:52.525817 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:55:52.525744    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:55:54.487481 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:54.487440    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:54.500735 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:54.500705    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:55:54.983315 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:55:54.983288    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:56:04.225894 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:04.225868    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"]
Nov 01 05:56:16.585904 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:16.585815    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"]
Nov 01 05:56:16.586506 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:16.586038    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr" podUID="b0e25f67-566f-4d28-a0b5-4df45774fd39" containerName="istio-proxy" containerID="cri-o://ec36ee7ba0e2fa333243fcc17344bf76db2bd049860229d119cf49cf50acb08e" gracePeriod=30
Nov 01 05:56:19.246397 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:19.246353    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr" podUID="f8d428c3-a558-4339-8dde-4ba23ab874a2" containerName="main" containerID="cri-o://0e89be52ad2f5ca06c431bb172e78b55ea90b5b79a857e78afbc6526c4e320e8" gracePeriod=45
Nov 01 05:56:19.525115 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:19.525089    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:56:19.605247 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:19.605218    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-home\" (UniqueName: \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-home\") pod \"f8d428c3-a558-4339-8dde-4ba23ab874a2\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") "
Nov 01 05:56:19.605445 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:19.605280    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8d428c3-a558-4339-8dde-4ba23ab874a2-kube-api-access-4wgxt\" (UniqueName: \"kubernetes.io/projected/f8d428c3-a558-4339-8dde-4ba23ab874a2-kube-api-access-4wgxt\") pod \"f8d428c3-a558-4339-8dde-4ba23ab874a2\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") "
Nov 01 05:56:19.605445 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:19.605305    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-tmp-dir\") pod \"f8d428c3-a558-4339-8dde-4ba23ab874a2\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") "
Nov 01 05:56:19.605445 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:19.605351    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f8d428c3-a558-4339-8dde-4ba23ab874a2-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8d428c3-a558-4339-8dde-4ba23ab874a2-tls-certs\") pod \"f8d428c3-a558-4339-8dde-4ba23ab874a2\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") "
Nov 01 05:56:19.605642 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:19.605457    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-dshm\") pod \"f8d428c3-a558-4339-8dde-4ba23ab874a2\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") "
Nov 01 05:56:19.605642 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:19.605505    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-model-cache\") pod \"f8d428c3-a558-4339-8dde-4ba23ab874a2\" (UID: \"f8d428c3-a558-4339-8dde-4ba23ab874a2\") "
Nov 01 05:56:19.605642 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:19.605549    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-home" pod "f8d428c3-a558-4339-8dde-4ba23ab874a2" (UID: "f8d428c3-a558-4339-8dde-4ba23ab874a2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:56:19.605928 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:19.605902    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-model-cache" pod "f8d428c3-a558-4339-8dde-4ba23ab874a2" (UID: "f8d428c3-a558-4339-8dde-4ba23ab874a2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:56:19.606035 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:19.605980    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-tmp-dir" pod "f8d428c3-a558-4339-8dde-4ba23ab874a2" (UID: "f8d428c3-a558-4339-8dde-4ba23ab874a2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:56:19.606104 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:19.606036    4864 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-model-cache\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:56:19.606104 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:19.606055    4864 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-home\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:56:19.607938 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:19.607915    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-dshm" pod "f8d428c3-a558-4339-8dde-4ba23ab874a2" (UID: "f8d428c3-a558-4339-8dde-4ba23ab874a2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:56:19.608038 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:19.607986    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8d428c3-a558-4339-8dde-4ba23ab874a2-tls-certs" pod "f8d428c3-a558-4339-8dde-4ba23ab874a2" (UID: "f8d428c3-a558-4339-8dde-4ba23ab874a2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:56:19.608123 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:19.608106    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8d428c3-a558-4339-8dde-4ba23ab874a2-kube-api-access-4wgxt" pod "f8d428c3-a558-4339-8dde-4ba23ab874a2" (UID: "f8d428c3-a558-4339-8dde-4ba23ab874a2"). InnerVolumeSpecName "kube-api-access-4wgxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:56:19.706819 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:19.706773    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wgxt\" (UniqueName: \"kubernetes.io/projected/f8d428c3-a558-4339-8dde-4ba23ab874a2-kube-api-access-4wgxt\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:56:19.706819 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:19.706820    4864 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-tmp-dir\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:56:19.706819 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:19.706829    4864 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8d428c3-a558-4339-8dde-4ba23ab874a2-tls-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:56:19.707123 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:19.706836    4864 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8d428c3-a558-4339-8dde-4ba23ab874a2-dshm\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:56:20.153812 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:20.153745    4864 generic.go:356] "Generic (PLEG): container finished" podID="f8d428c3-a558-4339-8dde-4ba23ab874a2" containerID="0e89be52ad2f5ca06c431bb172e78b55ea90b5b79a857e78afbc6526c4e320e8" exitCode=0
Nov 01 05:56:20.154003 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:20.153826    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"
Nov 01 05:56:20.154003 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:20.153829    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr" event={"ID":"f8d428c3-a558-4339-8dde-4ba23ab874a2","Type":"ContainerDied","Data":"0e89be52ad2f5ca06c431bb172e78b55ea90b5b79a857e78afbc6526c4e320e8"}
Nov 01 05:56:20.154003 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:20.153876    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr" event={"ID":"f8d428c3-a558-4339-8dde-4ba23ab874a2","Type":"ContainerDied","Data":"3c99ec05bcb5b69a1f08eae810d4c810f5888dae0e4424985db344e9909388e5"}
Nov 01 05:56:20.154003 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:20.153898    4864 scope.go:122] "RemoveContainer" containerID="0e89be52ad2f5ca06c431bb172e78b55ea90b5b79a857e78afbc6526c4e320e8"
Nov 01 05:56:20.165448 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:20.165428    4864 scope.go:122] "RemoveContainer" containerID="0e89be52ad2f5ca06c431bb172e78b55ea90b5b79a857e78afbc6526c4e320e8"
Nov 01 05:56:20.165765 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:56:20.165748    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e89be52ad2f5ca06c431bb172e78b55ea90b5b79a857e78afbc6526c4e320e8\": container with ID starting with 0e89be52ad2f5ca06c431bb172e78b55ea90b5b79a857e78afbc6526c4e320e8 not found: ID does not exist" containerID="0e89be52ad2f5ca06c431bb172e78b55ea90b5b79a857e78afbc6526c4e320e8"
Nov 01 05:56:20.165839 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:20.165776    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e89be52ad2f5ca06c431bb172e78b55ea90b5b79a857e78afbc6526c4e320e8"} err="failed to get container status \"0e89be52ad2f5ca06c431bb172e78b55ea90b5b79a857e78afbc6526c4e320e8\": rpc error: code = NotFound desc = could not find container \"0e89be52ad2f5ca06c431bb172e78b55ea90b5b79a857e78afbc6526c4e320e8\": container with ID starting with 0e89be52ad2f5ca06c431bb172e78b55ea90b5b79a857e78afbc6526c4e320e8 not found: ID does not exist"
Nov 01 05:56:20.318158 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:20.318122    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"]
Nov 01 05:56:20.367767 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:20.367722    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-40af1b/noten-model-40af1b-kserve-76546979c7-rx4pr"]
Nov 01 05:56:21.248380 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.248349    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f8d428c3-a558-4339-8dde-4ba23ab874a2" path="/var/lib/kubelet/pods/f8d428c3-a558-4339-8dde-4ba23ab874a2/volumes"
Nov 01 05:56:21.850132 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.850107    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:56:21.932245 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.932158    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-token\" (UniqueName: \"kubernetes.io/projected/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-token\") pod \"b0e25f67-566f-4d28-a0b5-4df45774fd39\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") "
Nov 01 05:56:21.932430 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.932262    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b0e25f67-566f-4d28-a0b5-4df45774fd39-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0e25f67-566f-4d28-a0b5-4df45774fd39-istiod-ca-cert\") pod \"b0e25f67-566f-4d28-a0b5-4df45774fd39\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") "
Nov 01 05:56:21.932430 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.932293    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-credential-socket\") pod \"b0e25f67-566f-4d28-a0b5-4df45774fd39\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") "
Nov 01 05:56:21.932430 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.932328    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-workload-socket\") pod \"b0e25f67-566f-4d28-a0b5-4df45774fd39\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") "
Nov 01 05:56:21.932430 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.932381    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-workload-certs\") pod \"b0e25f67-566f-4d28-a0b5-4df45774fd39\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") "
Nov 01 05:56:21.932430 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.932413    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-data\") pod \"b0e25f67-566f-4d28-a0b5-4df45774fd39\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") "
Nov 01 05:56:21.932709 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.932489    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0e25f67-566f-4d28-a0b5-4df45774fd39-kube-api-access-2q25f\" (UniqueName: \"kubernetes.io/projected/b0e25f67-566f-4d28-a0b5-4df45774fd39-kube-api-access-2q25f\") pod \"b0e25f67-566f-4d28-a0b5-4df45774fd39\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") "
Nov 01 05:56:21.932709 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.932526    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-envoy\") pod \"b0e25f67-566f-4d28-a0b5-4df45774fd39\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") "
Nov 01 05:56:21.932709 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.932619    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-workload-socket" pod "b0e25f67-566f-4d28-a0b5-4df45774fd39" (UID: "b0e25f67-566f-4d28-a0b5-4df45774fd39"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:56:21.932709 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.932627    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-podinfo\") pod \"b0e25f67-566f-4d28-a0b5-4df45774fd39\" (UID: \"b0e25f67-566f-4d28-a0b5-4df45774fd39\") "
Nov 01 05:56:21.932709 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.932633    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-credential-socket" pod "b0e25f67-566f-4d28-a0b5-4df45774fd39" (UID: "b0e25f67-566f-4d28-a0b5-4df45774fd39"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:56:21.932709 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.932673    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0e25f67-566f-4d28-a0b5-4df45774fd39-istiod-ca-cert" pod "b0e25f67-566f-4d28-a0b5-4df45774fd39" (UID: "b0e25f67-566f-4d28-a0b5-4df45774fd39"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 05:56:21.933047 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.932853    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-data" pod "b0e25f67-566f-4d28-a0b5-4df45774fd39" (UID: "b0e25f67-566f-4d28-a0b5-4df45774fd39"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:56:21.933047 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.932966    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-workload-certs" pod "b0e25f67-566f-4d28-a0b5-4df45774fd39" (UID: "b0e25f67-566f-4d28-a0b5-4df45774fd39"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:56:21.933362 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.933264    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:56:21.933362 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.933291    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:56:21.933362 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.933305    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:56:21.933362 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.933319    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:56:21.933362 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.933333    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0e25f67-566f-4d28-a0b5-4df45774fd39-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:56:21.934837 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.934813    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-token" pod "b0e25f67-566f-4d28-a0b5-4df45774fd39" (UID: "b0e25f67-566f-4d28-a0b5-4df45774fd39"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:56:21.935033 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.935010    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-envoy" pod "b0e25f67-566f-4d28-a0b5-4df45774fd39" (UID: "b0e25f67-566f-4d28-a0b5-4df45774fd39"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:56:21.935121 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.935100    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-podinfo" pod "b0e25f67-566f-4d28-a0b5-4df45774fd39" (UID: "b0e25f67-566f-4d28-a0b5-4df45774fd39"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 05:56:21.935199 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:21.935181    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0e25f67-566f-4d28-a0b5-4df45774fd39-kube-api-access-2q25f" pod "b0e25f67-566f-4d28-a0b5-4df45774fd39" (UID: "b0e25f67-566f-4d28-a0b5-4df45774fd39"). InnerVolumeSpecName "kube-api-access-2q25f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:56:22.034667 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:22.034637    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2q25f\" (UniqueName: \"kubernetes.io/projected/b0e25f67-566f-4d28-a0b5-4df45774fd39-kube-api-access-2q25f\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:56:22.034667 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:22.034664    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:56:22.034667 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:22.034673    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:56:22.034984 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:22.034681    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0e25f67-566f-4d28-a0b5-4df45774fd39-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:56:22.173128 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:22.173084    4864 generic.go:356] "Generic (PLEG): container finished" podID="b0e25f67-566f-4d28-a0b5-4df45774fd39" containerID="ec36ee7ba0e2fa333243fcc17344bf76db2bd049860229d119cf49cf50acb08e" exitCode=0
Nov 01 05:56:22.173302 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:22.173141    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"
Nov 01 05:56:22.173302 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:22.173130    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr" event={"ID":"b0e25f67-566f-4d28-a0b5-4df45774fd39","Type":"ContainerDied","Data":"ec36ee7ba0e2fa333243fcc17344bf76db2bd049860229d119cf49cf50acb08e"}
Nov 01 05:56:22.173302 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:22.173275    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr" event={"ID":"b0e25f67-566f-4d28-a0b5-4df45774fd39","Type":"ContainerDied","Data":"07f9808d3dcecab7b96330a959a693684fd91ab69fb6655966815a1574144601"}
Nov 01 05:56:22.173495 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:22.173305    4864 scope.go:122] "RemoveContainer" containerID="ec36ee7ba0e2fa333243fcc17344bf76db2bd049860229d119cf49cf50acb08e"
Nov 01 05:56:22.184025 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:22.184002    4864 scope.go:122] "RemoveContainer" containerID="ec36ee7ba0e2fa333243fcc17344bf76db2bd049860229d119cf49cf50acb08e"
Nov 01 05:56:22.184302 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:56:22.184280    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec36ee7ba0e2fa333243fcc17344bf76db2bd049860229d119cf49cf50acb08e\": container with ID starting with ec36ee7ba0e2fa333243fcc17344bf76db2bd049860229d119cf49cf50acb08e not found: ID does not exist" containerID="ec36ee7ba0e2fa333243fcc17344bf76db2bd049860229d119cf49cf50acb08e"
Nov 01 05:56:22.184353 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:22.184317    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec36ee7ba0e2fa333243fcc17344bf76db2bd049860229d119cf49cf50acb08e"} err="failed to get container status \"ec36ee7ba0e2fa333243fcc17344bf76db2bd049860229d119cf49cf50acb08e\": rpc error: code = NotFound desc = could not find container \"ec36ee7ba0e2fa333243fcc17344bf76db2bd049860229d119cf49cf50acb08e\": container with ID starting with ec36ee7ba0e2fa333243fcc17344bf76db2bd049860229d119cf49cf50acb08e not found: ID does not exist"
Nov 01 05:56:22.339521 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:22.339490    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"]
Nov 01 05:56:22.348768 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:22.348735    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr"]
Nov 01 05:56:22.525353 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:56:22.525265    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:56:23.247692 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:23.247660    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b0e25f67-566f-4d28-a0b5-4df45774fd39" path="/var/lib/kubelet/pods/b0e25f67-566f-4d28-a0b5-4df45774fd39/volumes"
Nov 01 05:56:47.824464 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:47.824431    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"]
Nov 01 05:56:47.825001 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:47.824957    4864 cpu_manager.go:414] "containerMap: removing container" podUID="f8d428c3-a558-4339-8dde-4ba23ab874a2" containerName="main"
Nov 01 05:56:47.825001 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:47.824969    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8d428c3-a558-4339-8dde-4ba23ab874a2" containerName="main"
Nov 01 05:56:47.825001 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:47.824986    4864 cpu_manager.go:414] "containerMap: removing container" podUID="b0e25f67-566f-4d28-a0b5-4df45774fd39" containerName="istio-proxy"
Nov 01 05:56:47.825001 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:47.824991    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0e25f67-566f-4d28-a0b5-4df45774fd39" containerName="istio-proxy"
Nov 01 05:56:47.825219 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:47.825025    4864 cpu_manager.go:414] "containerMap: removing container" podUID="d7238a45-e742-44f5-a321-1fd4d5daed4d" containerName="istio-proxy"
Nov 01 05:56:47.825219 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:47.825033    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7238a45-e742-44f5-a321-1fd4d5daed4d" containerName="istio-proxy"
Nov 01 05:56:47.825219 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:47.825151    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8d428c3-a558-4339-8dde-4ba23ab874a2" containerName="main"
Nov 01 05:56:47.825219 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:47.825164    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="b0e25f67-566f-4d28-a0b5-4df45774fd39" containerName="istio-proxy"
Nov 01 05:56:47.828357 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:47.828338    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:47.891286 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:47.891258    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-897c87-openshift-default-dockercfg-tqhjc\""
Nov 01 05:56:47.933203 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:47.933167    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wckjq\" (UniqueName: \"kubernetes.io/projected/ed66905d-dc1c-4969-a140-f04142163049-kube-api-access-wckjq\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:47.933479 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:47.933448    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-istio-envoy\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:47.933634 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:47.933501    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-workload-socket\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:47.933634 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:47.933556    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-workload-certs\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:47.933754 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:47.933633    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-istio-data\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:47.933754 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:47.933658    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed66905d-dc1c-4969-a140-f04142163049-istio-token\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:47.933754 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:47.933704    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-credential-socket\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:47.933754 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:47.933729    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed66905d-dc1c-4969-a140-f04142163049-istio-podinfo\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:47.934175 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:47.933764    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed66905d-dc1c-4969-a140-f04142163049-istiod-ca-cert\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:47.946951 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:47.946918    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"]
Nov 01 05:56:48.035136 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.035091    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-workload-certs\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:48.035342 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.035166    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-istio-data\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:48.035342 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.035190    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed66905d-dc1c-4969-a140-f04142163049-istio-token\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:48.035342 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.035221    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-credential-socket\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:48.035537 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.035411    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed66905d-dc1c-4969-a140-f04142163049-istio-podinfo\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:48.035537 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.035465    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed66905d-dc1c-4969-a140-f04142163049-istiod-ca-cert\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:48.035537 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.035506    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-workload-certs\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:48.035708 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.035544    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-credential-socket\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:48.035708 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.035607    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wckjq\" (UniqueName: \"kubernetes.io/projected/ed66905d-dc1c-4969-a140-f04142163049-kube-api-access-wckjq\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:48.035708 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.035616    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-istio-data\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:48.035708 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.035650    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-istio-envoy\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:48.035708 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.035687    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-workload-socket\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:48.036050 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.036027    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-workload-socket\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:48.036182 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.036128    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed66905d-dc1c-4969-a140-f04142163049-istiod-ca-cert\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:48.038108 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.038086    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed66905d-dc1c-4969-a140-f04142163049-istio-podinfo\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:48.038226 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.038109    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-istio-envoy\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:48.096147 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.096070    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed66905d-dc1c-4969-a140-f04142163049-istio-token\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:48.096322 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.096286    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wckjq\" (UniqueName: \"kubernetes.io/projected/ed66905d-dc1c-4969-a140-f04142163049-kube-api-access-wckjq\") pod \"e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") " pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:48.141654 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.141623    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:48.583555 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.583528    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"]
Nov 01 05:56:48.585089 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:56:48.585051    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded66905d_dc1c_4969_a140_f04142163049.slice/crio-54060c533388c54c4a5bf2aded71151a21076dc85e53c3e098c9c65d066edccd WatchSource:0}: Error finding container 54060c533388c54c4a5bf2aded71151a21076dc85e53c3e098c9c65d066edccd: Status 404 returned error can't find the container with id 54060c533388c54c4a5bf2aded71151a21076dc85e53c3e098c9c65d066edccd
Nov 01 05:56:48.587685 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.587651    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:56:48.587844 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.587731    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:56:48.587844 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:48.587778    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:56:49.367126 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:49.367087    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv" event={"ID":"ed66905d-dc1c-4969-a140-f04142163049","Type":"ContainerStarted","Data":"8b11561e4434013bc75ae1f368b0ef560e6db41c6fc7b44e9ca27821a1444ea0"}
Nov 01 05:56:49.367563 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:49.367164    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv" event={"ID":"ed66905d-dc1c-4969-a140-f04142163049","Type":"ContainerStarted","Data":"54060c533388c54c4a5bf2aded71151a21076dc85e53c3e098c9c65d066edccd"}
Nov 01 05:56:49.540547 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:49.540489    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv" podStartSLOduration=2.540476385 podStartE2EDuration="2.540476385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:56:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:56:49.539057791 +0000 UTC m=+3226.429433773" watchObservedRunningTime="2026-11-01 05:56:49.540476385 +0000 UTC m=+3226.430852355"
Nov 01 05:56:50.141882 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:50.141847    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:50.147578 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:50.147553    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:50.381077 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:50.381045    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:50.382101 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:50.382076    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:56:52.271098 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:52.271066    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"]
Nov 01 05:56:52.275432 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:52.275410    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"
Nov 01 05:56:52.327326 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:52.327299    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-098e4d-serving-cert\""
Nov 01 05:56:52.380053 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:52.380019    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"]
Nov 01 05:56:52.387925 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:52.387901    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61a331da-3dde-4e55-bd6a-047a02b6e94d-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh\" (UID: \"61a331da-3dde-4e55-bd6a-047a02b6e94d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"
Nov 01 05:56:52.388046 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:52.387970    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbqz6\" (UniqueName: \"kubernetes.io/projected/61a331da-3dde-4e55-bd6a-047a02b6e94d-kube-api-access-bbqz6\") pod \"maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh\" (UID: \"61a331da-3dde-4e55-bd6a-047a02b6e94d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"
Nov 01 05:56:52.388120 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:52.388098    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61a331da-3dde-4e55-bd6a-047a02b6e94d-metrics-certs\") pod \"maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh\" (UID: \"61a331da-3dde-4e55-bd6a-047a02b6e94d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"
Nov 01 05:56:52.388187 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:52.388171    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61a331da-3dde-4e55-bd6a-047a02b6e94d-maas-api-tls\") pod \"maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh\" (UID: \"61a331da-3dde-4e55-bd6a-047a02b6e94d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"
Nov 01 05:56:52.489311 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:52.489269    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbqz6\" (UniqueName: \"kubernetes.io/projected/61a331da-3dde-4e55-bd6a-047a02b6e94d-kube-api-access-bbqz6\") pod \"maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh\" (UID: \"61a331da-3dde-4e55-bd6a-047a02b6e94d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"
Nov 01 05:56:52.489485 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:52.489337    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61a331da-3dde-4e55-bd6a-047a02b6e94d-metrics-certs\") pod \"maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh\" (UID: \"61a331da-3dde-4e55-bd6a-047a02b6e94d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"
Nov 01 05:56:52.489485 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:52.489388    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61a331da-3dde-4e55-bd6a-047a02b6e94d-maas-api-tls\") pod \"maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh\" (UID: \"61a331da-3dde-4e55-bd6a-047a02b6e94d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"
Nov 01 05:56:52.489611 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:52.489541    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61a331da-3dde-4e55-bd6a-047a02b6e94d-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh\" (UID: \"61a331da-3dde-4e55-bd6a-047a02b6e94d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"
Nov 01 05:56:52.490320 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:52.490291    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61a331da-3dde-4e55-bd6a-047a02b6e94d-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh\" (UID: \"61a331da-3dde-4e55-bd6a-047a02b6e94d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"
Nov 01 05:56:52.492106 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:52.492086    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61a331da-3dde-4e55-bd6a-047a02b6e94d-metrics-certs\") pod \"maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh\" (UID: \"61a331da-3dde-4e55-bd6a-047a02b6e94d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"
Nov 01 05:56:52.492230 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:52.492213    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61a331da-3dde-4e55-bd6a-047a02b6e94d-maas-api-tls\") pod \"maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh\" (UID: \"61a331da-3dde-4e55-bd6a-047a02b6e94d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"
Nov 01 05:56:52.524209 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:56:52.524139    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:56:52.549448 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:52.549421    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbqz6\" (UniqueName: \"kubernetes.io/projected/61a331da-3dde-4e55-bd6a-047a02b6e94d-kube-api-access-bbqz6\") pod \"maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh\" (UID: \"61a331da-3dde-4e55-bd6a-047a02b6e94d\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"
Nov 01 05:56:52.590534 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:52.590507    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"
Nov 01 05:56:52.969332 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:56:52.969301    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61a331da_3dde_4e55_bd6a_047a02b6e94d.slice/crio-2e6495fc49b58ae4194142c558e925404967eb05aa9a7ac04f7a514307367309 WatchSource:0}: Error finding container 2e6495fc49b58ae4194142c558e925404967eb05aa9a7ac04f7a514307367309: Status 404 returned error can't find the container with id 2e6495fc49b58ae4194142c558e925404967eb05aa9a7ac04f7a514307367309
Nov 01 05:56:52.969704 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:52.969680    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"]
Nov 01 05:56:52.971227 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:52.971203    4864 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 05:56:53.403830 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:53.403798    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh" event={"ID":"61a331da-3dde-4e55-bd6a-047a02b6e94d","Type":"ContainerStarted","Data":"2e6495fc49b58ae4194142c558e925404967eb05aa9a7ac04f7a514307367309"}
Nov 01 05:56:54.415878 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:54.415844    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh" event={"ID":"61a331da-3dde-4e55-bd6a-047a02b6e94d","Type":"ContainerStarted","Data":"f68628b31409b48f062134a5604b721cfca1b4c5b4770cca4aecc864dbad02d6"}
Nov 01 05:56:54.416355 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:54.416030    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"
Nov 01 05:56:54.534998 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:56:54.534939    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh" podStartSLOduration=1.61406702 podStartE2EDuration="2.534923597s" totalImagesPullingTime="920.856577ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:56:54.532066924 +0000 UTC m=+3231.422442899" watchObservedRunningTime="2026-11-01 05:56:54.534923597 +0000 UTC m=+3231.425299573"
Nov 01 05:57:00.434329 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:57:00.434300    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"
Nov 01 05:57:22.524731 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:57:22.524691    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:57:52.523962 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:57:52.523917    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:58:04.069641 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:04.069602    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:58:04.081587 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:04.081560    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:58:04.081927 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:04.081910    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 05:58:04.090133 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:04.090114    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 05:58:22.526191 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:58:22.526147    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:58:28.049807 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:28.049738    4864 generic.go:356] "Generic (PLEG): container finished" podID="ed66905d-dc1c-4969-a140-f04142163049" containerID="8b11561e4434013bc75ae1f368b0ef560e6db41c6fc7b44e9ca27821a1444ea0" exitCode=0
Nov 01 05:58:28.050253 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:28.049833    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv" event={"ID":"ed66905d-dc1c-4969-a140-f04142163049","Type":"ContainerDied","Data":"8b11561e4434013bc75ae1f368b0ef560e6db41c6fc7b44e9ca27821a1444ea0"}
Nov 01 05:58:28.050301 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:28.050258    4864 scope.go:122] "RemoveContainer" containerID="8b11561e4434013bc75ae1f368b0ef560e6db41c6fc7b44e9ca27821a1444ea0"
Nov 01 05:58:28.051872 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:28.051840    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:58:28.051996 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:28.051926    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:58:28.051996 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:28.051974    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:58:29.083810 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:29.083203    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv" event={"ID":"ed66905d-dc1c-4969-a140-f04142163049","Type":"ContainerStarted","Data":"b7783ae1be9cba60a539c87c5fc105eaedf2ab3c845699ff802cda0f695f038c"}
Nov 01 05:58:29.141931 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:29.141895    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:58:29.142111 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:29.142084    4864 patch_prober.go:29] interesting pod/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Nov 01 05:58:29.142168 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:29.142129    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv" podUID="ed66905d-dc1c-4969-a140-f04142163049" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Nov 01 05:58:30.142430 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:30.142392    4864 patch_prober.go:29] interesting pod/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Nov 01 05:58:30.142899 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:30.142447    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv" podUID="ed66905d-dc1c-4969-a140-f04142163049" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Nov 01 05:58:31.142143 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:31.142107    4864 patch_prober.go:29] interesting pod/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Nov 01 05:58:31.142471 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:31.142436    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv" podUID="ed66905d-dc1c-4969-a140-f04142163049" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Nov 01 05:58:32.142961 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:32.142930    4864 patch_prober.go:29] interesting pod/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Nov 01 05:58:32.143368 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:32.142982    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv" podUID="ed66905d-dc1c-4969-a140-f04142163049" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Nov 01 05:58:33.142282 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:33.142239    4864 patch_prober.go:29] interesting pod/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Nov 01 05:58:33.142496 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:33.142313    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv" podUID="ed66905d-dc1c-4969-a140-f04142163049" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Nov 01 05:58:34.142949 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:34.142915    4864 patch_prober.go:29] interesting pod/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Nov 01 05:58:34.143369 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:34.142967    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv" podUID="ed66905d-dc1c-4969-a140-f04142163049" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Nov 01 05:58:35.147535 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:35.147509    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:58:35.148027 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:35.147812    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:58:35.148613 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:58:35.148596    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 05:58:52.526004 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:58:52.525968    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:59:07.588933 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:07.588895    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"]
Nov 01 05:59:07.637941 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:07.637906    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"]
Nov 01 05:59:21.510169 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.510072    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"]
Nov 01 05:59:21.514318 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.514292    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.566195 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.566162    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-fd2672-openshift-default-dockercfg-7b4qq\""
Nov 01 05:59:21.626384 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.625706    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-workload-socket\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.626384 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.625823    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-envoy\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.626384 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.625912    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-token\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.626384 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.625980    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-podinfo\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.626384 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.626050    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-credential-socket\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.626384 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.626083    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-data\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.626384 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.626109    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a179e91d-6c63-4820-8462-cf2ce37aacc9-istiod-ca-cert\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.626384 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.626151    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfsv5\" (UniqueName: \"kubernetes.io/projected/a179e91d-6c63-4820-8462-cf2ce37aacc9-kube-api-access-dfsv5\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.626384 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.626197    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-workload-certs\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.668567 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.668531    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"]
Nov 01 05:59:21.728351 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.728306    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-workload-socket\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.728534 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.728429    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-envoy\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.728534 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.728510    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-token\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.728616 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.728564    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-podinfo\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.728672 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.728645    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-credential-socket\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.728710 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.728694    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-data\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.728750 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.728717    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a179e91d-6c63-4820-8462-cf2ce37aacc9-istiod-ca-cert\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.728820 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.728805    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfsv5\" (UniqueName: \"kubernetes.io/projected/a179e91d-6c63-4820-8462-cf2ce37aacc9-kube-api-access-dfsv5\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.728884 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.728777    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-workload-socket\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.728884 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.728855    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-workload-certs\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.729131 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.729107    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-data\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.729317 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.729297    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-workload-certs\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.729506 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.729358    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-credential-socket\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.730014 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.729991    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a179e91d-6c63-4820-8462-cf2ce37aacc9-istiod-ca-cert\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.731873 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.731848    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-podinfo\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.732097 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.732078    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-envoy\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.790638 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.790532    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-token\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.791170 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.791148    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfsv5\" (UniqueName: \"kubernetes.io/projected/a179e91d-6c63-4820-8462-cf2ce37aacc9-kube-api-access-dfsv5\") pod \"e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") " pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:21.829833 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:21.829764    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:22.271047 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.271013    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"]
Nov 01 05:59:22.271268 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:59:22.271244    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda179e91d_6c63_4820_8462_cf2ce37aacc9.slice/crio-02368783dd5f0d7c967a914b07369578f0532c987421979a04f9079ff6d54f1f WatchSource:0}: Error finding container 02368783dd5f0d7c967a914b07369578f0532c987421979a04f9079ff6d54f1f: Status 404 returned error can't find the container with id 02368783dd5f0d7c967a914b07369578f0532c987421979a04f9079ff6d54f1f
Nov 01 05:59:22.274442 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.274401    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:59:22.274549 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.274487    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:59:22.274549 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.274531    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 05:59:22.462250 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.462208    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5" event={"ID":"a179e91d-6c63-4820-8462-cf2ce37aacc9","Type":"ContainerStarted","Data":"cb7f5272f0d450a913cb8c94fa4d561a460577e1b592d59502eefd04a750bafc"}
Nov 01 05:59:22.462440 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.462271    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5" event={"ID":"a179e91d-6c63-4820-8462-cf2ce37aacc9","Type":"ContainerStarted","Data":"02368783dd5f0d7c967a914b07369578f0532c987421979a04f9079ff6d54f1f"}
Nov 01 05:59:22.530004 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:59:22.529902    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:59:22.586807 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.586186    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5" podStartSLOduration=1.586166479 podStartE2EDuration="1.586166479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:59:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:59:22.580116834 +0000 UTC m=+3379.470492812" watchObservedRunningTime="2026-11-01 05:59:22.586166479 +0000 UTC m=+3379.476542457"
Nov 01 05:59:22.610511 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.610464    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx" podUID="d94a1258-dbb1-44b5-bf57-1b44b66fe286" containerName="main" containerID="cri-o://f64d7e83d7bc3f7ceb844e1ec23b486eb49272a5c50623b99156f59562c601b7" gracePeriod=45
Nov 01 05:59:22.646730 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.646675    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx" podUID="d94a1258-dbb1-44b5-bf57-1b44b66fe286" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": dial tcp 10.132.0.68:8000: connect: connection refused"
Nov 01 05:59:22.659484 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.659438    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk" podUID="b28a4792-6e4e-4f9b-80e0-737dc18ba934" containerName="main" containerID="cri-o://2d915a46cfc677786c61f627e76381ce8080fdb5150570722c35bd363c7ff0f5" gracePeriod=45
Nov 01 05:59:22.829995 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.829904    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:22.927557 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.927528    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:59:22.941661 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.941636    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d94a1258-dbb1-44b5-bf57-1b44b66fe286-kube-api-access-dqb7r\" (UniqueName: \"kubernetes.io/projected/d94a1258-dbb1-44b5-bf57-1b44b66fe286-kube-api-access-dqb7r\") pod \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") "
Nov 01 05:59:22.941815 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.941688    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-model-cache\") pod \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") "
Nov 01 05:59:22.941815 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.941751    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-dshm\") pod \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") "
Nov 01 05:59:22.941946 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.941854    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-tmp-dir\") pod \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") "
Nov 01 05:59:22.941946 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.941882    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-home\" (UniqueName: \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-home\") pod \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") "
Nov 01 05:59:22.942053 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.941950    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d94a1258-dbb1-44b5-bf57-1b44b66fe286-tls-certs\" (UniqueName: \"kubernetes.io/secret/d94a1258-dbb1-44b5-bf57-1b44b66fe286-tls-certs\") pod \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\" (UID: \"d94a1258-dbb1-44b5-bf57-1b44b66fe286\") "
Nov 01 05:59:22.942112 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.942043    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-model-cache" pod "d94a1258-dbb1-44b5-bf57-1b44b66fe286" (UID: "d94a1258-dbb1-44b5-bf57-1b44b66fe286"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:59:22.942254 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.942227    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-home" pod "d94a1258-dbb1-44b5-bf57-1b44b66fe286" (UID: "d94a1258-dbb1-44b5-bf57-1b44b66fe286"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:59:22.942382 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.942292    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-tmp-dir" pod "d94a1258-dbb1-44b5-bf57-1b44b66fe286" (UID: "d94a1258-dbb1-44b5-bf57-1b44b66fe286"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:59:22.942450 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.942437    4864 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-tmp-dir\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:59:22.942506 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.942457    4864 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-home\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:59:22.942506 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.942466    4864 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-model-cache\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:59:22.944314 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.944284    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-dshm" pod "d94a1258-dbb1-44b5-bf57-1b44b66fe286" (UID: "d94a1258-dbb1-44b5-bf57-1b44b66fe286"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:59:22.944547 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.944530    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d94a1258-dbb1-44b5-bf57-1b44b66fe286-kube-api-access-dqb7r" pod "d94a1258-dbb1-44b5-bf57-1b44b66fe286" (UID: "d94a1258-dbb1-44b5-bf57-1b44b66fe286"). InnerVolumeSpecName "kube-api-access-dqb7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:59:22.944609 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.944593    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d94a1258-dbb1-44b5-bf57-1b44b66fe286-tls-certs" pod "d94a1258-dbb1-44b5-bf57-1b44b66fe286" (UID: "d94a1258-dbb1-44b5-bf57-1b44b66fe286"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:59:22.974095 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:22.974074    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:59:23.042922 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.042889    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-home\" (UniqueName: \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-home\") pod \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") "
Nov 01 05:59:23.043120 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.042999    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-dshm\") pod \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") "
Nov 01 05:59:23.043120 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.043044    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-tmp-dir\") pod \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") "
Nov 01 05:59:23.043120 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.043094    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-model-cache\") pod \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") "
Nov 01 05:59:23.043315 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.043130    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b28a4792-6e4e-4f9b-80e0-737dc18ba934-tls-certs\" (UniqueName: \"kubernetes.io/secret/b28a4792-6e4e-4f9b-80e0-737dc18ba934-tls-certs\") pod \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") "
Nov 01 05:59:23.043315 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.043190    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-home" pod "b28a4792-6e4e-4f9b-80e0-737dc18ba934" (UID: "b28a4792-6e4e-4f9b-80e0-737dc18ba934"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:59:23.043315 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.043245    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b28a4792-6e4e-4f9b-80e0-737dc18ba934-kube-api-access-dk6lj\" (UniqueName: \"kubernetes.io/projected/b28a4792-6e4e-4f9b-80e0-737dc18ba934-kube-api-access-dk6lj\") pod \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\" (UID: \"b28a4792-6e4e-4f9b-80e0-737dc18ba934\") "
Nov 01 05:59:23.043492 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.043382    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-tmp-dir" pod "b28a4792-6e4e-4f9b-80e0-737dc18ba934" (UID: "b28a4792-6e4e-4f9b-80e0-737dc18ba934"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:59:23.043577 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.043556    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-model-cache" pod "b28a4792-6e4e-4f9b-80e0-737dc18ba934" (UID: "b28a4792-6e4e-4f9b-80e0-737dc18ba934"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:59:23.043686 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.043666    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqb7r\" (UniqueName: \"kubernetes.io/projected/d94a1258-dbb1-44b5-bf57-1b44b66fe286-kube-api-access-dqb7r\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:59:23.043745 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.043691    4864 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-home\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:59:23.043745 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.043704    4864 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d94a1258-dbb1-44b5-bf57-1b44b66fe286-dshm\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:59:23.043745 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.043718    4864 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-tmp-dir\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:59:23.043745 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.043732    4864 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-model-cache\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:59:23.043745 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.043744    4864 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d94a1258-dbb1-44b5-bf57-1b44b66fe286-tls-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:59:23.046170 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.046105    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-dshm" pod "b28a4792-6e4e-4f9b-80e0-737dc18ba934" (UID: "b28a4792-6e4e-4f9b-80e0-737dc18ba934"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 05:59:23.046287 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.046175    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b28a4792-6e4e-4f9b-80e0-737dc18ba934-tls-certs" pod "b28a4792-6e4e-4f9b-80e0-737dc18ba934" (UID: "b28a4792-6e4e-4f9b-80e0-737dc18ba934"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 05:59:23.046350 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.046329    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b28a4792-6e4e-4f9b-80e0-737dc18ba934-kube-api-access-dk6lj" pod "b28a4792-6e4e-4f9b-80e0-737dc18ba934" (UID: "b28a4792-6e4e-4f9b-80e0-737dc18ba934"). InnerVolumeSpecName "kube-api-access-dk6lj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 05:59:23.144997 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.144935    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dk6lj\" (UniqueName: \"kubernetes.io/projected/b28a4792-6e4e-4f9b-80e0-737dc18ba934-kube-api-access-dk6lj\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:59:23.144997 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.144971    4864 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b28a4792-6e4e-4f9b-80e0-737dc18ba934-dshm\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:59:23.144997 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.144986    4864 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b28a4792-6e4e-4f9b-80e0-737dc18ba934-tls-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 05:59:23.472097 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.472062    4864 generic.go:356] "Generic (PLEG): container finished" podID="d94a1258-dbb1-44b5-bf57-1b44b66fe286" containerID="f64d7e83d7bc3f7ceb844e1ec23b486eb49272a5c50623b99156f59562c601b7" exitCode=0
Nov 01 05:59:23.472297 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.472141    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"
Nov 01 05:59:23.472297 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.472148    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx" event={"ID":"d94a1258-dbb1-44b5-bf57-1b44b66fe286","Type":"ContainerDied","Data":"f64d7e83d7bc3f7ceb844e1ec23b486eb49272a5c50623b99156f59562c601b7"}
Nov 01 05:59:23.472297 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.472205    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx" event={"ID":"d94a1258-dbb1-44b5-bf57-1b44b66fe286","Type":"ContainerDied","Data":"8a8d4e39f94220b5e7969bc087908c6c61ec30b16473377f2f1ddd3256bbdced"}
Nov 01 05:59:23.472297 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.472231    4864 scope.go:122] "RemoveContainer" containerID="f64d7e83d7bc3f7ceb844e1ec23b486eb49272a5c50623b99156f59562c601b7"
Nov 01 05:59:23.474363 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.474337    4864 generic.go:356] "Generic (PLEG): container finished" podID="b28a4792-6e4e-4f9b-80e0-737dc18ba934" containerID="2d915a46cfc677786c61f627e76381ce8080fdb5150570722c35bd363c7ff0f5" exitCode=0
Nov 01 05:59:23.475081 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.475060    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"
Nov 01 05:59:23.475725 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.475183    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk" event={"ID":"b28a4792-6e4e-4f9b-80e0-737dc18ba934","Type":"ContainerDied","Data":"2d915a46cfc677786c61f627e76381ce8080fdb5150570722c35bd363c7ff0f5"}
Nov 01 05:59:23.475874 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.475738    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk" event={"ID":"b28a4792-6e4e-4f9b-80e0-737dc18ba934","Type":"ContainerDied","Data":"9a8db4b32e877236690a52bd3f79626d6d8bcc6eaee2c46b8a6931061d5b5b3c"}
Nov 01 05:59:23.484123 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.484104    4864 scope.go:122] "RemoveContainer" containerID="f64d7e83d7bc3f7ceb844e1ec23b486eb49272a5c50623b99156f59562c601b7"
Nov 01 05:59:23.484411 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:59:23.484390    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f64d7e83d7bc3f7ceb844e1ec23b486eb49272a5c50623b99156f59562c601b7\": container with ID starting with f64d7e83d7bc3f7ceb844e1ec23b486eb49272a5c50623b99156f59562c601b7 not found: ID does not exist" containerID="f64d7e83d7bc3f7ceb844e1ec23b486eb49272a5c50623b99156f59562c601b7"
Nov 01 05:59:23.484494 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.484420    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f64d7e83d7bc3f7ceb844e1ec23b486eb49272a5c50623b99156f59562c601b7"} err="failed to get container status \"f64d7e83d7bc3f7ceb844e1ec23b486eb49272a5c50623b99156f59562c601b7\": rpc error: code = NotFound desc = could not find container \"f64d7e83d7bc3f7ceb844e1ec23b486eb49272a5c50623b99156f59562c601b7\": container with ID starting with f64d7e83d7bc3f7ceb844e1ec23b486eb49272a5c50623b99156f59562c601b7 not found: ID does not exist"
Nov 01 05:59:23.484494 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.484438    4864 scope.go:122] "RemoveContainer" containerID="2d915a46cfc677786c61f627e76381ce8080fdb5150570722c35bd363c7ff0f5"
Nov 01 05:59:23.493131 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.493108    4864 scope.go:122] "RemoveContainer" containerID="2d915a46cfc677786c61f627e76381ce8080fdb5150570722c35bd363c7ff0f5"
Nov 01 05:59:23.493387 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:59:23.493369    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d915a46cfc677786c61f627e76381ce8080fdb5150570722c35bd363c7ff0f5\": container with ID starting with 2d915a46cfc677786c61f627e76381ce8080fdb5150570722c35bd363c7ff0f5 not found: ID does not exist" containerID="2d915a46cfc677786c61f627e76381ce8080fdb5150570722c35bd363c7ff0f5"
Nov 01 05:59:23.493444 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.493393    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d915a46cfc677786c61f627e76381ce8080fdb5150570722c35bd363c7ff0f5"} err="failed to get container status \"2d915a46cfc677786c61f627e76381ce8080fdb5150570722c35bd363c7ff0f5\": rpc error: code = NotFound desc = could not find container \"2d915a46cfc677786c61f627e76381ce8080fdb5150570722c35bd363c7ff0f5\": container with ID starting with 2d915a46cfc677786c61f627e76381ce8080fdb5150570722c35bd363c7ff0f5 not found: ID does not exist"
Nov 01 05:59:23.635504 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.635468    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"]
Nov 01 05:59:23.638828 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.638801    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-3d70ff/premium-w2-3d70ff-kserve-c975d498d-fhrkk"]
Nov 01 05:59:23.746734 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.746699    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"]
Nov 01 05:59:23.750834 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.750810    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-3d70ff/distinct-w2-3d70ff-kserve-76fb6fcd79-zctzx"]
Nov 01 05:59:23.836523 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:23.836494    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:24.483483 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:24.483454    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:24.484488 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:24.484469    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 05:59:25.247530 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:25.247494    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b28a4792-6e4e-4f9b-80e0-737dc18ba934" path="/var/lib/kubelet/pods/b28a4792-6e4e-4f9b-80e0-737dc18ba934/volumes"
Nov 01 05:59:25.248023 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:25.248003    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d94a1258-dbb1-44b5-bf57-1b44b66fe286" path="/var/lib/kubelet/pods/d94a1258-dbb1-44b5-bf57-1b44b66fe286/volumes"
Nov 01 05:59:46.327138 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.327106    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"]
Nov 01 05:59:46.327618 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.327594    4864 cpu_manager.go:414] "containerMap: removing container" podUID="b28a4792-6e4e-4f9b-80e0-737dc18ba934" containerName="main"
Nov 01 05:59:46.327618 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.327605    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b28a4792-6e4e-4f9b-80e0-737dc18ba934" containerName="main"
Nov 01 05:59:46.327695 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.327623    4864 cpu_manager.go:414] "containerMap: removing container" podUID="d94a1258-dbb1-44b5-bf57-1b44b66fe286" containerName="main"
Nov 01 05:59:46.327695 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.327628    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d94a1258-dbb1-44b5-bf57-1b44b66fe286" containerName="main"
Nov 01 05:59:46.327761 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.327712    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="b28a4792-6e4e-4f9b-80e0-737dc18ba934" containerName="main"
Nov 01 05:59:46.327761 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.327721    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="d94a1258-dbb1-44b5-bf57-1b44b66fe286" containerName="main"
Nov 01 05:59:46.331162 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.331146    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:46.346115 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.346089    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-model-cache\") pod \"e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:46.346261 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.346158    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-tmp-dir\") pod \"e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:46.346331 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.346311    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58c8l\" (UniqueName: \"kubernetes.io/projected/d9a45d6a-720d-45f0-973e-a69b24c10cc7-kube-api-access-58c8l\") pod \"e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:46.346435 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.346415    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-home\") pod \"e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:46.346494 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.346450    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-dshm\") pod \"e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:46.346560 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.346508    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9a45d6a-720d-45f0-973e-a69b24c10cc7-tls-certs\") pod \"e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:46.432236 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.432198    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-a48099\"/\"e2e-conflict-a-9da93470-kserve-self-signed-certs\""
Nov 01 05:59:46.435448 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.435416    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"]
Nov 01 05:59:46.447135 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.447107    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-home\") pod \"e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:46.447322 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.447143    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-dshm\") pod \"e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:46.447322 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.447166    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9a45d6a-720d-45f0-973e-a69b24c10cc7-tls-certs\") pod \"e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:46.447322 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.447213    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-model-cache\") pod \"e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:46.447322 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.447256    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-tmp-dir\") pod \"e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:46.447539 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.447353    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58c8l\" (UniqueName: \"kubernetes.io/projected/d9a45d6a-720d-45f0-973e-a69b24c10cc7-kube-api-access-58c8l\") pod \"e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:46.447621 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.447592    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-home\") pod \"e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:46.447763 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.447737    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-tmp-dir\") pod \"e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:46.448016 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.447972    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-model-cache\") pod \"e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:46.449703 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.449681    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-dshm\") pod \"e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:46.449971 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.449953    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9a45d6a-720d-45f0-973e-a69b24c10cc7-tls-certs\") pod \"e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:46.509285 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.509246    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58c8l\" (UniqueName: \"kubernetes.io/projected/d9a45d6a-720d-45f0-973e-a69b24c10cc7-kube-api-access-58c8l\") pod \"e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:46.643661 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:46.643571    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:47.055062 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:47.055035    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"]
Nov 01 05:59:47.056341 ip-10-0-134-253 kubenswrapper[4864]: W1101 05:59:47.056301    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9a45d6a_720d_45f0_973e_a69b24c10cc7.slice/crio-ef0b3953bad4a0473550597eb057d25c61b39bd8a2d38f2453b824217f394615 WatchSource:0}: Error finding container ef0b3953bad4a0473550597eb057d25c61b39bd8a2d38f2453b824217f394615: Status 404 returned error can't find the container with id ef0b3953bad4a0473550597eb057d25c61b39bd8a2d38f2453b824217f394615
Nov 01 05:59:47.658833 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:47.658794    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz" event={"ID":"d9a45d6a-720d-45f0-973e-a69b24c10cc7","Type":"ContainerStarted","Data":"70b5cc3a6a7b5758a3de0ef0e400e782eedeab635ab720a19405b1687e8bb575"}
Nov 01 05:59:47.658833 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:47.658838    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz" event={"ID":"d9a45d6a-720d-45f0-973e-a69b24c10cc7","Type":"ContainerStarted","Data":"ef0b3953bad4a0473550597eb057d25c61b39bd8a2d38f2453b824217f394615"}
Nov 01 05:59:47.774114 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:47.774058    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz" podStartSLOduration=1.7740402579999999 podStartE2EDuration="1.774040258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 05:59:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 05:59:47.773422622 +0000 UTC m=+3404.663798599" watchObservedRunningTime="2026-11-01 05:59:47.774040258 +0000 UTC m=+3404.664416239"
Nov 01 05:59:48.643771 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:48.643725    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:52.524570 ip-10-0-134-253 kubenswrapper[4864]: E1101 05:59:52.524532    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 05:59:56.643914 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:56.643762    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:56.675160 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:56.675124    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 05:59:56.733886 ip-10-0-134-253 kubenswrapper[4864]: I1101 05:59:56.733852    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 06:00:00.199455 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.199416    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld"]
Nov 01 06:00:00.204723 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.204702    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld"
Nov 01 06:00:00.215621 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.215596    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8"]
Nov 01 06:00:00.219911 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.219893    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8"
Nov 01 06:00:00.315087 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.315053    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb7rh\" (UniqueName: \"kubernetes.io/projected/0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee-kube-api-access-kb7rh\") pod \"maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld\" (UID: \"0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld"
Nov 01 06:00:00.315256 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.315223    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld\" (UID: \"0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld"
Nov 01 06:00:00.315378 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.315350    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4t6q\" (UniqueName: \"kubernetes.io/projected/c6ce5d10-4d50-48b5-8676-b252c00a9c03-kube-api-access-g4t6q\") pod \"maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8\" (UID: \"c6ce5d10-4d50-48b5-8676-b252c00a9c03\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8"
Nov 01 06:00:00.315549 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.315455    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6ce5d10-4d50-48b5-8676-b252c00a9c03-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8\" (UID: \"c6ce5d10-4d50-48b5-8676-b252c00a9c03\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8"
Nov 01 06:00:00.339419 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.339389    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sp8g2\""
Nov 01 06:00:00.378472 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.378437    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8"]
Nov 01 06:00:00.402042 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.402012    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld"]
Nov 01 06:00:00.416762 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.416727    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb7rh\" (UniqueName: \"kubernetes.io/projected/0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee-kube-api-access-kb7rh\") pod \"maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld\" (UID: \"0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld"
Nov 01 06:00:00.416968 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.416857    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld\" (UID: \"0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld"
Nov 01 06:00:00.416968 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.416945    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4t6q\" (UniqueName: \"kubernetes.io/projected/c6ce5d10-4d50-48b5-8676-b252c00a9c03-kube-api-access-g4t6q\") pod \"maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8\" (UID: \"c6ce5d10-4d50-48b5-8676-b252c00a9c03\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8"
Nov 01 06:00:00.417102 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.417014    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6ce5d10-4d50-48b5-8676-b252c00a9c03-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8\" (UID: \"c6ce5d10-4d50-48b5-8676-b252c00a9c03\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8"
Nov 01 06:00:00.417890 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.417866    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld\" (UID: \"0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld"
Nov 01 06:00:00.418003 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.417866    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6ce5d10-4d50-48b5-8676-b252c00a9c03-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8\" (UID: \"c6ce5d10-4d50-48b5-8676-b252c00a9c03\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8"
Nov 01 06:00:00.477337 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.477266    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb7rh\" (UniqueName: \"kubernetes.io/projected/0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee-kube-api-access-kb7rh\") pod \"maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld\" (UID: \"0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld"
Nov 01 06:00:00.477857 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.477839    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4t6q\" (UniqueName: \"kubernetes.io/projected/c6ce5d10-4d50-48b5-8676-b252c00a9c03-kube-api-access-g4t6q\") pod \"maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8\" (UID: \"c6ce5d10-4d50-48b5-8676-b252c00a9c03\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8"
Nov 01 06:00:00.521364 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.521334    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld"
Nov 01 06:00:00.533516 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.533493    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8"
Nov 01 06:00:00.903194 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.903167    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld"]
Nov 01 06:00:00.904382 ip-10-0-134-253 kubenswrapper[4864]: W1101 06:00:00.904351    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dfaecaf_71c8_4c1f_bba4_69bbae8b62ee.slice/crio-52782ec794bb06084a2558e5278a629423400340bcc7311f201501b9722087b3 WatchSource:0}: Error finding container 52782ec794bb06084a2558e5278a629423400340bcc7311f201501b9722087b3: Status 404 returned error can't find the container with id 52782ec794bb06084a2558e5278a629423400340bcc7311f201501b9722087b3
Nov 01 06:00:00.952770 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:00.952747    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8"]
Nov 01 06:00:00.953616 ip-10-0-134-253 kubenswrapper[4864]: W1101 06:00:00.953591    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6ce5d10_4d50_48b5_8676_b252c00a9c03.slice/crio-8e1b174dbe1e5ad39167199016f858a2194dbfb593947687bfeff05c2c298d7c WatchSource:0}: Error finding container 8e1b174dbe1e5ad39167199016f858a2194dbfb593947687bfeff05c2c298d7c: Status 404 returned error can't find the container with id 8e1b174dbe1e5ad39167199016f858a2194dbfb593947687bfeff05c2c298d7c
Nov 01 06:00:01.762890 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:01.762519    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8" event={"ID":"c6ce5d10-4d50-48b5-8676-b252c00a9c03","Type":"ContainerStarted","Data":"a11c3b6164c8a89fe456d9f8cd68ead6aef95ff6dd33789a6adcb2a26c2e5a47"}
Nov 01 06:00:01.762890 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:01.762615    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8" event={"ID":"c6ce5d10-4d50-48b5-8676-b252c00a9c03","Type":"ContainerStarted","Data":"8e1b174dbe1e5ad39167199016f858a2194dbfb593947687bfeff05c2c298d7c"}
Nov 01 06:00:01.764629 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:01.764600    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld" event={"ID":"0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee","Type":"ContainerStarted","Data":"0836e542a79670f09d370dd2e407c29701164057422b017851b1cf8f5e74b547"}
Nov 01 06:00:01.764756 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:01.764656    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld" event={"ID":"0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee","Type":"ContainerStarted","Data":"52782ec794bb06084a2558e5278a629423400340bcc7311f201501b9722087b3"}
Nov 01 06:00:01.878798 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:01.878742    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8" podStartSLOduration=1.878726916 podStartE2EDuration="1.878726916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 06:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 06:00:01.875808653 +0000 UTC m=+3418.766184632" watchObservedRunningTime="2026-11-01 06:00:01.878726916 +0000 UTC m=+3418.769102889"
Nov 01 06:00:01.990434 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:01.990376    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld" podStartSLOduration=1.990361845 podStartE2EDuration="1.990361845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 06:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 06:00:01.988861043 +0000 UTC m=+3418.879237020" watchObservedRunningTime="2026-11-01 06:00:01.990361845 +0000 UTC m=+3418.880737833"
Nov 01 06:00:02.773883 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:02.773841    4864 generic.go:356] "Generic (PLEG): container finished" podID="c6ce5d10-4d50-48b5-8676-b252c00a9c03" containerID="a11c3b6164c8a89fe456d9f8cd68ead6aef95ff6dd33789a6adcb2a26c2e5a47" exitCode=60
Nov 01 06:00:02.774392 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:02.773934    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8" event={"ID":"c6ce5d10-4d50-48b5-8676-b252c00a9c03","Type":"ContainerDied","Data":"a11c3b6164c8a89fe456d9f8cd68ead6aef95ff6dd33789a6adcb2a26c2e5a47"}
Nov 01 06:00:02.774392 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:02.774274    4864 scope.go:122] "RemoveContainer" containerID="a11c3b6164c8a89fe456d9f8cd68ead6aef95ff6dd33789a6adcb2a26c2e5a47"
Nov 01 06:00:02.775950 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:02.775921    4864 generic.go:356] "Generic (PLEG): container finished" podID="0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee" containerID="0836e542a79670f09d370dd2e407c29701164057422b017851b1cf8f5e74b547" exitCode=60
Nov 01 06:00:02.776079 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:02.776064    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld" event={"ID":"0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee","Type":"ContainerDied","Data":"0836e542a79670f09d370dd2e407c29701164057422b017851b1cf8f5e74b547"}
Nov 01 06:00:02.776468 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:02.776449    4864 scope.go:122] "RemoveContainer" containerID="0836e542a79670f09d370dd2e407c29701164057422b017851b1cf8f5e74b547"
Nov 01 06:00:03.785716 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:03.785682    4864 generic.go:356] "Generic (PLEG): container finished" podID="c6ce5d10-4d50-48b5-8676-b252c00a9c03" containerID="c896c0bef2a1e56c0c50432f2ba5919bc38749abe530db3ec03a52774d2cbfe7" exitCode=60
Nov 01 06:00:03.786184 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:03.785772    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8" event={"ID":"c6ce5d10-4d50-48b5-8676-b252c00a9c03","Type":"ContainerDied","Data":"c896c0bef2a1e56c0c50432f2ba5919bc38749abe530db3ec03a52774d2cbfe7"}
Nov 01 06:00:03.786184 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:03.785861    4864 scope.go:122] "RemoveContainer" containerID="a11c3b6164c8a89fe456d9f8cd68ead6aef95ff6dd33789a6adcb2a26c2e5a47"
Nov 01 06:00:03.786184 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:03.786115    4864 scope.go:122] "RemoveContainer" containerID="c896c0bef2a1e56c0c50432f2ba5919bc38749abe530db3ec03a52774d2cbfe7"
Nov 01 06:00:03.786424 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:00:03.786401    4864 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8_odh-ai-gateway-infra(c6ce5d10-4d50-48b5-8676-b252c00a9c03)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8" podUID="c6ce5d10-4d50-48b5-8676-b252c00a9c03"
Nov 01 06:00:03.788158 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:03.788136    4864 generic.go:356] "Generic (PLEG): container finished" podID="0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee" containerID="accaaa597c016480d7ea449b1957cd0eee2f7ff1aeac63fc44f586700d7b2eec" exitCode=60
Nov 01 06:00:03.788285 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:03.788238    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld" event={"ID":"0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee","Type":"ContainerDied","Data":"accaaa597c016480d7ea449b1957cd0eee2f7ff1aeac63fc44f586700d7b2eec"}
Nov 01 06:00:03.788474 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:03.788453    4864 scope.go:122] "RemoveContainer" containerID="accaaa597c016480d7ea449b1957cd0eee2f7ff1aeac63fc44f586700d7b2eec"
Nov 01 06:00:03.788777 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:00:03.788688    4864 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld_odh-ai-gateway-infra(0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld" podUID="0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee"
Nov 01 06:00:03.799482 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:03.799460    4864 scope.go:122] "RemoveContainer" containerID="0836e542a79670f09d370dd2e407c29701164057422b017851b1cf8f5e74b547"
Nov 01 06:00:04.797887 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:04.797854    4864 scope.go:122] "RemoveContainer" containerID="accaaa597c016480d7ea449b1957cd0eee2f7ff1aeac63fc44f586700d7b2eec"
Nov 01 06:00:04.798367 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:00:04.798182    4864 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld_odh-ai-gateway-infra(0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld" podUID="0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee"
Nov 01 06:00:04.799663 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:04.799641    4864 scope.go:122] "RemoveContainer" containerID="c896c0bef2a1e56c0c50432f2ba5919bc38749abe530db3ec03a52774d2cbfe7"
Nov 01 06:00:04.799941 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:00:04.799918    4864 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8_odh-ai-gateway-infra(c6ce5d10-4d50-48b5-8676-b252c00a9c03)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8" podUID="c6ce5d10-4d50-48b5-8676-b252c00a9c03"
Nov 01 06:00:18.241146 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:18.241113    4864 scope.go:122] "RemoveContainer" containerID="c896c0bef2a1e56c0c50432f2ba5919bc38749abe530db3ec03a52774d2cbfe7"
Nov 01 06:00:18.241662 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:18.241314    4864 scope.go:122] "RemoveContainer" containerID="accaaa597c016480d7ea449b1957cd0eee2f7ff1aeac63fc44f586700d7b2eec"
Nov 01 06:00:18.894883 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:18.894807    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld" event={"ID":"0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee","Type":"ContainerStarted","Data":"008aebff2041bc25e4386b9548e1f5ed3befc64b3e8d13cdd00f5261b33d3d45"}
Nov 01 06:00:18.897363 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:18.897325    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8" event={"ID":"c6ce5d10-4d50-48b5-8676-b252c00a9c03","Type":"ContainerStarted","Data":"c6f419cf50b85b8b5d2f9ebeaad9f8311accaefab3743f439606af632fc7699a"}
Nov 01 06:00:19.906452 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:19.906419    4864 generic.go:356] "Generic (PLEG): container finished" podID="c6ce5d10-4d50-48b5-8676-b252c00a9c03" containerID="c6f419cf50b85b8b5d2f9ebeaad9f8311accaefab3743f439606af632fc7699a" exitCode=60
Nov 01 06:00:19.906983 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:19.906500    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8" event={"ID":"c6ce5d10-4d50-48b5-8676-b252c00a9c03","Type":"ContainerDied","Data":"c6f419cf50b85b8b5d2f9ebeaad9f8311accaefab3743f439606af632fc7699a"}
Nov 01 06:00:19.906983 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:19.906584    4864 scope.go:122] "RemoveContainer" containerID="c896c0bef2a1e56c0c50432f2ba5919bc38749abe530db3ec03a52774d2cbfe7"
Nov 01 06:00:19.907061 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:19.906995    4864 scope.go:122] "RemoveContainer" containerID="c6f419cf50b85b8b5d2f9ebeaad9f8311accaefab3743f439606af632fc7699a"
Nov 01 06:00:19.907277 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:00:19.907252    4864 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8_odh-ai-gateway-infra(c6ce5d10-4d50-48b5-8676-b252c00a9c03)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8" podUID="c6ce5d10-4d50-48b5-8676-b252c00a9c03"
Nov 01 06:00:19.908644 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:19.908623    4864 generic.go:356] "Generic (PLEG): container finished" podID="0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee" containerID="008aebff2041bc25e4386b9548e1f5ed3befc64b3e8d13cdd00f5261b33d3d45" exitCode=60
Nov 01 06:00:19.908729 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:19.908691    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld" event={"ID":"0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee","Type":"ContainerDied","Data":"008aebff2041bc25e4386b9548e1f5ed3befc64b3e8d13cdd00f5261b33d3d45"}
Nov 01 06:00:19.908998 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:19.908984    4864 scope.go:122] "RemoveContainer" containerID="008aebff2041bc25e4386b9548e1f5ed3befc64b3e8d13cdd00f5261b33d3d45"
Nov 01 06:00:19.909202 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:00:19.909186    4864 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld_odh-ai-gateway-infra(0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld" podUID="0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee"
Nov 01 06:00:19.919975 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:19.919952    4864 scope.go:122] "RemoveContainer" containerID="accaaa597c016480d7ea449b1957cd0eee2f7ff1aeac63fc44f586700d7b2eec"
Nov 01 06:00:20.022911 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:20.022871    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld"]
Nov 01 06:00:20.145320 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:20.145275    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8"]
Nov 01 06:00:21.102039 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:21.102013    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8"
Nov 01 06:00:21.104137 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:21.104119    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld"
Nov 01 06:00:21.163594 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:21.163563    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee-kube-api-access-kb7rh\" (UniqueName: \"kubernetes.io/projected/0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee-kube-api-access-kb7rh\") pod \"0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee\" (UID: \"0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee\") "
Nov 01 06:00:21.163721 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:21.163674    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c6ce5d10-4d50-48b5-8676-b252c00a9c03-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6ce5d10-4d50-48b5-8676-b252c00a9c03-maas-api-service-ca\") pod \"c6ce5d10-4d50-48b5-8676-b252c00a9c03\" (UID: \"c6ce5d10-4d50-48b5-8676-b252c00a9c03\") "
Nov 01 06:00:21.163773 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:21.163728    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6ce5d10-4d50-48b5-8676-b252c00a9c03-kube-api-access-g4t6q\" (UniqueName: \"kubernetes.io/projected/c6ce5d10-4d50-48b5-8676-b252c00a9c03-kube-api-access-g4t6q\") pod \"c6ce5d10-4d50-48b5-8676-b252c00a9c03\" (UID: \"c6ce5d10-4d50-48b5-8676-b252c00a9c03\") "
Nov 01 06:00:21.163852 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:21.163815    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee-maas-api-service-ca\") pod \"0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee\" (UID: \"0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee\") "
Nov 01 06:00:21.164101 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:21.164063    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6ce5d10-4d50-48b5-8676-b252c00a9c03-maas-api-service-ca" pod "c6ce5d10-4d50-48b5-8676-b252c00a9c03" (UID: "c6ce5d10-4d50-48b5-8676-b252c00a9c03"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 06:00:21.164291 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:21.164200    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6ce5d10-4d50-48b5-8676-b252c00a9c03-maas-api-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:00:21.164369 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:21.164346    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee-maas-api-service-ca" pod "0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee" (UID: "0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 06:00:21.166533 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:21.166507    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6ce5d10-4d50-48b5-8676-b252c00a9c03-kube-api-access-g4t6q" pod "c6ce5d10-4d50-48b5-8676-b252c00a9c03" (UID: "c6ce5d10-4d50-48b5-8676-b252c00a9c03"). InnerVolumeSpecName "kube-api-access-g4t6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:00:21.166712 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:21.166692    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee-kube-api-access-kb7rh" pod "0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee" (UID: "0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee"). InnerVolumeSpecName "kube-api-access-kb7rh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:00:21.265210 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:21.265185    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee-maas-api-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:00:21.265326 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:21.265213    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb7rh\" (UniqueName: \"kubernetes.io/projected/0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee-kube-api-access-kb7rh\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:00:21.265326 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:21.265226    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4t6q\" (UniqueName: \"kubernetes.io/projected/c6ce5d10-4d50-48b5-8676-b252c00a9c03-kube-api-access-g4t6q\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:00:21.926951 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:21.926917    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld"
Nov 01 06:00:21.926951 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:21.926930    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld" event={"ID":"0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee","Type":"ContainerDied","Data":"52782ec794bb06084a2558e5278a629423400340bcc7311f201501b9722087b3"}
Nov 01 06:00:21.927201 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:21.927020    4864 scope.go:122] "RemoveContainer" containerID="008aebff2041bc25e4386b9548e1f5ed3befc64b3e8d13cdd00f5261b33d3d45"
Nov 01 06:00:21.929635 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:21.929606    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8" event={"ID":"c6ce5d10-4d50-48b5-8676-b252c00a9c03","Type":"ContainerDied","Data":"8e1b174dbe1e5ad39167199016f858a2194dbfb593947687bfeff05c2c298d7c"}
Nov 01 06:00:21.929755 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:21.929665    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8"
Nov 01 06:00:21.939017 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:21.938991    4864 scope.go:122] "RemoveContainer" containerID="c6f419cf50b85b8b5d2f9ebeaad9f8311accaefab3743f439606af632fc7699a"
Nov 01 06:00:22.094703 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:22.094671    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8"]
Nov 01 06:00:22.100847 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:22.100822    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2fd4c9-29891880-bqrv8"]
Nov 01 06:00:22.208550 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:22.208475    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld"]
Nov 01 06:00:22.218191 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:22.218167    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a12c35-29891880-g2qld"]
Nov 01 06:00:22.525325 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:00:22.525238    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:00:23.248047 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:23.248017    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee" path="/var/lib/kubelet/pods/0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee/volumes"
Nov 01 06:00:23.248472 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:23.248457    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c6ce5d10-4d50-48b5-8676-b252c00a9c03" path="/var/lib/kubelet/pods/c6ce5d10-4d50-48b5-8676-b252c00a9c03/volumes"
Nov 01 06:00:44.311505 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.311417    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"]
Nov 01 06:00:44.312177 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.312156    4864 cpu_manager.go:414] "containerMap: removing container" podUID="c6ce5d10-4d50-48b5-8676-b252c00a9c03" containerName="cleanup"
Nov 01 06:00:44.312276 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.312178    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6ce5d10-4d50-48b5-8676-b252c00a9c03" containerName="cleanup"
Nov 01 06:00:44.312276 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.312204    4864 cpu_manager.go:414] "containerMap: removing container" podUID="c6ce5d10-4d50-48b5-8676-b252c00a9c03" containerName="cleanup"
Nov 01 06:00:44.312276 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.312212    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6ce5d10-4d50-48b5-8676-b252c00a9c03" containerName="cleanup"
Nov 01 06:00:44.312276 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.312241    4864 cpu_manager.go:414] "containerMap: removing container" podUID="0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee" containerName="cleanup"
Nov 01 06:00:44.312276 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.312249    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee" containerName="cleanup"
Nov 01 06:00:44.312276 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.312262    4864 cpu_manager.go:414] "containerMap: removing container" podUID="0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee" containerName="cleanup"
Nov 01 06:00:44.312276 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.312268    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee" containerName="cleanup"
Nov 01 06:00:44.312633 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.312286    4864 cpu_manager.go:414] "containerMap: removing container" podUID="c6ce5d10-4d50-48b5-8676-b252c00a9c03" containerName="cleanup"
Nov 01 06:00:44.312633 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.312292    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6ce5d10-4d50-48b5-8676-b252c00a9c03" containerName="cleanup"
Nov 01 06:00:44.312633 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.312443    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6ce5d10-4d50-48b5-8676-b252c00a9c03" containerName="cleanup"
Nov 01 06:00:44.312633 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.312457    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6ce5d10-4d50-48b5-8676-b252c00a9c03" containerName="cleanup"
Nov 01 06:00:44.312633 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.312468    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee" containerName="cleanup"
Nov 01 06:00:44.312633 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.312479    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee" containerName="cleanup"
Nov 01 06:00:44.316003 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.315983    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:44.420044 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.420006    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-a48099\"/\"e2e-conflict-b-9da93470-kserve-self-signed-certs\""
Nov 01 06:00:44.423287 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.423260    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"]
Nov 01 06:00:44.428272 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.428247    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-tmp-dir\") pod \"e2e-conflict-b-9da93470-kserve-658979544-chdpr\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:44.428385 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.428363    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-dshm\") pod \"e2e-conflict-b-9da93470-kserve-658979544-chdpr\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:44.428446 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.428396    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acafe0a1-dc08-48f9-a20a-9774b0146dab-tls-certs\") pod \"e2e-conflict-b-9da93470-kserve-658979544-chdpr\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:44.428506 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.428484    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-home\") pod \"e2e-conflict-b-9da93470-kserve-658979544-chdpr\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:44.428608 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.428591    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdkjv\" (UniqueName: \"kubernetes.io/projected/acafe0a1-dc08-48f9-a20a-9774b0146dab-kube-api-access-fdkjv\") pod \"e2e-conflict-b-9da93470-kserve-658979544-chdpr\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:44.428662 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.428639    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-model-cache\") pod \"e2e-conflict-b-9da93470-kserve-658979544-chdpr\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:44.530313 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.530274    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-model-cache\") pod \"e2e-conflict-b-9da93470-kserve-658979544-chdpr\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:44.530520 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.530376    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-tmp-dir\") pod \"e2e-conflict-b-9da93470-kserve-658979544-chdpr\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:44.530520 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.530450    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-dshm\") pod \"e2e-conflict-b-9da93470-kserve-658979544-chdpr\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:44.530520 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.530477    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acafe0a1-dc08-48f9-a20a-9774b0146dab-tls-certs\") pod \"e2e-conflict-b-9da93470-kserve-658979544-chdpr\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:44.530520 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.530519    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-home\") pod \"e2e-conflict-b-9da93470-kserve-658979544-chdpr\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:44.530768 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.530601    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdkjv\" (UniqueName: \"kubernetes.io/projected/acafe0a1-dc08-48f9-a20a-9774b0146dab-kube-api-access-fdkjv\") pod \"e2e-conflict-b-9da93470-kserve-658979544-chdpr\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:44.530768 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.530750    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-model-cache\") pod \"e2e-conflict-b-9da93470-kserve-658979544-chdpr\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:44.531247 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.531209    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-home\") pod \"e2e-conflict-b-9da93470-kserve-658979544-chdpr\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:44.531572 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.531247    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-tmp-dir\") pod \"e2e-conflict-b-9da93470-kserve-658979544-chdpr\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:44.533317 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.533293    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-dshm\") pod \"e2e-conflict-b-9da93470-kserve-658979544-chdpr\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:44.533509 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.533488    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acafe0a1-dc08-48f9-a20a-9774b0146dab-tls-certs\") pod \"e2e-conflict-b-9da93470-kserve-658979544-chdpr\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:44.591168 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.591099    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdkjv\" (UniqueName: \"kubernetes.io/projected/acafe0a1-dc08-48f9-a20a-9774b0146dab-kube-api-access-fdkjv\") pod \"e2e-conflict-b-9da93470-kserve-658979544-chdpr\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") " pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:44.628539 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:44.628509    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:45.081619 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:45.081592    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"]
Nov 01 06:00:45.084012 ip-10-0-134-253 kubenswrapper[4864]: W1101 06:00:45.083980    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podacafe0a1_dc08_48f9_a20a_9774b0146dab.slice/crio-49bd4135c51cbccb179cddf5e18ff3c521b271de8a0ad4ec7f931338727138b3 WatchSource:0}: Error finding container 49bd4135c51cbccb179cddf5e18ff3c521b271de8a0ad4ec7f931338727138b3: Status 404 returned error can't find the container with id 49bd4135c51cbccb179cddf5e18ff3c521b271de8a0ad4ec7f931338727138b3
Nov 01 06:00:46.099969 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:46.099927    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr" event={"ID":"acafe0a1-dc08-48f9-a20a-9774b0146dab","Type":"ContainerStarted","Data":"c375b0a432e8dd877cf4f9ef8066448aecc37eda097bbb56f1826eb114b5e2c8"}
Nov 01 06:00:46.099969 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:46.099972    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr" event={"ID":"acafe0a1-dc08-48f9-a20a-9774b0146dab","Type":"ContainerStarted","Data":"49bd4135c51cbccb179cddf5e18ff3c521b271de8a0ad4ec7f931338727138b3"}
Nov 01 06:00:46.215472 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:46.215417    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr" podStartSLOduration=2.215401157 podStartE2EDuration="2.215401157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 06:00:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 06:00:46.21472474 +0000 UTC m=+3463.105100717" watchObservedRunningTime="2026-11-01 06:00:46.215401157 +0000 UTC m=+3463.105777132"
Nov 01 06:00:46.629641 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:46.629610    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:52.528241 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:00:52.528200    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:00:54.629140 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:54.629103    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:54.642086 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:54.642063    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:55.200020 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:55.199992    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:00:59.215752 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:59.215713    4864 generic.go:356] "Generic (PLEG): container finished" podID="a179e91d-6c63-4820-8462-cf2ce37aacc9" containerID="cb7f5272f0d450a913cb8c94fa4d561a460577e1b592d59502eefd04a750bafc" exitCode=0
Nov 01 06:00:59.216242 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:59.215805    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5" event={"ID":"a179e91d-6c63-4820-8462-cf2ce37aacc9","Type":"ContainerDied","Data":"cb7f5272f0d450a913cb8c94fa4d561a460577e1b592d59502eefd04a750bafc"}
Nov 01 06:00:59.216295 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:59.216281    4864 scope.go:122] "RemoveContainer" containerID="cb7f5272f0d450a913cb8c94fa4d561a460577e1b592d59502eefd04a750bafc"
Nov 01 06:00:59.218148 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:59.218114    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 06:00:59.218227 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:59.218204    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 06:00:59.218265 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:00:59.218246    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 06:01:00.224915 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:00.224880    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5" event={"ID":"a179e91d-6c63-4820-8462-cf2ce37aacc9","Type":"ContainerStarted","Data":"8a9704ab34c1183a816d4d7150c97daade8ba89b52cc84c2f7a7b95afa16cfef"}
Nov 01 06:01:00.830585 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:00.830541    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 06:01:00.830945 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:00.830767    4864 patch_prober.go:29] interesting pod/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 01 06:01:00.830945 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:00.830864    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5" podUID="a179e91d-6c63-4820-8462-cf2ce37aacc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 01 06:01:01.830653 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:01.830617    4864 patch_prober.go:29] interesting pod/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 01 06:01:01.831098 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:01.830683    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5" podUID="a179e91d-6c63-4820-8462-cf2ce37aacc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 01 06:01:02.830713 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:02.830680    4864 patch_prober.go:29] interesting pod/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 01 06:01:02.831187 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:02.830738    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5" podUID="a179e91d-6c63-4820-8462-cf2ce37aacc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 01 06:01:03.831105 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:03.831058    4864 patch_prober.go:29] interesting pod/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 01 06:01:03.831624 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:03.831121    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5" podUID="a179e91d-6c63-4820-8462-cf2ce37aacc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 01 06:01:04.830860 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:04.830820    4864 patch_prober.go:29] interesting pod/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 01 06:01:04.831101 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:04.830905    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5" podUID="a179e91d-6c63-4820-8462-cf2ce37aacc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 01 06:01:05.836525 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:05.836490    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 06:01:05.837049 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:05.836909    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 06:01:05.837736 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:05.837714    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 06:01:18.915114 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:18.915078    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"]
Nov 01 06:01:18.915637 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:18.915293    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt" podUID="0fe9c2f7-8c01-4c99-9b80-0b3442562b56" containerName="istio-proxy" containerID="cri-o://92cd4241647a7d0880575e1e6a9e10df526be282837e36708e9fb334e73a52de" gracePeriod=30
Nov 01 06:01:21.643873 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:21.643830    4864 patch_prober.go:29] interesting pod/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 06:01:21.644302 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:21.643916    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt" podUID="0fe9c2f7-8c01-4c99-9b80-0b3442562b56" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 06:01:22.524688 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:01:22.524648    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:01:24.200507 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.200481    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 06:01:24.372799 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.372748    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-envoy\") pod \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") "
Nov 01 06:01:24.373001 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.372830    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-workload-certs\") pod \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") "
Nov 01 06:01:24.373001 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.372945    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istiod-ca-cert\") pod \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") "
Nov 01 06:01:24.373001 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.372983    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-kube-api-access-z9mrl\" (UniqueName: \"kubernetes.io/projected/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-kube-api-access-z9mrl\") pod \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") "
Nov 01 06:01:24.373216 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.373089    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-credential-socket\") pod \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") "
Nov 01 06:01:24.373216 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.373137    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-token\" (UniqueName: \"kubernetes.io/projected/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-token\") pod \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") "
Nov 01 06:01:24.373216 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.373169    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-data\") pod \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") "
Nov 01 06:01:24.373380 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.373217    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-workload-socket\") pod \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") "
Nov 01 06:01:24.373380 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.373252    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-podinfo\") pod \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\" (UID: \"0fe9c2f7-8c01-4c99-9b80-0b3442562b56\") "
Nov 01 06:01:24.373486 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.373404    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-credential-socket" pod "0fe9c2f7-8c01-4c99-9b80-0b3442562b56" (UID: "0fe9c2f7-8c01-4c99-9b80-0b3442562b56"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:01:24.374209 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.373830    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:01:24.374209 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.373892    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-workload-socket" pod "0fe9c2f7-8c01-4c99-9b80-0b3442562b56" (UID: "0fe9c2f7-8c01-4c99-9b80-0b3442562b56"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:01:24.374209 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.373945    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-data" pod "0fe9c2f7-8c01-4c99-9b80-0b3442562b56" (UID: "0fe9c2f7-8c01-4c99-9b80-0b3442562b56"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:01:24.374480 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.374281    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istiod-ca-cert" pod "0fe9c2f7-8c01-4c99-9b80-0b3442562b56" (UID: "0fe9c2f7-8c01-4c99-9b80-0b3442562b56"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 06:01:24.374480 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.374343    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-workload-certs" pod "0fe9c2f7-8c01-4c99-9b80-0b3442562b56" (UID: "0fe9c2f7-8c01-4c99-9b80-0b3442562b56"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:01:24.376892 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.376862    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-podinfo" pod "0fe9c2f7-8c01-4c99-9b80-0b3442562b56" (UID: "0fe9c2f7-8c01-4c99-9b80-0b3442562b56"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 06:01:24.377341 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.377301    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-envoy" pod "0fe9c2f7-8c01-4c99-9b80-0b3442562b56" (UID: "0fe9c2f7-8c01-4c99-9b80-0b3442562b56"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:01:24.378146 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.377877    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-kube-api-access-z9mrl" pod "0fe9c2f7-8c01-4c99-9b80-0b3442562b56" (UID: "0fe9c2f7-8c01-4c99-9b80-0b3442562b56"). InnerVolumeSpecName "kube-api-access-z9mrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:01:24.378146 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.378122    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-token" pod "0fe9c2f7-8c01-4c99-9b80-0b3442562b56" (UID: "0fe9c2f7-8c01-4c99-9b80-0b3442562b56"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:01:24.405006 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.402112    4864 generic.go:356] "Generic (PLEG): container finished" podID="0fe9c2f7-8c01-4c99-9b80-0b3442562b56" containerID="92cd4241647a7d0880575e1e6a9e10df526be282837e36708e9fb334e73a52de" exitCode=0
Nov 01 06:01:24.405006 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.402164    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt" event={"ID":"0fe9c2f7-8c01-4c99-9b80-0b3442562b56","Type":"ContainerDied","Data":"92cd4241647a7d0880575e1e6a9e10df526be282837e36708e9fb334e73a52de"}
Nov 01 06:01:24.405006 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.402175    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"
Nov 01 06:01:24.405006 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.402252    4864 scope.go:122] "RemoveContainer" containerID="92cd4241647a7d0880575e1e6a9e10df526be282837e36708e9fb334e73a52de"
Nov 01 06:01:24.405006 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.402239    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt" event={"ID":"0fe9c2f7-8c01-4c99-9b80-0b3442562b56","Type":"ContainerDied","Data":"e658705ccb82b5046e4f1859cd3854b039d7a62d3a213d1d735c3ed4631eca3e"}
Nov 01 06:01:24.415212 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.415189    4864 scope.go:122] "RemoveContainer" containerID="92cd4241647a7d0880575e1e6a9e10df526be282837e36708e9fb334e73a52de"
Nov 01 06:01:24.415604 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:01:24.415571    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92cd4241647a7d0880575e1e6a9e10df526be282837e36708e9fb334e73a52de\": container with ID starting with 92cd4241647a7d0880575e1e6a9e10df526be282837e36708e9fb334e73a52de not found: ID does not exist" containerID="92cd4241647a7d0880575e1e6a9e10df526be282837e36708e9fb334e73a52de"
Nov 01 06:01:24.415726 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.415615    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92cd4241647a7d0880575e1e6a9e10df526be282837e36708e9fb334e73a52de"} err="failed to get container status \"92cd4241647a7d0880575e1e6a9e10df526be282837e36708e9fb334e73a52de\": rpc error: code = NotFound desc = could not find container \"92cd4241647a7d0880575e1e6a9e10df526be282837e36708e9fb334e73a52de\": container with ID starting with 92cd4241647a7d0880575e1e6a9e10df526be282837e36708e9fb334e73a52de not found: ID does not exist"
Nov 01 06:01:24.474721 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.474636    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:01:24.474721 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.474663    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:01:24.474721 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.474674    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:01:24.474721 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.474684    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:01:24.474721 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.474693    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:01:24.474721 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.474701    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:01:24.474721 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.474709    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:01:24.474721 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.474719    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9mrl\" (UniqueName: \"kubernetes.io/projected/0fe9c2f7-8c01-4c99-9b80-0b3442562b56-kube-api-access-z9mrl\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:01:24.566461 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.566418    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"]
Nov 01 06:01:24.570154 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:24.570129    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt"]
Nov 01 06:01:25.250732 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:25.250690    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0fe9c2f7-8c01-4c99-9b80-0b3442562b56" path="/var/lib/kubelet/pods/0fe9c2f7-8c01-4c99-9b80-0b3442562b56/volumes"
Nov 01 06:01:27.041996 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:27.041963    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"]
Nov 01 06:01:37.663602 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.663563    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"]
Nov 01 06:01:37.664543 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.664519    4864 cpu_manager.go:414] "containerMap: removing container" podUID="0fe9c2f7-8c01-4c99-9b80-0b3442562b56" containerName="istio-proxy"
Nov 01 06:01:37.664543 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.664545    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0fe9c2f7-8c01-4c99-9b80-0b3442562b56" containerName="istio-proxy"
Nov 01 06:01:37.664686 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.664591    4864 cpu_manager.go:414] "containerMap: removing container" podUID="0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee" containerName="cleanup"
Nov 01 06:01:37.664686 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.664599    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee" containerName="cleanup"
Nov 01 06:01:37.664756 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.664720    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="0dfaecaf-71c8-4c1f-bba4-69bbae8b62ee" containerName="cleanup"
Nov 01 06:01:37.664756 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.664739    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="0fe9c2f7-8c01-4c99-9b80-0b3442562b56" containerName="istio-proxy"
Nov 01 06:01:37.664756 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.664751    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6ce5d10-4d50-48b5-8676-b252c00a9c03" containerName="cleanup"
Nov 01 06:01:37.668449 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.668427    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.753457 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.753416    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgzwc\" (UniqueName: \"kubernetes.io/projected/0e6d5eb0-20cd-4c97-a68c-558982570e66-kube-api-access-fgzwc\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.753630 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.753466    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-workload-certs\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.753630 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.753555    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e6d5eb0-20cd-4c97-a68c-558982570e66-istiod-ca-cert\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.753630 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.753610    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-token\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.753740 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.753675    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-envoy\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.753777 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.753742    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-credential-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.753874 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.753859    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-workload-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.753914 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.753882    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-data\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.753951 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.753922    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-podinfo\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.769412 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.769358    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-a48099-openshift-default-dockercfg-8mhxp\""
Nov 01 06:01:37.774223 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.774195    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"]
Nov 01 06:01:37.855681 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.855637    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-envoy\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.855681 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.855691    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-credential-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.855974 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.855760    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-workload-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.855974 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.855822    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-data\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.855974 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.855884    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-podinfo\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.855974 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.855921    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgzwc\" (UniqueName: \"kubernetes.io/projected/0e6d5eb0-20cd-4c97-a68c-558982570e66-kube-api-access-fgzwc\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.855974 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.855956    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-workload-certs\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.856219 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.855988    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e6d5eb0-20cd-4c97-a68c-558982570e66-istiod-ca-cert\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.856219 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.856011    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-token\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.856219 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.856197    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-workload-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.856417 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.856237    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-credential-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.856417 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.856397    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-workload-certs\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.856728 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.856704    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-data\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.857078 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.857051    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e6d5eb0-20cd-4c97-a68c-558982570e66-istiod-ca-cert\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.858420 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.858390    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-envoy\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.858680 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.858661    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-podinfo\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.916855 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.916691    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgzwc\" (UniqueName: \"kubernetes.io/projected/0e6d5eb0-20cd-4c97-a68c-558982570e66-kube-api-access-fgzwc\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.916855 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.916727    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-token\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-qpjb7\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:37.985500 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:37.985464    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:38.418741 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:38.418716    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"]
Nov 01 06:01:38.419333 ip-10-0-134-253 kubenswrapper[4864]: W1101 06:01:38.419312    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e6d5eb0_20cd_4c97_a68c_558982570e66.slice/crio-fadf999e6725716f77ddf5e1b9d7085fb8973e330530ee14b1cb9f541430201a WatchSource:0}: Error finding container fadf999e6725716f77ddf5e1b9d7085fb8973e330530ee14b1cb9f541430201a: Status 404 returned error can't find the container with id fadf999e6725716f77ddf5e1b9d7085fb8973e330530ee14b1cb9f541430201a
Nov 01 06:01:38.421809 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:38.421760    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 06:01:38.421884 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:38.421849    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 06:01:38.421884 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:38.421879    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 06:01:38.506515 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:38.506475    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7" event={"ID":"0e6d5eb0-20cd-4c97-a68c-558982570e66","Type":"ContainerStarted","Data":"48e767f70daf962f81c25823b2c577fc2828f3ef1b7acd0d30246ea157aab93f"}
Nov 01 06:01:38.506695 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:38.506545    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7" event={"ID":"0e6d5eb0-20cd-4c97-a68c-558982570e66","Type":"ContainerStarted","Data":"fadf999e6725716f77ddf5e1b9d7085fb8973e330530ee14b1cb9f541430201a"}
Nov 01 06:01:38.626578 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:38.626506    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7" podStartSLOduration=1.626486738 podStartE2EDuration="1.626486738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 06:01:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 06:01:38.625168163 +0000 UTC m=+3515.515544139" watchObservedRunningTime="2026-11-01 06:01:38.626486738 +0000 UTC m=+3515.516862714"
Nov 01 06:01:38.985755 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:38.985724    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:39.988092 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:39.988052    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Nov 01 06:01:39.988742 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:39.988707    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7" podUID="0e6d5eb0-20cd-4c97-a68c-558982570e66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Nov 01 06:01:40.988296 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:40.988247    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Nov 01 06:01:40.988775 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:40.988316    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7" podUID="0e6d5eb0-20cd-4c97-a68c-558982570e66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Nov 01 06:01:41.986505 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:41.986461    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Nov 01 06:01:41.986736 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:41.986531    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7" podUID="0e6d5eb0-20cd-4c97-a68c-558982570e66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Nov 01 06:01:42.072025 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.071977    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr" podUID="acafe0a1-dc08-48f9-a20a-9774b0146dab" containerName="main" containerID="cri-o://c375b0a432e8dd877cf4f9ef8066448aecc37eda097bbb56f1826eb114b5e2c8" gracePeriod=45
Nov 01 06:01:42.181823 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.181228    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr" podUID="acafe0a1-dc08-48f9-a20a-9774b0146dab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.125:8000/ready\": dial tcp 10.132.0.125:8000: connect: connection refused"
Nov 01 06:01:42.370443 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.370014    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:01:42.528508 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.528358    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-home\" (UniqueName: \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-home\") pod \"acafe0a1-dc08-48f9-a20a-9774b0146dab\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") "
Nov 01 06:01:42.528508 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.528402    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/acafe0a1-dc08-48f9-a20a-9774b0146dab-kube-api-access-fdkjv\" (UniqueName: \"kubernetes.io/projected/acafe0a1-dc08-48f9-a20a-9774b0146dab-kube-api-access-fdkjv\") pod \"acafe0a1-dc08-48f9-a20a-9774b0146dab\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") "
Nov 01 06:01:42.528508 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.528496    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-model-cache\") pod \"acafe0a1-dc08-48f9-a20a-9774b0146dab\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") "
Nov 01 06:01:42.529627 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.528567    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/acafe0a1-dc08-48f9-a20a-9774b0146dab-tls-certs\" (UniqueName: \"kubernetes.io/secret/acafe0a1-dc08-48f9-a20a-9774b0146dab-tls-certs\") pod \"acafe0a1-dc08-48f9-a20a-9774b0146dab\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") "
Nov 01 06:01:42.529627 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.528606    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-dshm\" (UniqueName: \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-dshm\") pod \"acafe0a1-dc08-48f9-a20a-9774b0146dab\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") "
Nov 01 06:01:42.529627 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.528637    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-tmp-dir\") pod \"acafe0a1-dc08-48f9-a20a-9774b0146dab\" (UID: \"acafe0a1-dc08-48f9-a20a-9774b0146dab\") "
Nov 01 06:01:42.529627 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.528680    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-home" pod "acafe0a1-dc08-48f9-a20a-9774b0146dab" (UID: "acafe0a1-dc08-48f9-a20a-9774b0146dab"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:01:42.529627 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.528982    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-tmp-dir" pod "acafe0a1-dc08-48f9-a20a-9774b0146dab" (UID: "acafe0a1-dc08-48f9-a20a-9774b0146dab"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:01:42.529627 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.529078    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-model-cache" pod "acafe0a1-dc08-48f9-a20a-9774b0146dab" (UID: "acafe0a1-dc08-48f9-a20a-9774b0146dab"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:01:42.529627 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.529158    4864 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-tmp-dir\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:01:42.529627 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.529180    4864 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-home\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:01:42.532135 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.531970    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-dshm" pod "acafe0a1-dc08-48f9-a20a-9774b0146dab" (UID: "acafe0a1-dc08-48f9-a20a-9774b0146dab"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:01:42.532274 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.532099    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acafe0a1-dc08-48f9-a20a-9774b0146dab-tls-certs" pod "acafe0a1-dc08-48f9-a20a-9774b0146dab" (UID: "acafe0a1-dc08-48f9-a20a-9774b0146dab"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 06:01:42.532274 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.532240    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acafe0a1-dc08-48f9-a20a-9774b0146dab-kube-api-access-fdkjv" pod "acafe0a1-dc08-48f9-a20a-9774b0146dab" (UID: "acafe0a1-dc08-48f9-a20a-9774b0146dab"). InnerVolumeSpecName "kube-api-access-fdkjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:01:42.539316 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.539291    4864 generic.go:356] "Generic (PLEG): container finished" podID="acafe0a1-dc08-48f9-a20a-9774b0146dab" containerID="c375b0a432e8dd877cf4f9ef8066448aecc37eda097bbb56f1826eb114b5e2c8" exitCode=0
Nov 01 06:01:42.539435 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.539349    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"
Nov 01 06:01:42.539435 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.539377    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr" event={"ID":"acafe0a1-dc08-48f9-a20a-9774b0146dab","Type":"ContainerDied","Data":"c375b0a432e8dd877cf4f9ef8066448aecc37eda097bbb56f1826eb114b5e2c8"}
Nov 01 06:01:42.539435 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.539421    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr" event={"ID":"acafe0a1-dc08-48f9-a20a-9774b0146dab","Type":"ContainerDied","Data":"49bd4135c51cbccb179cddf5e18ff3c521b271de8a0ad4ec7f931338727138b3"}
Nov 01 06:01:42.539603 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.539438    4864 scope.go:122] "RemoveContainer" containerID="c375b0a432e8dd877cf4f9ef8066448aecc37eda097bbb56f1826eb114b5e2c8"
Nov 01 06:01:42.582878 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.582758    4864 scope.go:122] "RemoveContainer" containerID="c375b0a432e8dd877cf4f9ef8066448aecc37eda097bbb56f1826eb114b5e2c8"
Nov 01 06:01:42.583327 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:01:42.583299    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c375b0a432e8dd877cf4f9ef8066448aecc37eda097bbb56f1826eb114b5e2c8\": container with ID starting with c375b0a432e8dd877cf4f9ef8066448aecc37eda097bbb56f1826eb114b5e2c8 not found: ID does not exist" containerID="c375b0a432e8dd877cf4f9ef8066448aecc37eda097bbb56f1826eb114b5e2c8"
Nov 01 06:01:42.583433 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.583342    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c375b0a432e8dd877cf4f9ef8066448aecc37eda097bbb56f1826eb114b5e2c8"} err="failed to get container status \"c375b0a432e8dd877cf4f9ef8066448aecc37eda097bbb56f1826eb114b5e2c8\": rpc error: code = NotFound desc = could not find container \"c375b0a432e8dd877cf4f9ef8066448aecc37eda097bbb56f1826eb114b5e2c8\": container with ID starting with c375b0a432e8dd877cf4f9ef8066448aecc37eda097bbb56f1826eb114b5e2c8 not found: ID does not exist"
Nov 01 06:01:42.630743 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.630704    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdkjv\" (UniqueName: \"kubernetes.io/projected/acafe0a1-dc08-48f9-a20a-9774b0146dab-kube-api-access-fdkjv\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:01:42.630938 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.630749    4864 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-model-cache\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:01:42.630938 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.630765    4864 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acafe0a1-dc08-48f9-a20a-9774b0146dab-tls-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:01:42.630938 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.630779    4864 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acafe0a1-dc08-48f9-a20a-9774b0146dab-dshm\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:01:42.705645 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.705605    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"]
Nov 01 06:01:42.709077 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.709049    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-a48099/e2e-conflict-b-9da93470-kserve-658979544-chdpr"]
Nov 01 06:01:42.988890 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.988409    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Nov 01 06:01:42.988890 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:42.988475    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7" podUID="0e6d5eb0-20cd-4c97-a68c-558982570e66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Nov 01 06:01:43.248129 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:43.248049    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="acafe0a1-dc08-48f9-a20a-9774b0146dab" path="/var/lib/kubelet/pods/acafe0a1-dc08-48f9-a20a-9774b0146dab/volumes"
Nov 01 06:01:43.986486 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:43.986445    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Nov 01 06:01:43.986694 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:43.986508    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7" podUID="0e6d5eb0-20cd-4c97-a68c-558982570e66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Nov 01 06:01:44.986715 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:44.986682    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Nov 01 06:01:44.987124 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:44.986736    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7" podUID="0e6d5eb0-20cd-4c97-a68c-558982570e66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Nov 01 06:01:45.986848 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:45.986765    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Nov 01 06:01:45.987308 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:45.986885    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7" podUID="0e6d5eb0-20cd-4c97-a68c-558982570e66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Nov 01 06:01:47.990268 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:47.990218    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 06:01:47.990833 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:47.990288    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7" podUID="0e6d5eb0-20cd-4c97-a68c-558982570e66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 06:01:48.031619 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:48.031586    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:48.031988 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:48.031967    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:48.032745 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:48.032727    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:01:52.141320 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:01:52.141275    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"]
Nov 01 06:01:52.525320 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:01:52.525237    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:02:04.273477 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.273439    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"]
Nov 01 06:02:04.273941 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.273686    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk" podUID="e3588066-acd0-433f-96c3-f4461690728f" containerName="maas-api" containerID="cri-o://49cf6439743e5d40c4956e00e420bd0f6cdfbe27e1fb93c05b679a23d18a4fc9" gracePeriod=30
Nov 01 06:02:04.590366 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.590333    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"
Nov 01 06:02:04.700307 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.700272    4864 generic.go:356] "Generic (PLEG): container finished" podID="e3588066-acd0-433f-96c3-f4461690728f" containerID="49cf6439743e5d40c4956e00e420bd0f6cdfbe27e1fb93c05b679a23d18a4fc9" exitCode=0
Nov 01 06:02:04.700607 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.700317    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk" event={"ID":"e3588066-acd0-433f-96c3-f4461690728f","Type":"ContainerDied","Data":"49cf6439743e5d40c4956e00e420bd0f6cdfbe27e1fb93c05b679a23d18a4fc9"}
Nov 01 06:02:04.700734 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.700647    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk" event={"ID":"e3588066-acd0-433f-96c3-f4461690728f","Type":"ContainerDied","Data":"c5a0c0763a66d0d007361352e5ad1e9eb53825440fca94867489857459598f89"}
Nov 01 06:02:04.700734 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.700334    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"
Nov 01 06:02:04.700734 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.700685    4864 scope.go:122] "RemoveContainer" containerID="49cf6439743e5d40c4956e00e420bd0f6cdfbe27e1fb93c05b679a23d18a4fc9"
Nov 01 06:02:04.715710 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.715427    4864 scope.go:122] "RemoveContainer" containerID="49cf6439743e5d40c4956e00e420bd0f6cdfbe27e1fb93c05b679a23d18a4fc9"
Nov 01 06:02:04.715821 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:02:04.715702    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49cf6439743e5d40c4956e00e420bd0f6cdfbe27e1fb93c05b679a23d18a4fc9\": container with ID starting with 49cf6439743e5d40c4956e00e420bd0f6cdfbe27e1fb93c05b679a23d18a4fc9 not found: ID does not exist" containerID="49cf6439743e5d40c4956e00e420bd0f6cdfbe27e1fb93c05b679a23d18a4fc9"
Nov 01 06:02:04.715821 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.715739    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49cf6439743e5d40c4956e00e420bd0f6cdfbe27e1fb93c05b679a23d18a4fc9"} err="failed to get container status \"49cf6439743e5d40c4956e00e420bd0f6cdfbe27e1fb93c05b679a23d18a4fc9\": rpc error: code = NotFound desc = could not find container \"49cf6439743e5d40c4956e00e420bd0f6cdfbe27e1fb93c05b679a23d18a4fc9\": container with ID starting with 49cf6439743e5d40c4956e00e420bd0f6cdfbe27e1fb93c05b679a23d18a4fc9 not found: ID does not exist"
Nov 01 06:02:04.729129 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.729089    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3588066-acd0-433f-96c3-f4461690728f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3588066-acd0-433f-96c3-f4461690728f-openshift-service-ca\") pod \"e3588066-acd0-433f-96c3-f4461690728f\" (UID: \"e3588066-acd0-433f-96c3-f4461690728f\") "
Nov 01 06:02:04.729485 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.729456    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3588066-acd0-433f-96c3-f4461690728f-openshift-service-ca" pod "e3588066-acd0-433f-96c3-f4461690728f" (UID: "e3588066-acd0-433f-96c3-f4461690728f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 06:02:04.730390 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.730369    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e3588066-acd0-433f-96c3-f4461690728f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3588066-acd0-433f-96c3-f4461690728f-maas-api-tls\") pod \"e3588066-acd0-433f-96c3-f4461690728f\" (UID: \"e3588066-acd0-433f-96c3-f4461690728f\") "
Nov 01 06:02:04.730527 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.730497    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e3588066-acd0-433f-96c3-f4461690728f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3588066-acd0-433f-96c3-f4461690728f-metrics-certs\") pod \"e3588066-acd0-433f-96c3-f4461690728f\" (UID: \"e3588066-acd0-433f-96c3-f4461690728f\") "
Nov 01 06:02:04.730604 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.730553    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3588066-acd0-433f-96c3-f4461690728f-kube-api-access-7fmtg\" (UniqueName: \"kubernetes.io/projected/e3588066-acd0-433f-96c3-f4461690728f-kube-api-access-7fmtg\") pod \"e3588066-acd0-433f-96c3-f4461690728f\" (UID: \"e3588066-acd0-433f-96c3-f4461690728f\") "
Nov 01 06:02:04.731422 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.731404    4864 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3588066-acd0-433f-96c3-f4461690728f-openshift-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:02:04.734140 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.734101    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3588066-acd0-433f-96c3-f4461690728f-metrics-certs" pod "e3588066-acd0-433f-96c3-f4461690728f" (UID: "e3588066-acd0-433f-96c3-f4461690728f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 06:02:04.734226 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.734159    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3588066-acd0-433f-96c3-f4461690728f-kube-api-access-7fmtg" pod "e3588066-acd0-433f-96c3-f4461690728f" (UID: "e3588066-acd0-433f-96c3-f4461690728f"). InnerVolumeSpecName "kube-api-access-7fmtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:02:04.734354 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.734334    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3588066-acd0-433f-96c3-f4461690728f-maas-api-tls" pod "e3588066-acd0-433f-96c3-f4461690728f" (UID: "e3588066-acd0-433f-96c3-f4461690728f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 06:02:04.832562 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.832459    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3588066-acd0-433f-96c3-f4461690728f-maas-api-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:02:04.832562 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.832498    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3588066-acd0-433f-96c3-f4461690728f-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:02:04.832562 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:04.832514    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fmtg\" (UniqueName: \"kubernetes.io/projected/e3588066-acd0-433f-96c3-f4461690728f-kube-api-access-7fmtg\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:02:05.121230 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:05.121143    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"]
Nov 01 06:02:05.126553 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:05.126528    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-c73135-54b554556b-f4xmk"]
Nov 01 06:02:05.256036 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:05.255993    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3588066-acd0-433f-96c3-f4461690728f" path="/var/lib/kubelet/pods/e3588066-acd0-433f-96c3-f4461690728f/volumes"
Nov 01 06:02:07.162343 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.162295    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz" podUID="d9a45d6a-720d-45f0-973e-a69b24c10cc7" containerName="main" containerID="cri-o://70b5cc3a6a7b5758a3de0ef0e400e782eedeab635ab720a19405b1687e8bb575" gracePeriod=45
Nov 01 06:02:07.439382 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.439358    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 06:02:07.576387 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.576347    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-model-cache\") pod \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") "
Nov 01 06:02:07.576563 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.576495    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-tmp-dir\") pod \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") "
Nov 01 06:02:07.576563 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.576549    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-home\" (UniqueName: \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-home\") pod \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") "
Nov 01 06:02:07.576700 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.576598    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9a45d6a-720d-45f0-973e-a69b24c10cc7-kube-api-access-58c8l\" (UniqueName: \"kubernetes.io/projected/d9a45d6a-720d-45f0-973e-a69b24c10cc7-kube-api-access-58c8l\") pod \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") "
Nov 01 06:02:07.576700 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.576622    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-model-cache" pod "d9a45d6a-720d-45f0-973e-a69b24c10cc7" (UID: "d9a45d6a-720d-45f0-973e-a69b24c10cc7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:02:07.576700 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.576640    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-dshm\") pod \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") "
Nov 01 06:02:07.576917 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.576697    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d9a45d6a-720d-45f0-973e-a69b24c10cc7-tls-certs\" (UniqueName: \"kubernetes.io/secret/d9a45d6a-720d-45f0-973e-a69b24c10cc7-tls-certs\") pod \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\" (UID: \"d9a45d6a-720d-45f0-973e-a69b24c10cc7\") "
Nov 01 06:02:07.577078 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.577041    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-home" pod "d9a45d6a-720d-45f0-973e-a69b24c10cc7" (UID: "d9a45d6a-720d-45f0-973e-a69b24c10cc7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:02:07.577256 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.577173    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-tmp-dir" pod "d9a45d6a-720d-45f0-973e-a69b24c10cc7" (UID: "d9a45d6a-720d-45f0-973e-a69b24c10cc7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:02:07.577659 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.577584    4864 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-model-cache\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:02:07.577659 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.577613    4864 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-tmp-dir\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:02:07.577659 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.577627    4864 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-home\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:02:07.579492 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.579458    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-dshm" pod "d9a45d6a-720d-45f0-973e-a69b24c10cc7" (UID: "d9a45d6a-720d-45f0-973e-a69b24c10cc7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:02:07.579492 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.579480    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9a45d6a-720d-45f0-973e-a69b24c10cc7-tls-certs" pod "d9a45d6a-720d-45f0-973e-a69b24c10cc7" (UID: "d9a45d6a-720d-45f0-973e-a69b24c10cc7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 06:02:07.581321 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.581298    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9a45d6a-720d-45f0-973e-a69b24c10cc7-kube-api-access-58c8l" pod "d9a45d6a-720d-45f0-973e-a69b24c10cc7" (UID: "d9a45d6a-720d-45f0-973e-a69b24c10cc7"). InnerVolumeSpecName "kube-api-access-58c8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:02:07.679264 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.679130    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58c8l\" (UniqueName: \"kubernetes.io/projected/d9a45d6a-720d-45f0-973e-a69b24c10cc7-kube-api-access-58c8l\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:02:07.679264 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.679171    4864 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9a45d6a-720d-45f0-973e-a69b24c10cc7-dshm\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:02:07.679264 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.679224    4864 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9a45d6a-720d-45f0-973e-a69b24c10cc7-tls-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:02:07.727120 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.727088    4864 generic.go:356] "Generic (PLEG): container finished" podID="d9a45d6a-720d-45f0-973e-a69b24c10cc7" containerID="70b5cc3a6a7b5758a3de0ef0e400e782eedeab635ab720a19405b1687e8bb575" exitCode=0
Nov 01 06:02:07.727288 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.727139    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"
Nov 01 06:02:07.727288 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.727175    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz" event={"ID":"d9a45d6a-720d-45f0-973e-a69b24c10cc7","Type":"ContainerDied","Data":"70b5cc3a6a7b5758a3de0ef0e400e782eedeab635ab720a19405b1687e8bb575"}
Nov 01 06:02:07.727288 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.727223    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz" event={"ID":"d9a45d6a-720d-45f0-973e-a69b24c10cc7","Type":"ContainerDied","Data":"ef0b3953bad4a0473550597eb057d25c61b39bd8a2d38f2453b824217f394615"}
Nov 01 06:02:07.727288 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.727244    4864 scope.go:122] "RemoveContainer" containerID="70b5cc3a6a7b5758a3de0ef0e400e782eedeab635ab720a19405b1687e8bb575"
Nov 01 06:02:07.742143 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.742122    4864 scope.go:122] "RemoveContainer" containerID="70b5cc3a6a7b5758a3de0ef0e400e782eedeab635ab720a19405b1687e8bb575"
Nov 01 06:02:07.742444 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:02:07.742427    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70b5cc3a6a7b5758a3de0ef0e400e782eedeab635ab720a19405b1687e8bb575\": container with ID starting with 70b5cc3a6a7b5758a3de0ef0e400e782eedeab635ab720a19405b1687e8bb575 not found: ID does not exist" containerID="70b5cc3a6a7b5758a3de0ef0e400e782eedeab635ab720a19405b1687e8bb575"
Nov 01 06:02:07.742495 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.742453    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70b5cc3a6a7b5758a3de0ef0e400e782eedeab635ab720a19405b1687e8bb575"} err="failed to get container status \"70b5cc3a6a7b5758a3de0ef0e400e782eedeab635ab720a19405b1687e8bb575\": rpc error: code = NotFound desc = could not find container \"70b5cc3a6a7b5758a3de0ef0e400e782eedeab635ab720a19405b1687e8bb575\": container with ID starting with 70b5cc3a6a7b5758a3de0ef0e400e782eedeab635ab720a19405b1687e8bb575 not found: ID does not exist"
Nov 01 06:02:07.891730 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.891693    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"]
Nov 01 06:02:07.896774 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:07.896746    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-a48099/e2e-conflict-a-9da93470-kserve-7b54449bc4-7wxsz"]
Nov 01 06:02:09.247326 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:09.247295    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9a45d6a-720d-45f0-973e-a69b24c10cc7" path="/var/lib/kubelet/pods/d9a45d6a-720d-45f0-973e-a69b24c10cc7/volumes"
Nov 01 06:02:22.526232 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:02:22.526189    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:02:51.577114 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:51.577081    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"]
Nov 01 06:02:51.577621 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:51.577333    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4" podUID="48c3060a-f1c5-4ca4-8955-3b80c61eacbd" containerName="maas-api" containerID="cri-o://9fca6249a2e9333d6bbf022d51330919346d8124b7bf5621133dd734a1344d89" gracePeriod=30
Nov 01 06:02:51.899507 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:51.899480    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"
Nov 01 06:02:52.015258 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:52.015221    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-kube-api-access-zxq5t\" (UniqueName: \"kubernetes.io/projected/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-kube-api-access-zxq5t\") pod \"48c3060a-f1c5-4ca4-8955-3b80c61eacbd\" (UID: \"48c3060a-f1c5-4ca4-8955-3b80c61eacbd\") "
Nov 01 06:02:52.040958 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:52.040917    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-openshift-service-ca\") pod \"48c3060a-f1c5-4ca4-8955-3b80c61eacbd\" (UID: \"48c3060a-f1c5-4ca4-8955-3b80c61eacbd\") "
Nov 01 06:02:52.041190 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:52.041049    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-metrics-certs\") pod \"48c3060a-f1c5-4ca4-8955-3b80c61eacbd\" (UID: \"48c3060a-f1c5-4ca4-8955-3b80c61eacbd\") "
Nov 01 06:02:52.041270 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:52.041197    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-maas-api-tls\") pod \"48c3060a-f1c5-4ca4-8955-3b80c61eacbd\" (UID: \"48c3060a-f1c5-4ca4-8955-3b80c61eacbd\") "
Nov 01 06:02:52.042970 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:52.042931    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-kube-api-access-zxq5t" pod "48c3060a-f1c5-4ca4-8955-3b80c61eacbd" (UID: "48c3060a-f1c5-4ca4-8955-3b80c61eacbd"). InnerVolumeSpecName "kube-api-access-zxq5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:02:52.043337 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:52.043308    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-openshift-service-ca" pod "48c3060a-f1c5-4ca4-8955-3b80c61eacbd" (UID: "48c3060a-f1c5-4ca4-8955-3b80c61eacbd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 06:02:52.045534 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:52.045504    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-maas-api-tls" pod "48c3060a-f1c5-4ca4-8955-3b80c61eacbd" (UID: "48c3060a-f1c5-4ca4-8955-3b80c61eacbd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 06:02:52.057631 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:52.057593    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-metrics-certs" pod "48c3060a-f1c5-4ca4-8955-3b80c61eacbd" (UID: "48c3060a-f1c5-4ca4-8955-3b80c61eacbd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 06:02:52.086224 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:52.086188    4864 generic.go:356] "Generic (PLEG): container finished" podID="48c3060a-f1c5-4ca4-8955-3b80c61eacbd" containerID="9fca6249a2e9333d6bbf022d51330919346d8124b7bf5621133dd734a1344d89" exitCode=0
Nov 01 06:02:52.086415 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:52.086247    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"
Nov 01 06:02:52.086415 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:52.086264    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4" event={"ID":"48c3060a-f1c5-4ca4-8955-3b80c61eacbd","Type":"ContainerDied","Data":"9fca6249a2e9333d6bbf022d51330919346d8124b7bf5621133dd734a1344d89"}
Nov 01 06:02:52.086415 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:52.086355    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4" event={"ID":"48c3060a-f1c5-4ca4-8955-3b80c61eacbd","Type":"ContainerDied","Data":"2b4d13f87c265fa78c7b04f5098c48e1408e8b88012c28485d9563ff4cabbafb"}
Nov 01 06:02:52.086415 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:52.086386    4864 scope.go:122] "RemoveContainer" containerID="9fca6249a2e9333d6bbf022d51330919346d8124b7bf5621133dd734a1344d89"
Nov 01 06:02:52.097838 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:52.097821    4864 scope.go:122] "RemoveContainer" containerID="9fca6249a2e9333d6bbf022d51330919346d8124b7bf5621133dd734a1344d89"
Nov 01 06:02:52.098112 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:02:52.098094    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fca6249a2e9333d6bbf022d51330919346d8124b7bf5621133dd734a1344d89\": container with ID starting with 9fca6249a2e9333d6bbf022d51330919346d8124b7bf5621133dd734a1344d89 not found: ID does not exist" containerID="9fca6249a2e9333d6bbf022d51330919346d8124b7bf5621133dd734a1344d89"
Nov 01 06:02:52.098163 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:52.098121    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fca6249a2e9333d6bbf022d51330919346d8124b7bf5621133dd734a1344d89"} err="failed to get container status \"9fca6249a2e9333d6bbf022d51330919346d8124b7bf5621133dd734a1344d89\": rpc error: code = NotFound desc = could not find container \"9fca6249a2e9333d6bbf022d51330919346d8124b7bf5621133dd734a1344d89\": container with ID starting with 9fca6249a2e9333d6bbf022d51330919346d8124b7bf5621133dd734a1344d89 not found: ID does not exist"
Nov 01 06:02:52.142658 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:52.142588    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxq5t\" (UniqueName: \"kubernetes.io/projected/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-kube-api-access-zxq5t\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:02:52.142658 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:52.142616    4864 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-openshift-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:02:52.142658 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:52.142628    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:02:52.142658 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:52.142637    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48c3060a-f1c5-4ca4-8955-3b80c61eacbd-maas-api-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:02:52.249314 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:52.249281    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"]
Nov 01 06:02:52.255148 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:52.255122    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-dee6aa-59bfc7c446-wm9p4"]
Nov 01 06:02:52.524921 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:02:52.524841    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:02:53.248427 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:02:53.248392    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="48c3060a-f1c5-4ca4-8955-3b80c61eacbd" path="/var/lib/kubelet/pods/48c3060a-f1c5-4ca4-8955-3b80c61eacbd/volumes"
Nov 01 06:03:04.117993 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:04.117958    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 06:03:04.128019 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:04.127993    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 06:03:04.129836 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:04.129808    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 06:03:04.138595 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:04.138561    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 06:03:09.356002 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:09.355969    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"]
Nov 01 06:03:09.356417 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:09.356188    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv" podUID="cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22" containerName="maas-api" containerID="cri-o://12a553e1f7c8deecd9f6082be33a9272b262b686059a9adcbedd09b4a8496838" gracePeriod=30
Nov 01 06:03:09.673910 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:09.673581    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"
Nov 01 06:03:09.762491 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:09.762459    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-maas-api-tls\") pod \"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22\" (UID: \"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22\") "
Nov 01 06:03:09.762491 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:09.762501    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-metrics-certs\") pod \"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22\" (UID: \"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22\") "
Nov 01 06:03:09.762757 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:09.762723    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-openshift-service-ca\") pod \"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22\" (UID: \"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22\") "
Nov 01 06:03:09.762922 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:09.762885    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-kube-api-access-hqkrc\" (UniqueName: \"kubernetes.io/projected/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-kube-api-access-hqkrc\") pod \"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22\" (UID: \"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22\") "
Nov 01 06:03:09.763139 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:09.763067    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-openshift-service-ca" pod "cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22" (UID: "cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 06:03:09.763550 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:09.763528    4864 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-openshift-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:09.764852 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:09.764831    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-metrics-certs" pod "cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22" (UID: "cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 06:03:09.764964 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:09.764886    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-maas-api-tls" pod "cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22" (UID: "cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 06:03:09.765396 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:09.765371    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-kube-api-access-hqkrc" pod "cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22" (UID: "cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22"). InnerVolumeSpecName "kube-api-access-hqkrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:03:09.865219 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:09.865183    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-maas-api-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:09.865219 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:09.865220    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:09.865412 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:09.865238    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqkrc\" (UniqueName: \"kubernetes.io/projected/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22-kube-api-access-hqkrc\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:10.245128 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:10.245093    4864 generic.go:356] "Generic (PLEG): container finished" podID="cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22" containerID="12a553e1f7c8deecd9f6082be33a9272b262b686059a9adcbedd09b4a8496838" exitCode=0
Nov 01 06:03:10.245327 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:10.245137    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"
Nov 01 06:03:10.245327 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:10.245178    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv" event={"ID":"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22","Type":"ContainerDied","Data":"12a553e1f7c8deecd9f6082be33a9272b262b686059a9adcbedd09b4a8496838"}
Nov 01 06:03:10.245327 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:10.245237    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv" event={"ID":"cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22","Type":"ContainerDied","Data":"de405e3228c74ca0fe30297b0b2b1e78a4609b6c9e29a6103978848442e2d9c3"}
Nov 01 06:03:10.245327 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:10.245266    4864 scope.go:122] "RemoveContainer" containerID="12a553e1f7c8deecd9f6082be33a9272b262b686059a9adcbedd09b4a8496838"
Nov 01 06:03:10.256117 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:10.256098    4864 scope.go:122] "RemoveContainer" containerID="12a553e1f7c8deecd9f6082be33a9272b262b686059a9adcbedd09b4a8496838"
Nov 01 06:03:10.256379 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:03:10.256358    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12a553e1f7c8deecd9f6082be33a9272b262b686059a9adcbedd09b4a8496838\": container with ID starting with 12a553e1f7c8deecd9f6082be33a9272b262b686059a9adcbedd09b4a8496838 not found: ID does not exist" containerID="12a553e1f7c8deecd9f6082be33a9272b262b686059a9adcbedd09b4a8496838"
Nov 01 06:03:10.256441 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:10.256389    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12a553e1f7c8deecd9f6082be33a9272b262b686059a9adcbedd09b4a8496838"} err="failed to get container status \"12a553e1f7c8deecd9f6082be33a9272b262b686059a9adcbedd09b4a8496838\": rpc error: code = NotFound desc = could not find container \"12a553e1f7c8deecd9f6082be33a9272b262b686059a9adcbedd09b4a8496838\": container with ID starting with 12a553e1f7c8deecd9f6082be33a9272b262b686059a9adcbedd09b4a8496838 not found: ID does not exist"
Nov 01 06:03:10.409485 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:10.409457    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"]
Nov 01 06:03:10.415391 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:10.415363    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-7a360a-68c468896f-5m5jv"]
Nov 01 06:03:11.248025 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:11.247993    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22" path="/var/lib/kubelet/pods/cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22/volumes"
Nov 01 06:03:14.699682 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:14.699641    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"]
Nov 01 06:03:14.700187 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:14.699911    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft" podUID="702e5c95-5de0-4329-bae0-404d4d65a941" containerName="istio-proxy" containerID="cri-o://48c337a1fd2312924241a6ec34c583defe054e0246ee34e71ec2c8a4975e3d1d" gracePeriod=30
Nov 01 06:03:19.753761 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:03:19.753735    4864 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod702e5c95_5de0_4329_bae0_404d4d65a941.slice/crio-48c337a1fd2312924241a6ec34c583defe054e0246ee34e71ec2c8a4975e3d1d.scope\": RecentStats: unable to find data in memory cache]"
Nov 01 06:03:19.996372 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:19.996348    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 06:03:20.079537 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.079500    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/702e5c95-5de0-4329-bae0-404d4d65a941-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/702e5c95-5de0-4329-bae0-404d4d65a941-istio-podinfo\") pod \"702e5c95-5de0-4329-bae0-404d4d65a941\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") "
Nov 01 06:03:20.079668 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.079573    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/702e5c95-5de0-4329-bae0-404d4d65a941-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/702e5c95-5de0-4329-bae0-404d4d65a941-istiod-ca-cert\") pod \"702e5c95-5de0-4329-bae0-404d4d65a941\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") "
Nov 01 06:03:20.079724 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.079670    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/702e5c95-5de0-4329-bae0-404d4d65a941-istio-token\" (UniqueName: \"kubernetes.io/projected/702e5c95-5de0-4329-bae0-404d4d65a941-istio-token\") pod \"702e5c95-5de0-4329-bae0-404d4d65a941\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") "
Nov 01 06:03:20.079767 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.079718    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-workload-certs\") pod \"702e5c95-5de0-4329-bae0-404d4d65a941\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") "
Nov 01 06:03:20.079845 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.079821    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-credential-socket\") pod \"702e5c95-5de0-4329-bae0-404d4d65a941\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") "
Nov 01 06:03:20.079905 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.079867    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-istio-data\") pod \"702e5c95-5de0-4329-bae0-404d4d65a941\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") "
Nov 01 06:03:20.079960 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.079903    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-istio-envoy\") pod \"702e5c95-5de0-4329-bae0-404d4d65a941\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") "
Nov 01 06:03:20.080064 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.079955    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/702e5c95-5de0-4329-bae0-404d4d65a941-kube-api-access-8hmq5\" (UniqueName: \"kubernetes.io/projected/702e5c95-5de0-4329-bae0-404d4d65a941-kube-api-access-8hmq5\") pod \"702e5c95-5de0-4329-bae0-404d4d65a941\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") "
Nov 01 06:03:20.080064 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.079988    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-workload-socket\") pod \"702e5c95-5de0-4329-bae0-404d4d65a941\" (UID: \"702e5c95-5de0-4329-bae0-404d4d65a941\") "
Nov 01 06:03:20.080209 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.080078    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/702e5c95-5de0-4329-bae0-404d4d65a941-istiod-ca-cert" pod "702e5c95-5de0-4329-bae0-404d4d65a941" (UID: "702e5c95-5de0-4329-bae0-404d4d65a941"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 06:03:20.080209 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.080113    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-credential-socket" pod "702e5c95-5de0-4329-bae0-404d4d65a941" (UID: "702e5c95-5de0-4329-bae0-404d4d65a941"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:03:20.080344 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.080325    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-workload-socket" pod "702e5c95-5de0-4329-bae0-404d4d65a941" (UID: "702e5c95-5de0-4329-bae0-404d4d65a941"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:03:20.080404 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.080342    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-istio-data" pod "702e5c95-5de0-4329-bae0-404d4d65a941" (UID: "702e5c95-5de0-4329-bae0-404d4d65a941"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:03:20.080459 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.080414    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-workload-certs" pod "702e5c95-5de0-4329-bae0-404d4d65a941" (UID: "702e5c95-5de0-4329-bae0-404d4d65a941"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:03:20.080513 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.080490    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:20.080570 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.080514    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:20.080570 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.080529    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:20.080570 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.080544    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/702e5c95-5de0-4329-bae0-404d4d65a941-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:20.082434 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.082409    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/702e5c95-5de0-4329-bae0-404d4d65a941-istio-podinfo" pod "702e5c95-5de0-4329-bae0-404d4d65a941" (UID: "702e5c95-5de0-4329-bae0-404d4d65a941"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 06:03:20.082553 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.082444    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/702e5c95-5de0-4329-bae0-404d4d65a941-kube-api-access-8hmq5" pod "702e5c95-5de0-4329-bae0-404d4d65a941" (UID: "702e5c95-5de0-4329-bae0-404d4d65a941"). InnerVolumeSpecName "kube-api-access-8hmq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:03:20.082553 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.082523    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/702e5c95-5de0-4329-bae0-404d4d65a941-istio-token" pod "702e5c95-5de0-4329-bae0-404d4d65a941" (UID: "702e5c95-5de0-4329-bae0-404d4d65a941"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:03:20.082645 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.082604    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-istio-envoy" pod "702e5c95-5de0-4329-bae0-404d4d65a941" (UID: "702e5c95-5de0-4329-bae0-404d4d65a941"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:03:20.128444 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.128420    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"]
Nov 01 06:03:20.148677 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.148655    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"]
Nov 01 06:03:20.181267 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.181234    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hmq5\" (UniqueName: \"kubernetes.io/projected/702e5c95-5de0-4329-bae0-404d4d65a941-kube-api-access-8hmq5\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:20.181267 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.181262    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/702e5c95-5de0-4329-bae0-404d4d65a941-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:20.181267 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.181272    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/702e5c95-5de0-4329-bae0-404d4d65a941-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:20.181489 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.181281    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:20.181489 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.181289    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/702e5c95-5de0-4329-bae0-404d4d65a941-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:20.323762 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.323729    4864 generic.go:356] "Generic (PLEG): container finished" podID="702e5c95-5de0-4329-bae0-404d4d65a941" containerID="48c337a1fd2312924241a6ec34c583defe054e0246ee34e71ec2c8a4975e3d1d" exitCode=0
Nov 01 06:03:20.323945 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.323779    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"
Nov 01 06:03:20.323945 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.323829    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft" event={"ID":"702e5c95-5de0-4329-bae0-404d4d65a941","Type":"ContainerDied","Data":"48c337a1fd2312924241a6ec34c583defe054e0246ee34e71ec2c8a4975e3d1d"}
Nov 01 06:03:20.323945 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.323883    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft" event={"ID":"702e5c95-5de0-4329-bae0-404d4d65a941","Type":"ContainerDied","Data":"4597466e518e0419e476aaf48d770cd2ce0f1613c82a536d24071401ba427a0e"}
Nov 01 06:03:20.323945 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.323909    4864 scope.go:122] "RemoveContainer" containerID="48c337a1fd2312924241a6ec34c583defe054e0246ee34e71ec2c8a4975e3d1d"
Nov 01 06:03:20.334772 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.334755    4864 scope.go:122] "RemoveContainer" containerID="8d9fe1177dbd35d166b45319dbf191e197230b13b42d2e12cc3ed1f57341acd7"
Nov 01 06:03:20.343675 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.343656    4864 scope.go:122] "RemoveContainer" containerID="48c337a1fd2312924241a6ec34c583defe054e0246ee34e71ec2c8a4975e3d1d"
Nov 01 06:03:20.343957 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:03:20.343940    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48c337a1fd2312924241a6ec34c583defe054e0246ee34e71ec2c8a4975e3d1d\": container with ID starting with 48c337a1fd2312924241a6ec34c583defe054e0246ee34e71ec2c8a4975e3d1d not found: ID does not exist" containerID="48c337a1fd2312924241a6ec34c583defe054e0246ee34e71ec2c8a4975e3d1d"
Nov 01 06:03:20.344022 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.343965    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48c337a1fd2312924241a6ec34c583defe054e0246ee34e71ec2c8a4975e3d1d"} err="failed to get container status \"48c337a1fd2312924241a6ec34c583defe054e0246ee34e71ec2c8a4975e3d1d\": rpc error: code = NotFound desc = could not find container \"48c337a1fd2312924241a6ec34c583defe054e0246ee34e71ec2c8a4975e3d1d\": container with ID starting with 48c337a1fd2312924241a6ec34c583defe054e0246ee34e71ec2c8a4975e3d1d not found: ID does not exist"
Nov 01 06:03:20.344022 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.343981    4864 scope.go:122] "RemoveContainer" containerID="8d9fe1177dbd35d166b45319dbf191e197230b13b42d2e12cc3ed1f57341acd7"
Nov 01 06:03:20.344214 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:03:20.344193    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d9fe1177dbd35d166b45319dbf191e197230b13b42d2e12cc3ed1f57341acd7\": container with ID starting with 8d9fe1177dbd35d166b45319dbf191e197230b13b42d2e12cc3ed1f57341acd7 not found: ID does not exist" containerID="8d9fe1177dbd35d166b45319dbf191e197230b13b42d2e12cc3ed1f57341acd7"
Nov 01 06:03:20.344270 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.344222    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d9fe1177dbd35d166b45319dbf191e197230b13b42d2e12cc3ed1f57341acd7"} err="failed to get container status \"8d9fe1177dbd35d166b45319dbf191e197230b13b42d2e12cc3ed1f57341acd7\": rpc error: code = NotFound desc = could not find container \"8d9fe1177dbd35d166b45319dbf191e197230b13b42d2e12cc3ed1f57341acd7\": container with ID starting with 8d9fe1177dbd35d166b45319dbf191e197230b13b42d2e12cc3ed1f57341acd7 not found: ID does not exist"
Nov 01 06:03:20.517681 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.517642    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"]
Nov 01 06:03:20.524154 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:20.524128    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft"]
Nov 01 06:03:21.249449 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:21.249415    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="702e5c95-5de0-4329-bae0-404d4d65a941" path="/var/lib/kubelet/pods/702e5c95-5de0-4329-bae0-404d4d65a941/volumes"
Nov 01 06:03:22.524571 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:03:22.524531    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:03:35.148694 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.148653    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8" podUID="071c5c98-548c-439b-b8f6-3b07c70df932" containerName="main" containerID="cri-o://4862b1f9f66e9423b2516a3e1a9f5edfa6857d83c9dc9a59cf40c8d326e25fa0" gracePeriod=45
Nov 01 06:03:35.170220 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.170187    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx" podUID="7b847150-a3e8-4210-af71-4f2ec0010c03" containerName="main" containerID="cri-o://b8b9666ddce88d036d5be635b865379341eba436f0356259664609abd9f4dd43" gracePeriod=45
Nov 01 06:03:35.438853 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.438829    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 06:03:35.470850 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.470764    4864 generic.go:356] "Generic (PLEG): container finished" podID="071c5c98-548c-439b-b8f6-3b07c70df932" containerID="4862b1f9f66e9423b2516a3e1a9f5edfa6857d83c9dc9a59cf40c8d326e25fa0" exitCode=0
Nov 01 06:03:35.470850 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.470816    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"
Nov 01 06:03:35.471036 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.470816    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8" event={"ID":"071c5c98-548c-439b-b8f6-3b07c70df932","Type":"ContainerDied","Data":"4862b1f9f66e9423b2516a3e1a9f5edfa6857d83c9dc9a59cf40c8d326e25fa0"}
Nov 01 06:03:35.471036 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.470920    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8" event={"ID":"071c5c98-548c-439b-b8f6-3b07c70df932","Type":"ContainerDied","Data":"e92d7fc255b0a1a837b5043814ec5bc161b235f3e7320878aca7e9042f32f0ce"}
Nov 01 06:03:35.471036 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.470936    4864 scope.go:122] "RemoveContainer" containerID="4862b1f9f66e9423b2516a3e1a9f5edfa6857d83c9dc9a59cf40c8d326e25fa0"
Nov 01 06:03:35.472329 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.472305    4864 generic.go:356] "Generic (PLEG): container finished" podID="7b847150-a3e8-4210-af71-4f2ec0010c03" containerID="b8b9666ddce88d036d5be635b865379341eba436f0356259664609abd9f4dd43" exitCode=0
Nov 01 06:03:35.472426 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.472377    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx" event={"ID":"7b847150-a3e8-4210-af71-4f2ec0010c03","Type":"ContainerDied","Data":"b8b9666ddce88d036d5be635b865379341eba436f0356259664609abd9f4dd43"}
Nov 01 06:03:35.472426 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.472416    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx" event={"ID":"7b847150-a3e8-4210-af71-4f2ec0010c03","Type":"ContainerDied","Data":"6fdc72a3113734ca7a0f21771e646fe5460a2b1392c50b907522c4fe4bb547cb"}
Nov 01 06:03:35.472512 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.472434    4864 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6fdc72a3113734ca7a0f21771e646fe5460a2b1392c50b907522c4fe4bb547cb"
Nov 01 06:03:35.479565 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.479545    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 06:03:35.480807 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.480773    4864 scope.go:122] "RemoveContainer" containerID="4862b1f9f66e9423b2516a3e1a9f5edfa6857d83c9dc9a59cf40c8d326e25fa0"
Nov 01 06:03:35.481138 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:03:35.481121    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4862b1f9f66e9423b2516a3e1a9f5edfa6857d83c9dc9a59cf40c8d326e25fa0\": container with ID starting with 4862b1f9f66e9423b2516a3e1a9f5edfa6857d83c9dc9a59cf40c8d326e25fa0 not found: ID does not exist" containerID="4862b1f9f66e9423b2516a3e1a9f5edfa6857d83c9dc9a59cf40c8d326e25fa0"
Nov 01 06:03:35.481190 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.481148    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4862b1f9f66e9423b2516a3e1a9f5edfa6857d83c9dc9a59cf40c8d326e25fa0"} err="failed to get container status \"4862b1f9f66e9423b2516a3e1a9f5edfa6857d83c9dc9a59cf40c8d326e25fa0\": rpc error: code = NotFound desc = could not find container \"4862b1f9f66e9423b2516a3e1a9f5edfa6857d83c9dc9a59cf40c8d326e25fa0\": container with ID starting with 4862b1f9f66e9423b2516a3e1a9f5edfa6857d83c9dc9a59cf40c8d326e25fa0 not found: ID does not exist"
Nov 01 06:03:35.546838 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.546775    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-tmp-dir\") pod \"071c5c98-548c-439b-b8f6-3b07c70df932\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") "
Nov 01 06:03:35.547023 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.546856    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-home\" (UniqueName: \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-home\") pod \"071c5c98-548c-439b-b8f6-3b07c70df932\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") "
Nov 01 06:03:35.547023 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.546889    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-home\" (UniqueName: \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-home\") pod \"7b847150-a3e8-4210-af71-4f2ec0010c03\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") "
Nov 01 06:03:35.547023 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.546959    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-tmp-dir\") pod \"7b847150-a3e8-4210-af71-4f2ec0010c03\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") "
Nov 01 06:03:35.547213 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.547063    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-dshm\" (UniqueName: \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-dshm\") pod \"071c5c98-548c-439b-b8f6-3b07c70df932\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") "
Nov 01 06:03:35.547213 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.547059    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-tmp-dir" pod "071c5c98-548c-439b-b8f6-3b07c70df932" (UID: "071c5c98-548c-439b-b8f6-3b07c70df932"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:03:35.547213 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.547104    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7b847150-a3e8-4210-af71-4f2ec0010c03-tls-certs\" (UniqueName: \"kubernetes.io/secret/7b847150-a3e8-4210-af71-4f2ec0010c03-tls-certs\") pod \"7b847150-a3e8-4210-af71-4f2ec0010c03\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") "
Nov 01 06:03:35.547213 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.547133    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-model-cache\") pod \"071c5c98-548c-439b-b8f6-3b07c70df932\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") "
Nov 01 06:03:35.547213 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.547156    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-home" pod "071c5c98-548c-439b-b8f6-3b07c70df932" (UID: "071c5c98-548c-439b-b8f6-3b07c70df932"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:03:35.547213 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.547188    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/071c5c98-548c-439b-b8f6-3b07c70df932-tls-certs\" (UniqueName: \"kubernetes.io/secret/071c5c98-548c-439b-b8f6-3b07c70df932-tls-certs\") pod \"071c5c98-548c-439b-b8f6-3b07c70df932\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") "
Nov 01 06:03:35.547531 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.547217    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-model-cache\") pod \"7b847150-a3e8-4210-af71-4f2ec0010c03\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") "
Nov 01 06:03:35.547531 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.547279    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/071c5c98-548c-439b-b8f6-3b07c70df932-kube-api-access-449jj\" (UniqueName: \"kubernetes.io/projected/071c5c98-548c-439b-b8f6-3b07c70df932-kube-api-access-449jj\") pod \"071c5c98-548c-439b-b8f6-3b07c70df932\" (UID: \"071c5c98-548c-439b-b8f6-3b07c70df932\") "
Nov 01 06:03:35.547531 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.547289    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-tmp-dir" pod "7b847150-a3e8-4210-af71-4f2ec0010c03" (UID: "7b847150-a3e8-4210-af71-4f2ec0010c03"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:03:35.547531 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.547323    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-dshm\") pod \"7b847150-a3e8-4210-af71-4f2ec0010c03\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") "
Nov 01 06:03:35.547531 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.547353    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7b847150-a3e8-4210-af71-4f2ec0010c03-kube-api-access-kn8wq\" (UniqueName: \"kubernetes.io/projected/7b847150-a3e8-4210-af71-4f2ec0010c03-kube-api-access-kn8wq\") pod \"7b847150-a3e8-4210-af71-4f2ec0010c03\" (UID: \"7b847150-a3e8-4210-af71-4f2ec0010c03\") "
Nov 01 06:03:35.547531 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.547407    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-model-cache" pod "071c5c98-548c-439b-b8f6-3b07c70df932" (UID: "071c5c98-548c-439b-b8f6-3b07c70df932"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:03:35.547798 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.547596    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-model-cache" pod "7b847150-a3e8-4210-af71-4f2ec0010c03" (UID: "7b847150-a3e8-4210-af71-4f2ec0010c03"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:03:35.547910 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.547891    4864 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-tmp-dir\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:35.547983 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.547914    4864 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-model-cache\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:35.547983 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.547928    4864 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-model-cache\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:35.547983 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.547943    4864 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-tmp-dir\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:35.547983 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.547958    4864 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-home\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:35.548635 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.548606    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-home" pod "7b847150-a3e8-4210-af71-4f2ec0010c03" (UID: "7b847150-a3e8-4210-af71-4f2ec0010c03"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:03:35.550151 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.550123    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/071c5c98-548c-439b-b8f6-3b07c70df932-tls-certs" pod "071c5c98-548c-439b-b8f6-3b07c70df932" (UID: "071c5c98-548c-439b-b8f6-3b07c70df932"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 06:03:35.550566 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.550540    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b847150-a3e8-4210-af71-4f2ec0010c03-kube-api-access-kn8wq" pod "7b847150-a3e8-4210-af71-4f2ec0010c03" (UID: "7b847150-a3e8-4210-af71-4f2ec0010c03"). InnerVolumeSpecName "kube-api-access-kn8wq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:03:35.550774 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.550738    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/071c5c98-548c-439b-b8f6-3b07c70df932-kube-api-access-449jj" pod "071c5c98-548c-439b-b8f6-3b07c70df932" (UID: "071c5c98-548c-439b-b8f6-3b07c70df932"). InnerVolumeSpecName "kube-api-access-449jj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:03:35.550999 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.550964    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-dshm" pod "071c5c98-548c-439b-b8f6-3b07c70df932" (UID: "071c5c98-548c-439b-b8f6-3b07c70df932"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:03:35.551132 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.551113    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b847150-a3e8-4210-af71-4f2ec0010c03-tls-certs" pod "7b847150-a3e8-4210-af71-4f2ec0010c03" (UID: "7b847150-a3e8-4210-af71-4f2ec0010c03"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 06:03:35.551274 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.551249    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-dshm" pod "7b847150-a3e8-4210-af71-4f2ec0010c03" (UID: "7b847150-a3e8-4210-af71-4f2ec0010c03"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:03:35.649638 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.649600    4864 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-dshm\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:35.649638 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.649640    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kn8wq\" (UniqueName: \"kubernetes.io/projected/7b847150-a3e8-4210-af71-4f2ec0010c03-kube-api-access-kn8wq\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:35.649925 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.649654    4864 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b847150-a3e8-4210-af71-4f2ec0010c03-home\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:35.649925 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.649663    4864 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/071c5c98-548c-439b-b8f6-3b07c70df932-dshm\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:35.649925 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.649671    4864 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b847150-a3e8-4210-af71-4f2ec0010c03-tls-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:35.649925 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.649680    4864 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/071c5c98-548c-439b-b8f6-3b07c70df932-tls-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:35.649925 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.649688    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-449jj\" (UniqueName: \"kubernetes.io/projected/071c5c98-548c-439b-b8f6-3b07c70df932-kube-api-access-449jj\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:35.895505 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.895468    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"]
Nov 01 06:03:35.901320 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:35.901290    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-a48099/distinct-2-w4-a48099-kserve-8649ddd5-w7cz8"]
Nov 01 06:03:36.480926 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:36.480842    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"
Nov 01 06:03:36.643967 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:36.643934    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"]
Nov 01 06:03:36.693154 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:36.693119    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-a48099/premium-w4-a48099-kserve-6ccd45bcff-tp7mx"]
Nov 01 06:03:37.247321 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:37.247289    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="071c5c98-548c-439b-b8f6-3b07c70df932" path="/var/lib/kubelet/pods/071c5c98-548c-439b-b8f6-3b07c70df932/volumes"
Nov 01 06:03:37.247713 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:37.247700    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7b847150-a3e8-4210-af71-4f2ec0010c03" path="/var/lib/kubelet/pods/7b847150-a3e8-4210-af71-4f2ec0010c03/volumes"
Nov 01 06:03:41.202962 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:41.202926    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"]
Nov 01 06:03:41.203441 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:41.203147    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh" podUID="61a331da-3dde-4e55-bd6a-047a02b6e94d" containerName="maas-api" containerID="cri-o://f68628b31409b48f062134a5604b721cfca1b4c5b4770cca4aecc864dbad02d6" gracePeriod=30
Nov 01 06:03:41.519203 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:41.519172    4864 generic.go:356] "Generic (PLEG): container finished" podID="61a331da-3dde-4e55-bd6a-047a02b6e94d" containerID="f68628b31409b48f062134a5604b721cfca1b4c5b4770cca4aecc864dbad02d6" exitCode=0
Nov 01 06:03:41.519328 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:41.519250    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh" event={"ID":"61a331da-3dde-4e55-bd6a-047a02b6e94d","Type":"ContainerDied","Data":"f68628b31409b48f062134a5604b721cfca1b4c5b4770cca4aecc864dbad02d6"}
Nov 01 06:03:41.519328 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:41.519324    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh" event={"ID":"61a331da-3dde-4e55-bd6a-047a02b6e94d","Type":"ContainerDied","Data":"2e6495fc49b58ae4194142c558e925404967eb05aa9a7ac04f7a514307367309"}
Nov 01 06:03:41.519400 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:41.519340    4864 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e6495fc49b58ae4194142c558e925404967eb05aa9a7ac04f7a514307367309"
Nov 01 06:03:41.527082 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:41.527061    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"
Nov 01 06:03:41.619404 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:41.619367    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61a331da-3dde-4e55-bd6a-047a02b6e94d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/61a331da-3dde-4e55-bd6a-047a02b6e94d-metrics-certs\") pod \"61a331da-3dde-4e55-bd6a-047a02b6e94d\" (UID: \"61a331da-3dde-4e55-bd6a-047a02b6e94d\") "
Nov 01 06:03:41.619614 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:41.619414    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61a331da-3dde-4e55-bd6a-047a02b6e94d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61a331da-3dde-4e55-bd6a-047a02b6e94d-maas-api-tls\") pod \"61a331da-3dde-4e55-bd6a-047a02b6e94d\" (UID: \"61a331da-3dde-4e55-bd6a-047a02b6e94d\") "
Nov 01 06:03:41.619614 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:41.619507    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61a331da-3dde-4e55-bd6a-047a02b6e94d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61a331da-3dde-4e55-bd6a-047a02b6e94d-openshift-service-ca\") pod \"61a331da-3dde-4e55-bd6a-047a02b6e94d\" (UID: \"61a331da-3dde-4e55-bd6a-047a02b6e94d\") "
Nov 01 06:03:41.619614 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:41.619573    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61a331da-3dde-4e55-bd6a-047a02b6e94d-kube-api-access-bbqz6\" (UniqueName: \"kubernetes.io/projected/61a331da-3dde-4e55-bd6a-047a02b6e94d-kube-api-access-bbqz6\") pod \"61a331da-3dde-4e55-bd6a-047a02b6e94d\" (UID: \"61a331da-3dde-4e55-bd6a-047a02b6e94d\") "
Nov 01 06:03:41.620354 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:41.620312    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61a331da-3dde-4e55-bd6a-047a02b6e94d-openshift-service-ca" pod "61a331da-3dde-4e55-bd6a-047a02b6e94d" (UID: "61a331da-3dde-4e55-bd6a-047a02b6e94d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 06:03:41.622195 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:41.622158    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61a331da-3dde-4e55-bd6a-047a02b6e94d-kube-api-access-bbqz6" pod "61a331da-3dde-4e55-bd6a-047a02b6e94d" (UID: "61a331da-3dde-4e55-bd6a-047a02b6e94d"). InnerVolumeSpecName "kube-api-access-bbqz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:03:41.622550 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:41.622524    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61a331da-3dde-4e55-bd6a-047a02b6e94d-metrics-certs" pod "61a331da-3dde-4e55-bd6a-047a02b6e94d" (UID: "61a331da-3dde-4e55-bd6a-047a02b6e94d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 06:03:41.622779 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:41.622755    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61a331da-3dde-4e55-bd6a-047a02b6e94d-maas-api-tls" pod "61a331da-3dde-4e55-bd6a-047a02b6e94d" (UID: "61a331da-3dde-4e55-bd6a-047a02b6e94d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 06:03:41.720960 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:41.720931    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61a331da-3dde-4e55-bd6a-047a02b6e94d-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:41.720960 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:41.720957    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61a331da-3dde-4e55-bd6a-047a02b6e94d-maas-api-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:41.721127 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:41.720968    4864 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61a331da-3dde-4e55-bd6a-047a02b6e94d-openshift-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:41.721127 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:41.720978    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbqz6\" (UniqueName: \"kubernetes.io/projected/61a331da-3dde-4e55-bd6a-047a02b6e94d-kube-api-access-bbqz6\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:03:42.524888 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:42.524814    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"
Nov 01 06:03:42.688135 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:42.688097    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"]
Nov 01 06:03:42.737585 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:42.737554    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-098e4d-6bfbc5bcd7-bqtmh"]
Nov 01 06:03:43.246275 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:03:43.246245    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61a331da-3dde-4e55-bd6a-047a02b6e94d" path="/var/lib/kubelet/pods/61a331da-3dde-4e55-bd6a-047a02b6e94d/volumes"
Nov 01 06:03:52.526223 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:03:52.526184    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:04:03.589264 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:03.589226    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"]
Nov 01 06:04:03.589821 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:03.589478    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q" podUID="2143de82-6f2b-403c-a702-f248c5013318" containerName="maas-api" containerID="cri-o://c588e1cee2c779b3c031622eedf53fd68dea3939d6d526d055ac3895a48c8711" gracePeriod=30
Nov 01 06:04:03.731245 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:03.731214    4864 generic.go:356] "Generic (PLEG): container finished" podID="2143de82-6f2b-403c-a702-f248c5013318" containerID="c588e1cee2c779b3c031622eedf53fd68dea3939d6d526d055ac3895a48c8711" exitCode=0
Nov 01 06:04:03.731430 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:03.731283    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q" event={"ID":"2143de82-6f2b-403c-a702-f248c5013318","Type":"ContainerDied","Data":"c588e1cee2c779b3c031622eedf53fd68dea3939d6d526d055ac3895a48c8711"}
Nov 01 06:04:03.897569 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:03.897543    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"
Nov 01 06:04:03.954427 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:03.954392    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2143de82-6f2b-403c-a702-f248c5013318-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2143de82-6f2b-403c-a702-f248c5013318-openshift-service-ca\") pod \"2143de82-6f2b-403c-a702-f248c5013318\" (UID: \"2143de82-6f2b-403c-a702-f248c5013318\") "
Nov 01 06:04:03.954597 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:03.954474    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2143de82-6f2b-403c-a702-f248c5013318-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2143de82-6f2b-403c-a702-f248c5013318-maas-api-tls\") pod \"2143de82-6f2b-403c-a702-f248c5013318\" (UID: \"2143de82-6f2b-403c-a702-f248c5013318\") "
Nov 01 06:04:03.954597 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:03.954531    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2143de82-6f2b-403c-a702-f248c5013318-kube-api-access-vln7s\" (UniqueName: \"kubernetes.io/projected/2143de82-6f2b-403c-a702-f248c5013318-kube-api-access-vln7s\") pod \"2143de82-6f2b-403c-a702-f248c5013318\" (UID: \"2143de82-6f2b-403c-a702-f248c5013318\") "
Nov 01 06:04:03.954717 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:03.954602    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2143de82-6f2b-403c-a702-f248c5013318-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2143de82-6f2b-403c-a702-f248c5013318-metrics-certs\") pod \"2143de82-6f2b-403c-a702-f248c5013318\" (UID: \"2143de82-6f2b-403c-a702-f248c5013318\") "
Nov 01 06:04:03.954842 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:03.954818    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2143de82-6f2b-403c-a702-f248c5013318-openshift-service-ca" pod "2143de82-6f2b-403c-a702-f248c5013318" (UID: "2143de82-6f2b-403c-a702-f248c5013318"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 06:04:03.955071 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:03.955045    4864 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2143de82-6f2b-403c-a702-f248c5013318-openshift-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:03.957017 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:03.956981    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2143de82-6f2b-403c-a702-f248c5013318-maas-api-tls" pod "2143de82-6f2b-403c-a702-f248c5013318" (UID: "2143de82-6f2b-403c-a702-f248c5013318"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 06:04:03.957138 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:03.957079    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2143de82-6f2b-403c-a702-f248c5013318-metrics-certs" pod "2143de82-6f2b-403c-a702-f248c5013318" (UID: "2143de82-6f2b-403c-a702-f248c5013318"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 06:04:03.960197 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:03.960165    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2143de82-6f2b-403c-a702-f248c5013318-kube-api-access-vln7s" pod "2143de82-6f2b-403c-a702-f248c5013318" (UID: "2143de82-6f2b-403c-a702-f248c5013318"). InnerVolumeSpecName "kube-api-access-vln7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:04:04.056070 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:04.056040    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2143de82-6f2b-403c-a702-f248c5013318-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:04.056070 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:04.056068    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2143de82-6f2b-403c-a702-f248c5013318-maas-api-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:04.056270 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:04.056077    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vln7s\" (UniqueName: \"kubernetes.io/projected/2143de82-6f2b-403c-a702-f248c5013318-kube-api-access-vln7s\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:04.423112 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:04.423071    4864 scope.go:122] "RemoveContainer" containerID="f68628b31409b48f062134a5604b721cfca1b4c5b4770cca4aecc864dbad02d6"
Nov 01 06:04:04.432976 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:04.432954    4864 scope.go:122] "RemoveContainer" containerID="c588e1cee2c779b3c031622eedf53fd68dea3939d6d526d055ac3895a48c8711"
Nov 01 06:04:04.441695 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:04.441677    4864 scope.go:122] "RemoveContainer" containerID="b8b9666ddce88d036d5be635b865379341eba436f0356259664609abd9f4dd43"
Nov 01 06:04:04.737711 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:04.737612    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q" event={"ID":"2143de82-6f2b-403c-a702-f248c5013318","Type":"ContainerDied","Data":"ee8d08466ba846b7e42607593565f0561b8449069adab1140c79fb3d0bf3dd63"}
Nov 01 06:04:04.737711 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:04.737650    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"
Nov 01 06:04:04.964096 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:04.964058    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"]
Nov 01 06:04:04.969704 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:04.969676    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48099-68dfd84b8b-xpc9q"]
Nov 01 06:04:05.246290 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:05.246252    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2143de82-6f2b-403c-a702-f248c5013318" path="/var/lib/kubelet/pods/2143de82-6f2b-403c-a702-f248c5013318/volumes"
Nov 01 06:04:08.812481 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.812448    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"]
Nov 01 06:04:08.813281 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813259    4864 cpu_manager.go:414] "containerMap: removing container" podUID="48c3060a-f1c5-4ca4-8955-3b80c61eacbd" containerName="maas-api"
Nov 01 06:04:08.813353 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813285    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48c3060a-f1c5-4ca4-8955-3b80c61eacbd" containerName="maas-api"
Nov 01 06:04:08.813353 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813300    4864 cpu_manager.go:414] "containerMap: removing container" podUID="e3588066-acd0-433f-96c3-f4461690728f" containerName="maas-api"
Nov 01 06:04:08.813353 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813308    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3588066-acd0-433f-96c3-f4461690728f" containerName="maas-api"
Nov 01 06:04:08.813353 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813330    4864 cpu_manager.go:414] "containerMap: removing container" podUID="702e5c95-5de0-4329-bae0-404d4d65a941" containerName="istio-proxy"
Nov 01 06:04:08.813353 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813338    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="702e5c95-5de0-4329-bae0-404d4d65a941" containerName="istio-proxy"
Nov 01 06:04:08.813353 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813351    4864 cpu_manager.go:414] "containerMap: removing container" podUID="7b847150-a3e8-4210-af71-4f2ec0010c03" containerName="main"
Nov 01 06:04:08.813533 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813358    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7b847150-a3e8-4210-af71-4f2ec0010c03" containerName="main"
Nov 01 06:04:08.813533 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813371    4864 cpu_manager.go:414] "containerMap: removing container" podUID="61a331da-3dde-4e55-bd6a-047a02b6e94d" containerName="maas-api"
Nov 01 06:04:08.813533 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813378    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61a331da-3dde-4e55-bd6a-047a02b6e94d" containerName="maas-api"
Nov 01 06:04:08.813533 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813387    4864 cpu_manager.go:414] "containerMap: removing container" podUID="d9a45d6a-720d-45f0-973e-a69b24c10cc7" containerName="main"
Nov 01 06:04:08.813533 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813394    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9a45d6a-720d-45f0-973e-a69b24c10cc7" containerName="main"
Nov 01 06:04:08.813533 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813410    4864 cpu_manager.go:414] "containerMap: removing container" podUID="071c5c98-548c-439b-b8f6-3b07c70df932" containerName="main"
Nov 01 06:04:08.813533 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813416    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="071c5c98-548c-439b-b8f6-3b07c70df932" containerName="main"
Nov 01 06:04:08.813533 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813426    4864 cpu_manager.go:414] "containerMap: removing container" podUID="acafe0a1-dc08-48f9-a20a-9774b0146dab" containerName="main"
Nov 01 06:04:08.813533 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813431    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="acafe0a1-dc08-48f9-a20a-9774b0146dab" containerName="main"
Nov 01 06:04:08.813533 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813450    4864 cpu_manager.go:414] "containerMap: removing container" podUID="2143de82-6f2b-403c-a702-f248c5013318" containerName="maas-api"
Nov 01 06:04:08.813533 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813456    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2143de82-6f2b-403c-a702-f248c5013318" containerName="maas-api"
Nov 01 06:04:08.813533 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813467    4864 cpu_manager.go:414] "containerMap: removing container" podUID="702e5c95-5de0-4329-bae0-404d4d65a941" containerName="istio-proxy"
Nov 01 06:04:08.813533 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813474    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="702e5c95-5de0-4329-bae0-404d4d65a941" containerName="istio-proxy"
Nov 01 06:04:08.813533 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813505    4864 cpu_manager.go:414] "containerMap: removing container" podUID="cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22" containerName="maas-api"
Nov 01 06:04:08.813533 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813513    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22" containerName="maas-api"
Nov 01 06:04:08.814036 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813612    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="702e5c95-5de0-4329-bae0-404d4d65a941" containerName="istio-proxy"
Nov 01 06:04:08.814036 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813621    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="7b847150-a3e8-4210-af71-4f2ec0010c03" containerName="main"
Nov 01 06:04:08.814036 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813626    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="61a331da-3dde-4e55-bd6a-047a02b6e94d" containerName="maas-api"
Nov 01 06:04:08.814036 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813631    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9a45d6a-720d-45f0-973e-a69b24c10cc7" containerName="main"
Nov 01 06:04:08.814036 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813639    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="071c5c98-548c-439b-b8f6-3b07c70df932" containerName="main"
Nov 01 06:04:08.814036 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813644    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="acafe0a1-dc08-48f9-a20a-9774b0146dab" containerName="main"
Nov 01 06:04:08.814036 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813650    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="2143de82-6f2b-403c-a702-f248c5013318" containerName="maas-api"
Nov 01 06:04:08.814036 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813655    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="702e5c95-5de0-4329-bae0-404d4d65a941" containerName="istio-proxy"
Nov 01 06:04:08.814036 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813661    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf5e70e7-fe8f-46d9-b46a-cffe0c54ee22" containerName="maas-api"
Nov 01 06:04:08.814036 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813667    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="48c3060a-f1c5-4ca4-8955-3b80c61eacbd" containerName="maas-api"
Nov 01 06:04:08.814036 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.813673    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3588066-acd0-433f-96c3-f4461690728f" containerName="maas-api"
Nov 01 06:04:08.816913 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.816898    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:08.861668 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.861646    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"]
Nov 01 06:04:08.865344 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.865322    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:08.911206 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.911177    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"]
Nov 01 06:04:08.921587 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.921561    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9z2g\" (UniqueName: \"kubernetes.io/projected/ca90edd7-9e6f-497b-8c41-71d14efe2a09-kube-api-access-p9z2g\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:08.921719 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.921606    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-podinfo\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:08.921719 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.921698    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-token\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:08.921874 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.921833    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istiod-ca-cert\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:08.921926 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.921878    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-credential-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:08.921926 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.921909    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-token\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:08.922005 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.921932    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-envoy\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:08.922005 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.921964    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-data\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:08.922005 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.921988    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-podinfo\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:08.922123 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.922026    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-workload-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:08.922123 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.922054    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-workload-certs\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:08.922123 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.922083    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-workload-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:08.922123 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.922111    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0add254-e457-48d7-85f0-9c1cebb0d7de-istiod-ca-cert\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:08.922319 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.922223    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-credential-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:08.922465 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.922442    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-workload-certs\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:08.922532 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.922486    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdpd7\" (UniqueName: \"kubernetes.io/projected/a0add254-e457-48d7-85f0-9c1cebb0d7de-kube-api-access-qdpd7\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:08.922532 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.922513    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-data\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:08.922694 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.922547    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-envoy\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:08.948014 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:08.947988    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"]
Nov 01 06:04:09.023624 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.023586    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-workload-certs\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:09.023624 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.023628    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdpd7\" (UniqueName: \"kubernetes.io/projected/a0add254-e457-48d7-85f0-9c1cebb0d7de-kube-api-access-qdpd7\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:09.023931 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.023645    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-data\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:09.023931 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.023671    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-envoy\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:09.023931 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.023691    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9z2g\" (UniqueName: \"kubernetes.io/projected/ca90edd7-9e6f-497b-8c41-71d14efe2a09-kube-api-access-p9z2g\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:09.023931 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.023712    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-podinfo\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:09.023931 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.023856    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-token\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:09.024209 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.023950    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istiod-ca-cert\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:09.024209 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.023989    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-credential-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:09.024209 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.024018    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-token\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:09.024209 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.024036    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-envoy\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:09.024209 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.024058    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-data\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:09.024209 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.024081    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-podinfo\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:09.024209 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.024111    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-workload-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:09.024209 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.024129    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-workload-certs\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:09.024209 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.024147    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-workload-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:09.024209 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.024178    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0add254-e457-48d7-85f0-9c1cebb0d7de-istiod-ca-cert\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:09.024729 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.024233    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-credential-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:09.024729 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.024537    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-data\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:09.024729 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.024629    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-workload-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:09.024947 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.024898    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-credential-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:09.025030 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.025004    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istiod-ca-cert\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:09.025096 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.025019    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-data\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:09.025096 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.025061    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0add254-e457-48d7-85f0-9c1cebb0d7de-istiod-ca-cert\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:09.025096 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.025066    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-workload-certs\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:09.025250 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.025167    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-credential-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:09.025308 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.025287    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-workload-certs\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:09.025484 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.025457    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-workload-socket\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:09.026589 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.026556    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-envoy\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:09.027048 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.027028    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-envoy\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:09.027142 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.027050    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-podinfo\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:09.027496 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.027478    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-podinfo\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:09.085435 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.085373    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-token\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:09.085560 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.085480    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-token\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:09.085716 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.085694    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdpd7\" (UniqueName: \"kubernetes.io/projected/a0add254-e457-48d7-85f0-9c1cebb0d7de-kube-api-access-qdpd7\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-nkc44\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:09.085751 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.085711    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9z2g\" (UniqueName: \"kubernetes.io/projected/ca90edd7-9e6f-497b-8c41-71d14efe2a09-kube-api-access-p9z2g\") pod \"e2e-worker-w4-a48099-openshift-default-577864759-vf6t5\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") " pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:09.129994 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.129964    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:09.178234 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.178203    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:09.572189 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.572159    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"]
Nov 01 06:04:09.573631 ip-10-0-134-253 kubenswrapper[4864]: W1101 06:04:09.573603    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca90edd7_9e6f_497b_8c41_71d14efe2a09.slice/crio-3003ef524080fb40cde446ec7a4f83eccb3a485dc06b345f70b222092063cb7f WatchSource:0}: Error finding container 3003ef524080fb40cde446ec7a4f83eccb3a485dc06b345f70b222092063cb7f: Status 404 returned error can't find the container with id 3003ef524080fb40cde446ec7a4f83eccb3a485dc06b345f70b222092063cb7f
Nov 01 06:04:09.576048 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.576010    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 06:04:09.576153 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.576082    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 06:04:09.576153 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.576114    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 06:04:09.624376 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.624355    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"]
Nov 01 06:04:09.632847 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.632813    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 06:04:09.632965 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.632892    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 06:04:09.632965 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.632939    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 06:04:09.769176 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.769139    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5" event={"ID":"ca90edd7-9e6f-497b-8c41-71d14efe2a09","Type":"ContainerStarted","Data":"5ed4c038bcec2ff1e551554bc9b3f8e212e805ec0efb07ad147162e02101780a"}
Nov 01 06:04:09.769365 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.769196    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5" event={"ID":"ca90edd7-9e6f-497b-8c41-71d14efe2a09","Type":"ContainerStarted","Data":"3003ef524080fb40cde446ec7a4f83eccb3a485dc06b345f70b222092063cb7f"}
Nov 01 06:04:09.772309 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.771630    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44" event={"ID":"a0add254-e457-48d7-85f0-9c1cebb0d7de","Type":"ContainerStarted","Data":"c2481bcec3708e2d7667672f4f77a383ff725b13158ad24c8ae3ee5c48451e1b"}
Nov 01 06:04:09.772309 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.771669    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44" event={"ID":"a0add254-e457-48d7-85f0-9c1cebb0d7de","Type":"ContainerStarted","Data":"338addd15409df4ea6e9e41b53cef810d0486c370c7524c18509942c203f31b2"}
Nov 01 06:04:09.892864 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:09.892686    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5" podStartSLOduration=1.8926701970000002 podStartE2EDuration="1.892670197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 06:04:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 06:04:09.890046578 +0000 UTC m=+3666.780422591" watchObservedRunningTime="2026-11-01 06:04:09.892670197 +0000 UTC m=+3666.783046242"
Nov 01 06:04:10.010559 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:10.010475    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44" podStartSLOduration=2.010460554 podStartE2EDuration="2.010460554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 06:04:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 06:04:10.009115815 +0000 UTC m=+3666.899491803" watchObservedRunningTime="2026-11-01 06:04:10.010460554 +0000 UTC m=+3666.900836521"
Nov 01 06:04:10.131908 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:10.131866    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:10.137810 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:10.134077    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body=
Nov 01 06:04:10.137810 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:10.134158    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5" podUID="ca90edd7-9e6f-497b-8c41-71d14efe2a09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused"
Nov 01 06:04:10.182867 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:10.182154    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:10.189806 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:10.185912    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-nkc44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" start-of-body=
Nov 01 06:04:10.189806 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:10.185973    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44" podUID="a0add254-e457-48d7-85f0-9c1cebb0d7de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused"
Nov 01 06:04:10.544875 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:10.542680    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"]
Nov 01 06:04:10.591503 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:10.590256    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"]
Nov 01 06:04:10.591503 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:10.590511    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7" podUID="0e6d5eb0-20cd-4c97-a68c-558982570e66" containerName="istio-proxy" containerID="cri-o://48e767f70daf962f81c25823b2c577fc2828f3ef1b7acd0d30246ea157aab93f" gracePeriod=30
Nov 01 06:04:10.628163 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:10.628130    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"]
Nov 01 06:04:10.803869 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:10.794930    4864 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44" secret="" err="secret \"e2e-worker-w4-a48099-openshift-default-dockercfg-8mhxp\" not found"
Nov 01 06:04:10.803869 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:10.795757    4864 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5" secret="" err="secret \"e2e-worker-w4-a48099-openshift-default-dockercfg-8mhxp\" not found"
Nov 01 06:04:11.131342 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:11.131258    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body=
Nov 01 06:04:11.131342 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:11.131312    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5" podUID="ca90edd7-9e6f-497b-8c41-71d14efe2a09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused"
Nov 01 06:04:11.181909 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:11.179193    4864 patch_prober.go:29] interesting pod/e2e-worker-w4-a48099-openshift-default-577864759-nkc44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" start-of-body=
Nov 01 06:04:11.181909 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:11.179279    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44" podUID="a0add254-e457-48d7-85f0-9c1cebb0d7de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused"
Nov 01 06:04:11.808674 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:11.808626    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5" podUID="ca90edd7-9e6f-497b-8c41-71d14efe2a09" containerName="istio-proxy" containerID="cri-o://5ed4c038bcec2ff1e551554bc9b3f8e212e805ec0efb07ad147162e02101780a" gracePeriod=30
Nov 01 06:04:11.808921 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:11.808891    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44" podUID="a0add254-e457-48d7-85f0-9c1cebb0d7de" containerName="istio-proxy" containerID="cri-o://c2481bcec3708e2d7667672f4f77a383ff725b13158ad24c8ae3ee5c48451e1b" gracePeriod=30
Nov 01 06:04:15.857851 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.856826    4864 generic.go:356] "Generic (PLEG): container finished" podID="0e6d5eb0-20cd-4c97-a68c-558982570e66" containerID="48e767f70daf962f81c25823b2c577fc2828f3ef1b7acd0d30246ea157aab93f" exitCode=0
Nov 01 06:04:15.857851 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.856843    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7" event={"ID":"0e6d5eb0-20cd-4c97-a68c-558982570e66","Type":"ContainerDied","Data":"48e767f70daf962f81c25823b2c577fc2828f3ef1b7acd0d30246ea157aab93f"}
Nov 01 06:04:15.913369 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.913347    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:04:15.952638 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.952614    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-token\" (UniqueName: \"kubernetes.io/projected/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-token\") pod \"0e6d5eb0-20cd-4c97-a68c-558982570e66\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") "
Nov 01 06:04:15.952768 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.952657    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-podinfo\") pod \"0e6d5eb0-20cd-4c97-a68c-558982570e66\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") "
Nov 01 06:04:15.952768 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.952738    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e6d5eb0-20cd-4c97-a68c-558982570e66-kube-api-access-fgzwc\" (UniqueName: \"kubernetes.io/projected/0e6d5eb0-20cd-4c97-a68c-558982570e66-kube-api-access-fgzwc\") pod \"0e6d5eb0-20cd-4c97-a68c-558982570e66\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") "
Nov 01 06:04:15.952879 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.952774    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-workload-socket\") pod \"0e6d5eb0-20cd-4c97-a68c-558982570e66\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") "
Nov 01 06:04:15.952879 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.952849    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-envoy\") pod \"0e6d5eb0-20cd-4c97-a68c-558982570e66\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") "
Nov 01 06:04:15.952955 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.952875    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-credential-socket\") pod \"0e6d5eb0-20cd-4c97-a68c-558982570e66\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") "
Nov 01 06:04:15.952955 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.952907    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-workload-certs\") pod \"0e6d5eb0-20cd-4c97-a68c-558982570e66\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") "
Nov 01 06:04:15.953050 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.952973    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-data\") pod \"0e6d5eb0-20cd-4c97-a68c-558982570e66\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") "
Nov 01 06:04:15.953050 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.953006    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e6d5eb0-20cd-4c97-a68c-558982570e66-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e6d5eb0-20cd-4c97-a68c-558982570e66-istiod-ca-cert\") pod \"0e6d5eb0-20cd-4c97-a68c-558982570e66\" (UID: \"0e6d5eb0-20cd-4c97-a68c-558982570e66\") "
Nov 01 06:04:15.953447 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.953409    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-workload-socket" pod "0e6d5eb0-20cd-4c97-a68c-558982570e66" (UID: "0e6d5eb0-20cd-4c97-a68c-558982570e66"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:04:15.953567 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.953536    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-data" pod "0e6d5eb0-20cd-4c97-a68c-558982570e66" (UID: "0e6d5eb0-20cd-4c97-a68c-558982570e66"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:04:15.953653 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.953630    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:15.953717 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.953658    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:15.953717 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.953669    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-credential-socket" pod "0e6d5eb0-20cd-4c97-a68c-558982570e66" (UID: "0e6d5eb0-20cd-4c97-a68c-558982570e66"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:04:15.954118 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.954085    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-workload-certs" pod "0e6d5eb0-20cd-4c97-a68c-558982570e66" (UID: "0e6d5eb0-20cd-4c97-a68c-558982570e66"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:04:15.954207 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.954136    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e6d5eb0-20cd-4c97-a68c-558982570e66-istiod-ca-cert" pod "0e6d5eb0-20cd-4c97-a68c-558982570e66" (UID: "0e6d5eb0-20cd-4c97-a68c-558982570e66"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 06:04:15.955319 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.955283    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-podinfo" pod "0e6d5eb0-20cd-4c97-a68c-558982570e66" (UID: "0e6d5eb0-20cd-4c97-a68c-558982570e66"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 06:04:15.955939 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.955858    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-token" pod "0e6d5eb0-20cd-4c97-a68c-558982570e66" (UID: "0e6d5eb0-20cd-4c97-a68c-558982570e66"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:04:15.956361 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.956329    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-envoy" pod "0e6d5eb0-20cd-4c97-a68c-558982570e66" (UID: "0e6d5eb0-20cd-4c97-a68c-558982570e66"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:04:15.956565 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:15.956531    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e6d5eb0-20cd-4c97-a68c-558982570e66-kube-api-access-fgzwc" pod "0e6d5eb0-20cd-4c97-a68c-558982570e66" (UID: "0e6d5eb0-20cd-4c97-a68c-558982570e66"). InnerVolumeSpecName "kube-api-access-fgzwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:04:16.055035 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:16.054978    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgzwc\" (UniqueName: \"kubernetes.io/projected/0e6d5eb0-20cd-4c97-a68c-558982570e66-kube-api-access-fgzwc\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:16.055035 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:16.055010    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:16.055035 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:16.055020    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:16.055035 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:16.055028    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e6d5eb0-20cd-4c97-a68c-558982570e66-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:16.055035 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:16.055039    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e6d5eb0-20cd-4c97-a68c-558982570e66-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:16.055035 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:16.055048    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:16.055564 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:16.055056    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e6d5eb0-20cd-4c97-a68c-558982570e66-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:16.872811 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:16.869020    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"
Nov 01 06:04:16.872811 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:16.869056    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7" event={"ID":"0e6d5eb0-20cd-4c97-a68c-558982570e66","Type":"ContainerDied","Data":"fadf999e6725716f77ddf5e1b9d7085fb8973e330530ee14b1cb9f541430201a"}
Nov 01 06:04:16.872811 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:16.869135    4864 scope.go:122] "RemoveContainer" containerID="48e767f70daf962f81c25823b2c577fc2828f3ef1b7acd0d30246ea157aab93f"
Nov 01 06:04:17.091102 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:17.091058    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"]
Nov 01 06:04:17.133587 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:17.133509    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7"]
Nov 01 06:04:17.247750 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:17.247708    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e6d5eb0-20cd-4c97-a68c-558982570e66" path="/var/lib/kubelet/pods/0e6d5eb0-20cd-4c97-a68c-558982570e66/volumes"
Nov 01 06:04:22.380560 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.380534    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:22.407759 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.407704    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:22.523593 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:04:22.523560    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:04:22.526284 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526264    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-token\" (UniqueName: \"kubernetes.io/projected/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-token\") pod \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") "
Nov 01 06:04:22.526413 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526302    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-workload-certs\") pod \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") "
Nov 01 06:04:22.526413 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526334    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-workload-socket\") pod \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") "
Nov 01 06:04:22.526519 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526446    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-token\" (UniqueName: \"kubernetes.io/projected/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-token\") pod \"a0add254-e457-48d7-85f0-9c1cebb0d7de\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") "
Nov 01 06:04:22.526519 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526483    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-data\") pod \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") "
Nov 01 06:04:22.526638 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526522    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-credential-socket\") pod \"a0add254-e457-48d7-85f0-9c1cebb0d7de\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") "
Nov 01 06:04:22.526638 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526547    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-workload-certs" pod "ca90edd7-9e6f-497b-8c41-71d14efe2a09" (UID: "ca90edd7-9e6f-497b-8c41-71d14efe2a09"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:04:22.526638 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526594    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-credential-socket\") pod \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") "
Nov 01 06:04:22.526638 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526632    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istiod-ca-cert\") pod \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") "
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526637    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-workload-socket" pod "ca90edd7-9e6f-497b-8c41-71d14efe2a09" (UID: "ca90edd7-9e6f-497b-8c41-71d14efe2a09"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526678    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-data\") pod \"a0add254-e457-48d7-85f0-9c1cebb0d7de\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") "
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526705    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-workload-certs\") pod \"a0add254-e457-48d7-85f0-9c1cebb0d7de\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") "
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526737    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a0add254-e457-48d7-85f0-9c1cebb0d7de-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0add254-e457-48d7-85f0-9c1cebb0d7de-istiod-ca-cert\") pod \"a0add254-e457-48d7-85f0-9c1cebb0d7de\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") "
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526766    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-workload-socket\") pod \"a0add254-e457-48d7-85f0-9c1cebb0d7de\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") "
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526823    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-data" pod "ca90edd7-9e6f-497b-8c41-71d14efe2a09" (UID: "ca90edd7-9e6f-497b-8c41-71d14efe2a09"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526839    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-credential-socket" pod "ca90edd7-9e6f-497b-8c41-71d14efe2a09" (UID: "ca90edd7-9e6f-497b-8c41-71d14efe2a09"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526828    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-podinfo\") pod \"a0add254-e457-48d7-85f0-9c1cebb0d7de\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") "
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526850    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-credential-socket" pod "a0add254-e457-48d7-85f0-9c1cebb0d7de" (UID: "a0add254-e457-48d7-85f0-9c1cebb0d7de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526914    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0add254-e457-48d7-85f0-9c1cebb0d7de-kube-api-access-qdpd7\" (UniqueName: \"kubernetes.io/projected/a0add254-e457-48d7-85f0-9c1cebb0d7de-kube-api-access-qdpd7\") pod \"a0add254-e457-48d7-85f0-9c1cebb0d7de\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") "
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526920    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-data" pod "a0add254-e457-48d7-85f0-9c1cebb0d7de" (UID: "a0add254-e457-48d7-85f0-9c1cebb0d7de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526952    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-envoy\") pod \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") "
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.526988    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-podinfo\") pod \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") "
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.527044    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca90edd7-9e6f-497b-8c41-71d14efe2a09-kube-api-access-p9z2g\" (UniqueName: \"kubernetes.io/projected/ca90edd7-9e6f-497b-8c41-71d14efe2a09-kube-api-access-p9z2g\") pod \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\" (UID: \"ca90edd7-9e6f-497b-8c41-71d14efe2a09\") "
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.527064    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-workload-socket" pod "a0add254-e457-48d7-85f0-9c1cebb0d7de" (UID: "a0add254-e457-48d7-85f0-9c1cebb0d7de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.527082    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-envoy\") pod \"a0add254-e457-48d7-85f0-9c1cebb0d7de\" (UID: \"a0add254-e457-48d7-85f0-9c1cebb0d7de\") "
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.527090    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-workload-certs" pod "a0add254-e457-48d7-85f0-9c1cebb0d7de" (UID: "a0add254-e457-48d7-85f0-9c1cebb0d7de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.527314    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istiod-ca-cert" pod "ca90edd7-9e6f-497b-8c41-71d14efe2a09" (UID: "ca90edd7-9e6f-497b-8c41-71d14efe2a09"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.527384    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0add254-e457-48d7-85f0-9c1cebb0d7de-istiod-ca-cert" pod "a0add254-e457-48d7-85f0-9c1cebb0d7de" (UID: "a0add254-e457-48d7-85f0-9c1cebb0d7de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.527531    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.527553    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.527570    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.527584    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.527597    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.527611    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.527624    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.527640    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.527697    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0add254-e457-48d7-85f0-9c1cebb0d7de-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:22.527978 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.527718    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:22.529746 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.529328    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-token" pod "ca90edd7-9e6f-497b-8c41-71d14efe2a09" (UID: "ca90edd7-9e6f-497b-8c41-71d14efe2a09"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:04:22.529746 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.529514    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-token" pod "a0add254-e457-48d7-85f0-9c1cebb0d7de" (UID: "a0add254-e457-48d7-85f0-9c1cebb0d7de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:04:22.529746 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.529582    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0add254-e457-48d7-85f0-9c1cebb0d7de-kube-api-access-qdpd7" pod "a0add254-e457-48d7-85f0-9c1cebb0d7de" (UID: "a0add254-e457-48d7-85f0-9c1cebb0d7de"). InnerVolumeSpecName "kube-api-access-qdpd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:04:22.530694 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.530674    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-podinfo" pod "a0add254-e457-48d7-85f0-9c1cebb0d7de" (UID: "a0add254-e457-48d7-85f0-9c1cebb0d7de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 06:04:22.530777 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.530732    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-envoy" pod "ca90edd7-9e6f-497b-8c41-71d14efe2a09" (UID: "ca90edd7-9e6f-497b-8c41-71d14efe2a09"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:04:22.530777 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.530741    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca90edd7-9e6f-497b-8c41-71d14efe2a09-kube-api-access-p9z2g" pod "ca90edd7-9e6f-497b-8c41-71d14efe2a09" (UID: "ca90edd7-9e6f-497b-8c41-71d14efe2a09"). InnerVolumeSpecName "kube-api-access-p9z2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:04:22.531454 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.531438    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-envoy" pod "a0add254-e457-48d7-85f0-9c1cebb0d7de" (UID: "a0add254-e457-48d7-85f0-9c1cebb0d7de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:04:22.532328 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.532305    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-podinfo" pod "ca90edd7-9e6f-497b-8c41-71d14efe2a09" (UID: "ca90edd7-9e6f-497b-8c41-71d14efe2a09"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 06:04:22.628702 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.628670    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:22.628702 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.628701    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdpd7\" (UniqueName: \"kubernetes.io/projected/a0add254-e457-48d7-85f0-9c1cebb0d7de-kube-api-access-qdpd7\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:22.628926 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.628716    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:22.628926 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.628728    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:22.628926 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.628740    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9z2g\" (UniqueName: \"kubernetes.io/projected/ca90edd7-9e6f-497b-8c41-71d14efe2a09-kube-api-access-p9z2g\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:22.628926 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.628751    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:22.628926 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.628762    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca90edd7-9e6f-497b-8c41-71d14efe2a09-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:22.628926 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.628773    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0add254-e457-48d7-85f0-9c1cebb0d7de-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:04:22.916392 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.916357    4864 generic.go:356] "Generic (PLEG): container finished" podID="a0add254-e457-48d7-85f0-9c1cebb0d7de" containerID="c2481bcec3708e2d7667672f4f77a383ff725b13158ad24c8ae3ee5c48451e1b" exitCode=0
Nov 01 06:04:22.916573 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.916410    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"
Nov 01 06:04:22.916573 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.916445    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44" event={"ID":"a0add254-e457-48d7-85f0-9c1cebb0d7de","Type":"ContainerDied","Data":"c2481bcec3708e2d7667672f4f77a383ff725b13158ad24c8ae3ee5c48451e1b"}
Nov 01 06:04:22.916573 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.916501    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44" event={"ID":"a0add254-e457-48d7-85f0-9c1cebb0d7de","Type":"ContainerDied","Data":"338addd15409df4ea6e9e41b53cef810d0486c370c7524c18509942c203f31b2"}
Nov 01 06:04:22.916573 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.916526    4864 scope.go:122] "RemoveContainer" containerID="c2481bcec3708e2d7667672f4f77a383ff725b13158ad24c8ae3ee5c48451e1b"
Nov 01 06:04:22.918118 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.918096    4864 generic.go:356] "Generic (PLEG): container finished" podID="ca90edd7-9e6f-497b-8c41-71d14efe2a09" containerID="5ed4c038bcec2ff1e551554bc9b3f8e212e805ec0efb07ad147162e02101780a" exitCode=0
Nov 01 06:04:22.918240 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.918145    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5" event={"ID":"ca90edd7-9e6f-497b-8c41-71d14efe2a09","Type":"ContainerDied","Data":"5ed4c038bcec2ff1e551554bc9b3f8e212e805ec0efb07ad147162e02101780a"}
Nov 01 06:04:22.918240 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.918158    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"
Nov 01 06:04:22.918240 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.918177    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5" event={"ID":"ca90edd7-9e6f-497b-8c41-71d14efe2a09","Type":"ContainerDied","Data":"3003ef524080fb40cde446ec7a4f83eccb3a485dc06b345f70b222092063cb7f"}
Nov 01 06:04:22.927526 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.927507    4864 scope.go:122] "RemoveContainer" containerID="c2481bcec3708e2d7667672f4f77a383ff725b13158ad24c8ae3ee5c48451e1b"
Nov 01 06:04:22.927838 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:04:22.927814    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2481bcec3708e2d7667672f4f77a383ff725b13158ad24c8ae3ee5c48451e1b\": container with ID starting with c2481bcec3708e2d7667672f4f77a383ff725b13158ad24c8ae3ee5c48451e1b not found: ID does not exist" containerID="c2481bcec3708e2d7667672f4f77a383ff725b13158ad24c8ae3ee5c48451e1b"
Nov 01 06:04:22.927935 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.927843    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2481bcec3708e2d7667672f4f77a383ff725b13158ad24c8ae3ee5c48451e1b"} err="failed to get container status \"c2481bcec3708e2d7667672f4f77a383ff725b13158ad24c8ae3ee5c48451e1b\": rpc error: code = NotFound desc = could not find container \"c2481bcec3708e2d7667672f4f77a383ff725b13158ad24c8ae3ee5c48451e1b\": container with ID starting with c2481bcec3708e2d7667672f4f77a383ff725b13158ad24c8ae3ee5c48451e1b not found: ID does not exist"
Nov 01 06:04:22.927935 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.927862    4864 scope.go:122] "RemoveContainer" containerID="5ed4c038bcec2ff1e551554bc9b3f8e212e805ec0efb07ad147162e02101780a"
Nov 01 06:04:22.937173 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.937151    4864 scope.go:122] "RemoveContainer" containerID="5ed4c038bcec2ff1e551554bc9b3f8e212e805ec0efb07ad147162e02101780a"
Nov 01 06:04:22.937410 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:04:22.937391    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ed4c038bcec2ff1e551554bc9b3f8e212e805ec0efb07ad147162e02101780a\": container with ID starting with 5ed4c038bcec2ff1e551554bc9b3f8e212e805ec0efb07ad147162e02101780a not found: ID does not exist" containerID="5ed4c038bcec2ff1e551554bc9b3f8e212e805ec0efb07ad147162e02101780a"
Nov 01 06:04:22.937463 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:22.937418    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ed4c038bcec2ff1e551554bc9b3f8e212e805ec0efb07ad147162e02101780a"} err="failed to get container status \"5ed4c038bcec2ff1e551554bc9b3f8e212e805ec0efb07ad147162e02101780a\": rpc error: code = NotFound desc = could not find container \"5ed4c038bcec2ff1e551554bc9b3f8e212e805ec0efb07ad147162e02101780a\": container with ID starting with 5ed4c038bcec2ff1e551554bc9b3f8e212e805ec0efb07ad147162e02101780a not found: ID does not exist"
Nov 01 06:04:23.080590 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:23.080559    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"]
Nov 01 06:04:23.084058 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:23.084032    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44"]
Nov 01 06:04:23.192455 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:23.192389    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"]
Nov 01 06:04:23.196189 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:23.196169    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5"]
Nov 01 06:04:23.247238 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:23.247216    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a0add254-e457-48d7-85f0-9c1cebb0d7de" path="/var/lib/kubelet/pods/a0add254-e457-48d7-85f0-9c1cebb0d7de/volumes"
Nov 01 06:04:23.247622 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:23.247610    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca90edd7-9e6f-497b-8c41-71d14efe2a09" path="/var/lib/kubelet/pods/ca90edd7-9e6f-497b-8c41-71d14efe2a09/volumes"
Nov 01 06:04:44.694765 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:44.694732    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"]
Nov 01 06:04:52.527143 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:04:52.527105    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:04:59.715987 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:59.715937    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c" podUID="61ef19b8-157e-4f82-b624-49971d51b8c7" containerName="main" containerID="cri-o://eeba4677a8a601eecfed89397c4fa35153ed431d402a96869d0a756fb05e82bd" gracePeriod=45
Nov 01 06:04:59.721691 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:59.721657    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c" podUID="61ef19b8-157e-4f82-b624-49971d51b8c7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": dial tcp 10.132.0.82:8000: connect: connection refused"
Nov 01 06:04:59.976067 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:04:59.976007    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 06:05:00.116468 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.116432    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61ef19b8-157e-4f82-b624-49971d51b8c7-tls-certs\" (UniqueName: \"kubernetes.io/secret/61ef19b8-157e-4f82-b624-49971d51b8c7-tls-certs\") pod \"61ef19b8-157e-4f82-b624-49971d51b8c7\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") "
Nov 01 06:05:00.116655 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.116472    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-dshm\" (UniqueName: \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-dshm\") pod \"61ef19b8-157e-4f82-b624-49971d51b8c7\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") "
Nov 01 06:05:00.116655 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.116518    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61ef19b8-157e-4f82-b624-49971d51b8c7-kube-api-access-xmrw7\" (UniqueName: \"kubernetes.io/projected/61ef19b8-157e-4f82-b624-49971d51b8c7-kube-api-access-xmrw7\") pod \"61ef19b8-157e-4f82-b624-49971d51b8c7\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") "
Nov 01 06:05:00.116655 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.116604    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-home\" (UniqueName: \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-home\") pod \"61ef19b8-157e-4f82-b624-49971d51b8c7\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") "
Nov 01 06:05:00.116655 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.116641    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-tmp-dir\") pod \"61ef19b8-157e-4f82-b624-49971d51b8c7\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") "
Nov 01 06:05:00.116921 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.116676    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-model-cache\") pod \"61ef19b8-157e-4f82-b624-49971d51b8c7\" (UID: \"61ef19b8-157e-4f82-b624-49971d51b8c7\") "
Nov 01 06:05:00.116976 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.116917    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-tmp-dir" pod "61ef19b8-157e-4f82-b624-49971d51b8c7" (UID: "61ef19b8-157e-4f82-b624-49971d51b8c7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:05:00.116976 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.116933    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-home" pod "61ef19b8-157e-4f82-b624-49971d51b8c7" (UID: "61ef19b8-157e-4f82-b624-49971d51b8c7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:05:00.117061 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.116995    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-model-cache" pod "61ef19b8-157e-4f82-b624-49971d51b8c7" (UID: "61ef19b8-157e-4f82-b624-49971d51b8c7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:05:00.117288 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.117210    4864 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-home\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:00.117369 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.117350    4864 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-tmp-dir\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:00.117481 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.117371    4864 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-model-cache\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:00.119039 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.119014    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61ef19b8-157e-4f82-b624-49971d51b8c7-tls-certs" pod "61ef19b8-157e-4f82-b624-49971d51b8c7" (UID: "61ef19b8-157e-4f82-b624-49971d51b8c7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 06:05:00.119352 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.119331    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-dshm" pod "61ef19b8-157e-4f82-b624-49971d51b8c7" (UID: "61ef19b8-157e-4f82-b624-49971d51b8c7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:05:00.119352 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.119337    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61ef19b8-157e-4f82-b624-49971d51b8c7-kube-api-access-xmrw7" pod "61ef19b8-157e-4f82-b624-49971d51b8c7" (UID: "61ef19b8-157e-4f82-b624-49971d51b8c7"). InnerVolumeSpecName "kube-api-access-xmrw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:05:00.157865 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.157832    4864 generic.go:356] "Generic (PLEG): container finished" podID="61ef19b8-157e-4f82-b624-49971d51b8c7" containerID="eeba4677a8a601eecfed89397c4fa35153ed431d402a96869d0a756fb05e82bd" exitCode=0
Nov 01 06:05:00.158035 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.157880    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c" event={"ID":"61ef19b8-157e-4f82-b624-49971d51b8c7","Type":"ContainerDied","Data":"eeba4677a8a601eecfed89397c4fa35153ed431d402a96869d0a756fb05e82bd"}
Nov 01 06:05:00.158035 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.157887    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"
Nov 01 06:05:00.158035 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.157918    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c" event={"ID":"61ef19b8-157e-4f82-b624-49971d51b8c7","Type":"ContainerDied","Data":"551d1d82b57563fd9bc6e9b2263f0230bec2c025ab2f8c860d00b334da3ad0d3"}
Nov 01 06:05:00.158035 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.157936    4864 scope.go:122] "RemoveContainer" containerID="eeba4677a8a601eecfed89397c4fa35153ed431d402a96869d0a756fb05e82bd"
Nov 01 06:05:00.168428 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.168409    4864 scope.go:122] "RemoveContainer" containerID="eeba4677a8a601eecfed89397c4fa35153ed431d402a96869d0a756fb05e82bd"
Nov 01 06:05:00.168740 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:05:00.168713    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeba4677a8a601eecfed89397c4fa35153ed431d402a96869d0a756fb05e82bd\": container with ID starting with eeba4677a8a601eecfed89397c4fa35153ed431d402a96869d0a756fb05e82bd not found: ID does not exist" containerID="eeba4677a8a601eecfed89397c4fa35153ed431d402a96869d0a756fb05e82bd"
Nov 01 06:05:00.168876 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.168748    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeba4677a8a601eecfed89397c4fa35153ed431d402a96869d0a756fb05e82bd"} err="failed to get container status \"eeba4677a8a601eecfed89397c4fa35153ed431d402a96869d0a756fb05e82bd\": rpc error: code = NotFound desc = could not find container \"eeba4677a8a601eecfed89397c4fa35153ed431d402a96869d0a756fb05e82bd\": container with ID starting with eeba4677a8a601eecfed89397c4fa35153ed431d402a96869d0a756fb05e82bd not found: ID does not exist"
Nov 01 06:05:00.218335 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.218310    4864 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61ef19b8-157e-4f82-b624-49971d51b8c7-tls-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:00.218335 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.218332    4864 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61ef19b8-157e-4f82-b624-49971d51b8c7-dshm\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:00.218510 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.218343    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmrw7\" (UniqueName: \"kubernetes.io/projected/61ef19b8-157e-4f82-b624-49971d51b8c7-kube-api-access-xmrw7\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:00.323576 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.323545    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"]
Nov 01 06:05:00.327719 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:00.327688    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-7d1ec3/distinct-2-w0-7d1ec3-kserve-8484dcf8d9-d5c4c"]
Nov 01 06:05:01.247242 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:01.247207    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61ef19b8-157e-4f82-b624-49971d51b8c7" path="/var/lib/kubelet/pods/61ef19b8-157e-4f82-b624-49971d51b8c7/volumes"
Nov 01 06:05:01.953412 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:01.953366    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"]
Nov 01 06:05:01.953622 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:01.953600    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl" podUID="51619ded-3924-4581-a8ae-8d2cb8dd9609" containerName="istio-proxy" containerID="cri-o://9d12fac431256cd89730835ced6eb81440f525cac8db8ea004b9f70eaec3467c" gracePeriod=30
Nov 01 06:05:07.212462 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.212422    4864 generic.go:356] "Generic (PLEG): container finished" podID="51619ded-3924-4581-a8ae-8d2cb8dd9609" containerID="9d12fac431256cd89730835ced6eb81440f525cac8db8ea004b9f70eaec3467c" exitCode=0
Nov 01 06:05:07.212883 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.212542    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl" event={"ID":"51619ded-3924-4581-a8ae-8d2cb8dd9609","Type":"ContainerDied","Data":"9d12fac431256cd89730835ced6eb81440f525cac8db8ea004b9f70eaec3467c"}
Nov 01 06:05:07.212883 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.212632    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl" event={"ID":"51619ded-3924-4581-a8ae-8d2cb8dd9609","Type":"ContainerDied","Data":"d093ff081788c03db35f37729c9795225e6525b842e35958604a60f831b52add"}
Nov 01 06:05:07.212883 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.212655    4864 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d093ff081788c03db35f37729c9795225e6525b842e35958604a60f831b52add"
Nov 01 06:05:07.218143 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.218122    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 06:05:07.293087 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.293025    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51619ded-3924-4581-a8ae-8d2cb8dd9609-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51619ded-3924-4581-a8ae-8d2cb8dd9609-istiod-ca-cert\") pod \"51619ded-3924-4581-a8ae-8d2cb8dd9609\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") "
Nov 01 06:05:07.293087 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.293064    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-credential-socket\") pod \"51619ded-3924-4581-a8ae-8d2cb8dd9609\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") "
Nov 01 06:05:07.293087 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.293085    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-data\") pod \"51619ded-3924-4581-a8ae-8d2cb8dd9609\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") "
Nov 01 06:05:07.293332 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.293111    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-workload-socket\") pod \"51619ded-3924-4581-a8ae-8d2cb8dd9609\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") "
Nov 01 06:05:07.293332 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.293172    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-podinfo\") pod \"51619ded-3924-4581-a8ae-8d2cb8dd9609\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") "
Nov 01 06:05:07.293332 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.293226    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-workload-certs\") pod \"51619ded-3924-4581-a8ae-8d2cb8dd9609\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") "
Nov 01 06:05:07.293332 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.293252    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-token\" (UniqueName: \"kubernetes.io/projected/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-token\") pod \"51619ded-3924-4581-a8ae-8d2cb8dd9609\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") "
Nov 01 06:05:07.293332 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.293279    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51619ded-3924-4581-a8ae-8d2cb8dd9609-kube-api-access-h9x5t\" (UniqueName: \"kubernetes.io/projected/51619ded-3924-4581-a8ae-8d2cb8dd9609-kube-api-access-h9x5t\") pod \"51619ded-3924-4581-a8ae-8d2cb8dd9609\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") "
Nov 01 06:05:07.293605 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.293374    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-envoy\") pod \"51619ded-3924-4581-a8ae-8d2cb8dd9609\" (UID: \"51619ded-3924-4581-a8ae-8d2cb8dd9609\") "
Nov 01 06:05:07.293605 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.293394    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-data" pod "51619ded-3924-4581-a8ae-8d2cb8dd9609" (UID: "51619ded-3924-4581-a8ae-8d2cb8dd9609"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:05:07.293605 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.293425    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51619ded-3924-4581-a8ae-8d2cb8dd9609-istiod-ca-cert" pod "51619ded-3924-4581-a8ae-8d2cb8dd9609" (UID: "51619ded-3924-4581-a8ae-8d2cb8dd9609"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 06:05:07.293605 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.293526    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-workload-socket" pod "51619ded-3924-4581-a8ae-8d2cb8dd9609" (UID: "51619ded-3924-4581-a8ae-8d2cb8dd9609"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:05:07.293605 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.293544    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-workload-certs" pod "51619ded-3924-4581-a8ae-8d2cb8dd9609" (UID: "51619ded-3924-4581-a8ae-8d2cb8dd9609"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:05:07.293891 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.293724    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51619ded-3924-4581-a8ae-8d2cb8dd9609-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:07.293891 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.293731    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-credential-socket" pod "51619ded-3924-4581-a8ae-8d2cb8dd9609" (UID: "51619ded-3924-4581-a8ae-8d2cb8dd9609"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:05:07.293891 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.293744    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:07.293891 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.293759    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:07.293891 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.293772    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:07.295649 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.295623    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-token" pod "51619ded-3924-4581-a8ae-8d2cb8dd9609" (UID: "51619ded-3924-4581-a8ae-8d2cb8dd9609"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:05:07.295649 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.295633    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-podinfo" pod "51619ded-3924-4581-a8ae-8d2cb8dd9609" (UID: "51619ded-3924-4581-a8ae-8d2cb8dd9609"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 06:05:07.295870 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.295728    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-envoy" pod "51619ded-3924-4581-a8ae-8d2cb8dd9609" (UID: "51619ded-3924-4581-a8ae-8d2cb8dd9609"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:05:07.295870 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.295763    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51619ded-3924-4581-a8ae-8d2cb8dd9609-kube-api-access-h9x5t" pod "51619ded-3924-4581-a8ae-8d2cb8dd9609" (UID: "51619ded-3924-4581-a8ae-8d2cb8dd9609"). InnerVolumeSpecName "kube-api-access-h9x5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:05:07.395123 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.395095    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:07.395123 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.395119    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:07.395123 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.395129    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:07.395335 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.395137    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9x5t\" (UniqueName: \"kubernetes.io/projected/51619ded-3924-4581-a8ae-8d2cb8dd9609-kube-api-access-h9x5t\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:07.395335 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:07.395148    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51619ded-3924-4581-a8ae-8d2cb8dd9609-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:08.218469 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:08.218438    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"
Nov 01 06:05:08.432282 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:08.432242    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"]
Nov 01 06:05:08.481658 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:08.481579    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl"]
Nov 01 06:05:09.247051 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:09.247016    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51619ded-3924-4581-a8ae-8d2cb8dd9609" path="/var/lib/kubelet/pods/51619ded-3924-4581-a8ae-8d2cb8dd9609/volumes"
Nov 01 06:05:22.523829 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:05:22.523722    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:05:25.766516 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:25.766484    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"]
Nov 01 06:05:25.767036 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:25.766752    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn" podUID="d5d265aa-c0b7-4df3-a8ca-238c1e002561" containerName="istio-proxy" containerID="cri-o://8c0d51bb913a6154f95c9f2eb34efe901b99431326b1262163de6d6453e0c82e" gracePeriod=30
Nov 01 06:05:30.200074 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:30.200017    4864 patch_prober.go:29] interesting pod/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 06:05:30.200596 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:30.200095    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn" podUID="d5d265aa-c0b7-4df3-a8ca-238c1e002561" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 06:05:31.044857 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.044776    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 06:05:31.140885 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.140848    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-envoy\") pod \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") "
Nov 01 06:05:31.141085 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.140912    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5d265aa-c0b7-4df3-a8ca-238c1e002561-kube-api-access-tgnrz\" (UniqueName: \"kubernetes.io/projected/d5d265aa-c0b7-4df3-a8ca-238c1e002561-kube-api-access-tgnrz\") pod \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") "
Nov 01 06:05:31.141085 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.140947    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-token\" (UniqueName: \"kubernetes.io/projected/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-token\") pod \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") "
Nov 01 06:05:31.141085 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.140975    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-credential-socket\") pod \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") "
Nov 01 06:05:31.141085 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.141040    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-workload-certs\") pod \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") "
Nov 01 06:05:31.141319 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.141087    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istiod-ca-cert\") pod \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") "
Nov 01 06:05:31.141319 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.141125    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-podinfo\") pod \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") "
Nov 01 06:05:31.141319 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.141166    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-data\") pod \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") "
Nov 01 06:05:31.141319 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.141250    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-workload-socket\") pod \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\" (UID: \"d5d265aa-c0b7-4df3-a8ca-238c1e002561\") "
Nov 01 06:05:31.141523 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.141360    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-credential-socket" pod "d5d265aa-c0b7-4df3-a8ca-238c1e002561" (UID: "d5d265aa-c0b7-4df3-a8ca-238c1e002561"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:05:31.141523 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.141376    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-workload-certs" pod "d5d265aa-c0b7-4df3-a8ca-238c1e002561" (UID: "d5d265aa-c0b7-4df3-a8ca-238c1e002561"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:05:31.141821 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.141582    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-workload-socket" pod "d5d265aa-c0b7-4df3-a8ca-238c1e002561" (UID: "d5d265aa-c0b7-4df3-a8ca-238c1e002561"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:05:31.141821 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.141598    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-data" pod "d5d265aa-c0b7-4df3-a8ca-238c1e002561" (UID: "d5d265aa-c0b7-4df3-a8ca-238c1e002561"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:05:31.141821 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.141730    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:31.141821 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.141750    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:31.141821 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.141764    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:31.141821 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.141779    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:31.142170 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.141833    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istiod-ca-cert" pod "d5d265aa-c0b7-4df3-a8ca-238c1e002561" (UID: "d5d265aa-c0b7-4df3-a8ca-238c1e002561"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 06:05:31.143412 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.143381    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-envoy" pod "d5d265aa-c0b7-4df3-a8ca-238c1e002561" (UID: "d5d265aa-c0b7-4df3-a8ca-238c1e002561"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:05:31.143412 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.143392    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-token" pod "d5d265aa-c0b7-4df3-a8ca-238c1e002561" (UID: "d5d265aa-c0b7-4df3-a8ca-238c1e002561"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:05:31.143412 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.143406    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5d265aa-c0b7-4df3-a8ca-238c1e002561-kube-api-access-tgnrz" pod "d5d265aa-c0b7-4df3-a8ca-238c1e002561" (UID: "d5d265aa-c0b7-4df3-a8ca-238c1e002561"). InnerVolumeSpecName "kube-api-access-tgnrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:05:31.143674 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.143653    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-podinfo" pod "d5d265aa-c0b7-4df3-a8ca-238c1e002561" (UID: "d5d265aa-c0b7-4df3-a8ca-238c1e002561"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 06:05:31.242710 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.242663    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:31.242710 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.242706    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgnrz\" (UniqueName: \"kubernetes.io/projected/d5d265aa-c0b7-4df3-a8ca-238c1e002561-kube-api-access-tgnrz\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:31.243234 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.242723    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:31.243234 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.242737    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:31.243234 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.242750    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5d265aa-c0b7-4df3-a8ca-238c1e002561-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:05:31.384825 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.384758    4864 generic.go:356] "Generic (PLEG): container finished" podID="d5d265aa-c0b7-4df3-a8ca-238c1e002561" containerID="8c0d51bb913a6154f95c9f2eb34efe901b99431326b1262163de6d6453e0c82e" exitCode=0
Nov 01 06:05:31.385148 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.384851    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"
Nov 01 06:05:31.385380 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.384872    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn" event={"ID":"d5d265aa-c0b7-4df3-a8ca-238c1e002561","Type":"ContainerDied","Data":"8c0d51bb913a6154f95c9f2eb34efe901b99431326b1262163de6d6453e0c82e"}
Nov 01 06:05:31.385501 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.385425    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn" event={"ID":"d5d265aa-c0b7-4df3-a8ca-238c1e002561","Type":"ContainerDied","Data":"43afabb98eaa8a5dda8cf180fb706787e7d352145019669ec8bbc2a2e5ec192e"}
Nov 01 06:05:31.385501 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.385467    4864 scope.go:122] "RemoveContainer" containerID="8c0d51bb913a6154f95c9f2eb34efe901b99431326b1262163de6d6453e0c82e"
Nov 01 06:05:31.396987 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.396947    4864 scope.go:122] "RemoveContainer" containerID="8c0d51bb913a6154f95c9f2eb34efe901b99431326b1262163de6d6453e0c82e"
Nov 01 06:05:31.397346 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:05:31.397320    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c0d51bb913a6154f95c9f2eb34efe901b99431326b1262163de6d6453e0c82e\": container with ID starting with 8c0d51bb913a6154f95c9f2eb34efe901b99431326b1262163de6d6453e0c82e not found: ID does not exist" containerID="8c0d51bb913a6154f95c9f2eb34efe901b99431326b1262163de6d6453e0c82e"
Nov 01 06:05:31.397440 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.397362    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c0d51bb913a6154f95c9f2eb34efe901b99431326b1262163de6d6453e0c82e"} err="failed to get container status \"8c0d51bb913a6154f95c9f2eb34efe901b99431326b1262163de6d6453e0c82e\": rpc error: code = NotFound desc = could not find container \"8c0d51bb913a6154f95c9f2eb34efe901b99431326b1262163de6d6453e0c82e\": container with ID starting with 8c0d51bb913a6154f95c9f2eb34efe901b99431326b1262163de6d6453e0c82e not found: ID does not exist"
Nov 01 06:05:31.548808 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.548760    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"]
Nov 01 06:05:31.552410 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:31.552385    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn"]
Nov 01 06:05:33.248682 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:05:33.248642    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d5d265aa-c0b7-4df3-a8ca-238c1e002561" path="/var/lib/kubelet/pods/d5d265aa-c0b7-4df3-a8ca-238c1e002561/volumes"
Nov 01 06:05:52.528484 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:05:52.528439    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:06:04.525658 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:06:04.525627    4864 scope.go:122] "RemoveContainer" containerID="9d12fac431256cd89730835ced6eb81440f525cac8db8ea004b9f70eaec3467c"
Nov 01 06:06:22.524467 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:06:22.524426    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:06:52.525910 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:06:52.525828    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:07:22.523668 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:07:22.523631    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:07:52.523605 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:07:52.523567    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:08:04.162715 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:04.161098    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 06:08:04.171079 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:04.171052    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 06:08:04.174101 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:04.174081    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 06:08:04.181987 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:04.181969    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 06:08:04.889631 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:04.889600    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"]
Nov 01 06:08:04.889866 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:04.889828    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv" podUID="ed66905d-dc1c-4969-a140-f04142163049" containerName="istio-proxy" containerID="cri-o://b7783ae1be9cba60a539c87c5fc105eaedf2ab3c845699ff802cda0f695f038c" gracePeriod=30
Nov 01 06:08:05.150150 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:05.150072    4864 patch_prober.go:29] interesting pod/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 06:08:05.150150 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:05.150121    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv" podUID="ed66905d-dc1c-4969-a140-f04142163049" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 06:08:10.148461 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.148398    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 06:08:10.306541 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.306512    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-istio-envoy\") pod \"ed66905d-dc1c-4969-a140-f04142163049\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") "
Nov 01 06:08:10.306733 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.306565    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-credential-socket\") pod \"ed66905d-dc1c-4969-a140-f04142163049\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") "
Nov 01 06:08:10.306733 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.306613    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-workload-certs\") pod \"ed66905d-dc1c-4969-a140-f04142163049\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") "
Nov 01 06:08:10.306733 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.306673    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed66905d-dc1c-4969-a140-f04142163049-istio-token\" (UniqueName: \"kubernetes.io/projected/ed66905d-dc1c-4969-a140-f04142163049-istio-token\") pod \"ed66905d-dc1c-4969-a140-f04142163049\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") "
Nov 01 06:08:10.306733 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.306697    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ed66905d-dc1c-4969-a140-f04142163049-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed66905d-dc1c-4969-a140-f04142163049-istiod-ca-cert\") pod \"ed66905d-dc1c-4969-a140-f04142163049\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") "
Nov 01 06:08:10.306733 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.306724    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-workload-socket\") pod \"ed66905d-dc1c-4969-a140-f04142163049\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") "
Nov 01 06:08:10.307045 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.306797    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-istio-data\") pod \"ed66905d-dc1c-4969-a140-f04142163049\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") "
Nov 01 06:08:10.307045 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.306868    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed66905d-dc1c-4969-a140-f04142163049-kube-api-access-wckjq\" (UniqueName: \"kubernetes.io/projected/ed66905d-dc1c-4969-a140-f04142163049-kube-api-access-wckjq\") pod \"ed66905d-dc1c-4969-a140-f04142163049\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") "
Nov 01 06:08:10.307045 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.306913    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ed66905d-dc1c-4969-a140-f04142163049-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed66905d-dc1c-4969-a140-f04142163049-istio-podinfo\") pod \"ed66905d-dc1c-4969-a140-f04142163049\" (UID: \"ed66905d-dc1c-4969-a140-f04142163049\") "
Nov 01 06:08:10.307045 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.306971    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-credential-socket" pod "ed66905d-dc1c-4969-a140-f04142163049" (UID: "ed66905d-dc1c-4969-a140-f04142163049"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:08:10.307045 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.306977    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-workload-certs" pod "ed66905d-dc1c-4969-a140-f04142163049" (UID: "ed66905d-dc1c-4969-a140-f04142163049"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:08:10.307314 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.307171    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-istio-data" pod "ed66905d-dc1c-4969-a140-f04142163049" (UID: "ed66905d-dc1c-4969-a140-f04142163049"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:08:10.307314 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.307261    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed66905d-dc1c-4969-a140-f04142163049-istiod-ca-cert" pod "ed66905d-dc1c-4969-a140-f04142163049" (UID: "ed66905d-dc1c-4969-a140-f04142163049"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 06:08:10.307314 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.307272    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-workload-socket" pod "ed66905d-dc1c-4969-a140-f04142163049" (UID: "ed66905d-dc1c-4969-a140-f04142163049"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:08:10.307606 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.307485    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:10.307606 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.307506    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:10.307606 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.307522    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:10.309073 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.309046    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-istio-envoy" pod "ed66905d-dc1c-4969-a140-f04142163049" (UID: "ed66905d-dc1c-4969-a140-f04142163049"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:08:10.309303 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.309269    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed66905d-dc1c-4969-a140-f04142163049-istio-token" pod "ed66905d-dc1c-4969-a140-f04142163049" (UID: "ed66905d-dc1c-4969-a140-f04142163049"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:08:10.309371 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.309349    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed66905d-dc1c-4969-a140-f04142163049-kube-api-access-wckjq" pod "ed66905d-dc1c-4969-a140-f04142163049" (UID: "ed66905d-dc1c-4969-a140-f04142163049"). InnerVolumeSpecName "kube-api-access-wckjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:08:10.309408 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.309363    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ed66905d-dc1c-4969-a140-f04142163049-istio-podinfo" pod "ed66905d-dc1c-4969-a140-f04142163049" (UID: "ed66905d-dc1c-4969-a140-f04142163049"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 06:08:10.327498 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.327181    4864 generic.go:356] "Generic (PLEG): container finished" podID="ed66905d-dc1c-4969-a140-f04142163049" containerID="b7783ae1be9cba60a539c87c5fc105eaedf2ab3c845699ff802cda0f695f038c" exitCode=0
Nov 01 06:08:10.327498 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.327249    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"
Nov 01 06:08:10.327498 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.327299    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv" event={"ID":"ed66905d-dc1c-4969-a140-f04142163049","Type":"ContainerDied","Data":"b7783ae1be9cba60a539c87c5fc105eaedf2ab3c845699ff802cda0f695f038c"}
Nov 01 06:08:10.327498 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.327350    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv" event={"ID":"ed66905d-dc1c-4969-a140-f04142163049","Type":"ContainerDied","Data":"54060c533388c54c4a5bf2aded71151a21076dc85e53c3e098c9c65d066edccd"}
Nov 01 06:08:10.327498 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.327376    4864 scope.go:122] "RemoveContainer" containerID="b7783ae1be9cba60a539c87c5fc105eaedf2ab3c845699ff802cda0f695f038c"
Nov 01 06:08:10.338520 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.338502    4864 scope.go:122] "RemoveContainer" containerID="8b11561e4434013bc75ae1f368b0ef560e6db41c6fc7b44e9ca27821a1444ea0"
Nov 01 06:08:10.349855 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.349836    4864 scope.go:122] "RemoveContainer" containerID="b7783ae1be9cba60a539c87c5fc105eaedf2ab3c845699ff802cda0f695f038c"
Nov 01 06:08:10.350507 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:08:10.350482    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7783ae1be9cba60a539c87c5fc105eaedf2ab3c845699ff802cda0f695f038c\": container with ID starting with b7783ae1be9cba60a539c87c5fc105eaedf2ab3c845699ff802cda0f695f038c not found: ID does not exist" containerID="b7783ae1be9cba60a539c87c5fc105eaedf2ab3c845699ff802cda0f695f038c"
Nov 01 06:08:10.350593 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.350517    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7783ae1be9cba60a539c87c5fc105eaedf2ab3c845699ff802cda0f695f038c"} err="failed to get container status \"b7783ae1be9cba60a539c87c5fc105eaedf2ab3c845699ff802cda0f695f038c\": rpc error: code = NotFound desc = could not find container \"b7783ae1be9cba60a539c87c5fc105eaedf2ab3c845699ff802cda0f695f038c\": container with ID starting with b7783ae1be9cba60a539c87c5fc105eaedf2ab3c845699ff802cda0f695f038c not found: ID does not exist"
Nov 01 06:08:10.350593 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.350536    4864 scope.go:122] "RemoveContainer" containerID="8b11561e4434013bc75ae1f368b0ef560e6db41c6fc7b44e9ca27821a1444ea0"
Nov 01 06:08:10.351231 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:08:10.351202    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b11561e4434013bc75ae1f368b0ef560e6db41c6fc7b44e9ca27821a1444ea0\": container with ID starting with 8b11561e4434013bc75ae1f368b0ef560e6db41c6fc7b44e9ca27821a1444ea0 not found: ID does not exist" containerID="8b11561e4434013bc75ae1f368b0ef560e6db41c6fc7b44e9ca27821a1444ea0"
Nov 01 06:08:10.351334 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.351241    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b11561e4434013bc75ae1f368b0ef560e6db41c6fc7b44e9ca27821a1444ea0"} err="failed to get container status \"8b11561e4434013bc75ae1f368b0ef560e6db41c6fc7b44e9ca27821a1444ea0\": rpc error: code = NotFound desc = could not find container \"8b11561e4434013bc75ae1f368b0ef560e6db41c6fc7b44e9ca27821a1444ea0\": container with ID starting with 8b11561e4434013bc75ae1f368b0ef560e6db41c6fc7b44e9ca27821a1444ea0 not found: ID does not exist"
Nov 01 06:08:10.408345 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.408311    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed66905d-dc1c-4969-a140-f04142163049-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:10.408345 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.408340    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed66905d-dc1c-4969-a140-f04142163049-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:10.408345 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.408350    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:10.408676 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.408359    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wckjq\" (UniqueName: \"kubernetes.io/projected/ed66905d-dc1c-4969-a140-f04142163049-kube-api-access-wckjq\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:10.408676 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.408376    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed66905d-dc1c-4969-a140-f04142163049-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:10.408676 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.408389    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed66905d-dc1c-4969-a140-f04142163049-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:10.493546 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.493501    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"]
Nov 01 06:08:10.499797 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:10.499745    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv"]
Nov 01 06:08:11.248333 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:11.248302    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ed66905d-dc1c-4969-a140-f04142163049" path="/var/lib/kubelet/pods/ed66905d-dc1c-4969-a140-f04142163049/volumes"
Nov 01 06:08:22.524485 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:08:22.524401    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:08:25.328138 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:25.328104    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"]
Nov 01 06:08:25.328584 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:25.328309    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5" podUID="a179e91d-6c63-4820-8462-cf2ce37aacc9" containerName="istio-proxy" containerID="cri-o://8a9704ab34c1183a816d4d7150c97daade8ba89b52cc84c2f7a7b95afa16cfef" gracePeriod=30
Nov 01 06:08:30.444744 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.444709    4864 generic.go:356] "Generic (PLEG): container finished" podID="a179e91d-6c63-4820-8462-cf2ce37aacc9" containerID="8a9704ab34c1183a816d4d7150c97daade8ba89b52cc84c2f7a7b95afa16cfef" exitCode=0
Nov 01 06:08:30.445165 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.444775    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5" event={"ID":"a179e91d-6c63-4820-8462-cf2ce37aacc9","Type":"ContainerDied","Data":"8a9704ab34c1183a816d4d7150c97daade8ba89b52cc84c2f7a7b95afa16cfef"}
Nov 01 06:08:30.445165 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.444859    4864 scope.go:122] "RemoveContainer" containerID="cb7f5272f0d450a913cb8c94fa4d561a460577e1b592d59502eefd04a750bafc"
Nov 01 06:08:30.579871 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.579848    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 06:08:30.597870 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.597820    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-token\" (UniqueName: \"kubernetes.io/projected/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-token\") pod \"a179e91d-6c63-4820-8462-cf2ce37aacc9\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") "
Nov 01 06:08:30.597870 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.597859    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-podinfo\") pod \"a179e91d-6c63-4820-8462-cf2ce37aacc9\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") "
Nov 01 06:08:30.598059 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.597881    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a179e91d-6c63-4820-8462-cf2ce37aacc9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a179e91d-6c63-4820-8462-cf2ce37aacc9-istiod-ca-cert\") pod \"a179e91d-6c63-4820-8462-cf2ce37aacc9\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") "
Nov 01 06:08:30.598059 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.597909    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-credential-socket\") pod \"a179e91d-6c63-4820-8462-cf2ce37aacc9\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") "
Nov 01 06:08:30.598059 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.598018    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a179e91d-6c63-4820-8462-cf2ce37aacc9-kube-api-access-dfsv5\" (UniqueName: \"kubernetes.io/projected/a179e91d-6c63-4820-8462-cf2ce37aacc9-kube-api-access-dfsv5\") pod \"a179e91d-6c63-4820-8462-cf2ce37aacc9\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") "
Nov 01 06:08:30.598230 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.598073    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-data\") pod \"a179e91d-6c63-4820-8462-cf2ce37aacc9\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") "
Nov 01 06:08:30.598230 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.598110    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-workload-certs\") pod \"a179e91d-6c63-4820-8462-cf2ce37aacc9\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") "
Nov 01 06:08:30.598230 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.598153    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-workload-socket\") pod \"a179e91d-6c63-4820-8462-cf2ce37aacc9\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") "
Nov 01 06:08:30.598230 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.598183    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-credential-socket" pod "a179e91d-6c63-4820-8462-cf2ce37aacc9" (UID: "a179e91d-6c63-4820-8462-cf2ce37aacc9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:08:30.598230 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.598221    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-envoy\") pod \"a179e91d-6c63-4820-8462-cf2ce37aacc9\" (UID: \"a179e91d-6c63-4820-8462-cf2ce37aacc9\") "
Nov 01 06:08:30.598495 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.598279    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a179e91d-6c63-4820-8462-cf2ce37aacc9-istiod-ca-cert" pod "a179e91d-6c63-4820-8462-cf2ce37aacc9" (UID: "a179e91d-6c63-4820-8462-cf2ce37aacc9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 06:08:30.598495 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.598401    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-workload-certs" pod "a179e91d-6c63-4820-8462-cf2ce37aacc9" (UID: "a179e91d-6c63-4820-8462-cf2ce37aacc9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:08:30.598776 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.598495    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-workload-socket" pod "a179e91d-6c63-4820-8462-cf2ce37aacc9" (UID: "a179e91d-6c63-4820-8462-cf2ce37aacc9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:08:30.598776 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.598584    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:30.598776 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.598594    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-data" pod "a179e91d-6c63-4820-8462-cf2ce37aacc9" (UID: "a179e91d-6c63-4820-8462-cf2ce37aacc9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:08:30.598776 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.598605    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a179e91d-6c63-4820-8462-cf2ce37aacc9-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:30.598776 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.598619    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:30.598776 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.598632    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:30.600263 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.600236    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a179e91d-6c63-4820-8462-cf2ce37aacc9-kube-api-access-dfsv5" pod "a179e91d-6c63-4820-8462-cf2ce37aacc9" (UID: "a179e91d-6c63-4820-8462-cf2ce37aacc9"). InnerVolumeSpecName "kube-api-access-dfsv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:08:30.600263 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.600253    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-podinfo" pod "a179e91d-6c63-4820-8462-cf2ce37aacc9" (UID: "a179e91d-6c63-4820-8462-cf2ce37aacc9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 06:08:30.600422 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.600366    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-token" pod "a179e91d-6c63-4820-8462-cf2ce37aacc9" (UID: "a179e91d-6c63-4820-8462-cf2ce37aacc9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:08:30.600546 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.600523    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-envoy" pod "a179e91d-6c63-4820-8462-cf2ce37aacc9" (UID: "a179e91d-6c63-4820-8462-cf2ce37aacc9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:08:30.699821 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.699776    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:30.699821 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.699815    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:30.699821 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.699825    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:30.700037 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.699834    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfsv5\" (UniqueName: \"kubernetes.io/projected/a179e91d-6c63-4820-8462-cf2ce37aacc9-kube-api-access-dfsv5\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:30.700037 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:30.699844    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a179e91d-6c63-4820-8462-cf2ce37aacc9-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:31.452194 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:31.452162    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"
Nov 01 06:08:31.452652 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:31.452167    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5" event={"ID":"a179e91d-6c63-4820-8462-cf2ce37aacc9","Type":"ContainerDied","Data":"02368783dd5f0d7c967a914b07369578f0532c987421979a04f9079ff6d54f1f"}
Nov 01 06:08:31.452652 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:31.452289    4864 scope.go:122] "RemoveContainer" containerID="8a9704ab34c1183a816d4d7150c97daade8ba89b52cc84c2f7a7b95afa16cfef"
Nov 01 06:08:31.615829 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:31.615796    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"]
Nov 01 06:08:31.665482 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:31.665447    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5"]
Nov 01 06:08:33.247064 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:33.247035    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a179e91d-6c63-4820-8462-cf2ce37aacc9" path="/var/lib/kubelet/pods/a179e91d-6c63-4820-8462-cf2ce37aacc9/volumes"
Nov 01 06:08:37.037655 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.037622    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"]
Nov 01 06:08:37.038247 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.037847    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw" podUID="22f93dd6-caf5-4352-a81f-98878a3c485f" containerName="maas-api" containerID="cri-o://afe00036852b503cb69bae64c9c52c0006dddb37ee3877cdeb47063ef419d49b" gracePeriod=30
Nov 01 06:08:37.306500 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.306474    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"
Nov 01 06:08:37.369594 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.369563    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/22f93dd6-caf5-4352-a81f-98878a3c485f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22f93dd6-caf5-4352-a81f-98878a3c485f-maas-api-tls\") pod \"22f93dd6-caf5-4352-a81f-98878a3c485f\" (UID: \"22f93dd6-caf5-4352-a81f-98878a3c485f\") "
Nov 01 06:08:37.369744 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.369632    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/22f93dd6-caf5-4352-a81f-98878a3c485f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22f93dd6-caf5-4352-a81f-98878a3c485f-openshift-service-ca\") pod \"22f93dd6-caf5-4352-a81f-98878a3c485f\" (UID: \"22f93dd6-caf5-4352-a81f-98878a3c485f\") "
Nov 01 06:08:37.369744 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.369673    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/22f93dd6-caf5-4352-a81f-98878a3c485f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/22f93dd6-caf5-4352-a81f-98878a3c485f-metrics-certs\") pod \"22f93dd6-caf5-4352-a81f-98878a3c485f\" (UID: \"22f93dd6-caf5-4352-a81f-98878a3c485f\") "
Nov 01 06:08:37.369868 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.369811    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22f93dd6-caf5-4352-a81f-98878a3c485f-kube-api-access-f5qrj\" (UniqueName: \"kubernetes.io/projected/22f93dd6-caf5-4352-a81f-98878a3c485f-kube-api-access-f5qrj\") pod \"22f93dd6-caf5-4352-a81f-98878a3c485f\" (UID: \"22f93dd6-caf5-4352-a81f-98878a3c485f\") "
Nov 01 06:08:37.370046 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.370019    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22f93dd6-caf5-4352-a81f-98878a3c485f-openshift-service-ca" pod "22f93dd6-caf5-4352-a81f-98878a3c485f" (UID: "22f93dd6-caf5-4352-a81f-98878a3c485f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 06:08:37.370220 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.370203    4864 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22f93dd6-caf5-4352-a81f-98878a3c485f-openshift-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:37.372030 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.372001    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22f93dd6-caf5-4352-a81f-98878a3c485f-maas-api-tls" pod "22f93dd6-caf5-4352-a81f-98878a3c485f" (UID: "22f93dd6-caf5-4352-a81f-98878a3c485f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 06:08:37.372145 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.372065    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22f93dd6-caf5-4352-a81f-98878a3c485f-metrics-certs" pod "22f93dd6-caf5-4352-a81f-98878a3c485f" (UID: "22f93dd6-caf5-4352-a81f-98878a3c485f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 06:08:37.372145 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.372110    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22f93dd6-caf5-4352-a81f-98878a3c485f-kube-api-access-f5qrj" pod "22f93dd6-caf5-4352-a81f-98878a3c485f" (UID: "22f93dd6-caf5-4352-a81f-98878a3c485f"). InnerVolumeSpecName "kube-api-access-f5qrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:08:37.470945 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.470914    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5qrj\" (UniqueName: \"kubernetes.io/projected/22f93dd6-caf5-4352-a81f-98878a3c485f-kube-api-access-f5qrj\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:37.470945 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.470941    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22f93dd6-caf5-4352-a81f-98878a3c485f-maas-api-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:37.470945 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.470952    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22f93dd6-caf5-4352-a81f-98878a3c485f-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:08:37.492043 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.492012    4864 generic.go:356] "Generic (PLEG): container finished" podID="22f93dd6-caf5-4352-a81f-98878a3c485f" containerID="afe00036852b503cb69bae64c9c52c0006dddb37ee3877cdeb47063ef419d49b" exitCode=0
Nov 01 06:08:37.492190 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.492063    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"
Nov 01 06:08:37.492190 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.492103    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw" event={"ID":"22f93dd6-caf5-4352-a81f-98878a3c485f","Type":"ContainerDied","Data":"afe00036852b503cb69bae64c9c52c0006dddb37ee3877cdeb47063ef419d49b"}
Nov 01 06:08:37.492190 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.492165    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw" event={"ID":"22f93dd6-caf5-4352-a81f-98878a3c485f","Type":"ContainerDied","Data":"c620e5907fcb569506c0ef0ab515290f3d975ddc78b22c16f870902700f3e01e"}
Nov 01 06:08:37.492300 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.492196    4864 scope.go:122] "RemoveContainer" containerID="afe00036852b503cb69bae64c9c52c0006dddb37ee3877cdeb47063ef419d49b"
Nov 01 06:08:37.502711 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.502689    4864 scope.go:122] "RemoveContainer" containerID="afe00036852b503cb69bae64c9c52c0006dddb37ee3877cdeb47063ef419d49b"
Nov 01 06:08:37.503020 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:08:37.503001    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afe00036852b503cb69bae64c9c52c0006dddb37ee3877cdeb47063ef419d49b\": container with ID starting with afe00036852b503cb69bae64c9c52c0006dddb37ee3877cdeb47063ef419d49b not found: ID does not exist" containerID="afe00036852b503cb69bae64c9c52c0006dddb37ee3877cdeb47063ef419d49b"
Nov 01 06:08:37.503073 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.503028    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afe00036852b503cb69bae64c9c52c0006dddb37ee3877cdeb47063ef419d49b"} err="failed to get container status \"afe00036852b503cb69bae64c9c52c0006dddb37ee3877cdeb47063ef419d49b\": rpc error: code = NotFound desc = could not find container \"afe00036852b503cb69bae64c9c52c0006dddb37ee3877cdeb47063ef419d49b\": container with ID starting with afe00036852b503cb69bae64c9c52c0006dddb37ee3877cdeb47063ef419d49b not found: ID does not exist"
Nov 01 06:08:37.654872 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.654840    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"]
Nov 01 06:08:37.660581 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:37.660558    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-a12c35-5cdb8757d5-4jbpw"]
Nov 01 06:08:39.247449 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:08:39.247414    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="22f93dd6-caf5-4352-a81f-98878a3c485f" path="/var/lib/kubelet/pods/22f93dd6-caf5-4352-a81f-98878a3c485f/volumes"
Nov 01 06:08:52.525166 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:08:52.525129    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:09:08.956388 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:08.956358    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-k2nkj_cb788d11-cc64-4f1c-b1e0-0bed74754f03/manager/0.log"
Nov 01 06:09:09.200756 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:09.200724    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-9t2hp_194e1c98-db83-4d11-b279-b43783313267/manager/0.log"
Nov 01 06:09:09.436638 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:09.436609    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-crc2n_60ec3670-705a-45e2-9dbe-e71edd5fb2fc/kuadrant-console-plugin/0.log"
Nov 01 06:09:10.702801 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:10.702748    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-kfkl9_a763a5c5-de38-4a17-9cff-69cf629c1270/manager/0.log"
Nov 01 06:09:11.186136 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:11.184105    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-wttv8_af89e1f3-fb0d-44e2-b35f-748885de9474/manager/0.log"
Nov 01 06:09:12.728159 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:12.728130    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-8qncv_39569001-5722-4b9e-ac79-84952518f6ca/manager/0.log"
Nov 01 06:09:12.976589 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:12.976557    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-8krz2_f4aae4c0-bc4d-4889-92e6-b5e4d22d7a06/manager/0.log"
Nov 01 06:09:13.489569 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:13.489536    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-74b5bc6fff-vqr9s_90275ed4-c333-4c1f-8583-853bf05c6165/manager/1.log"
Nov 01 06:09:13.723001 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:13.722971    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-h2pkm_3340fa76-a621-451b-8c96-82b31e6b40d1/server/0.log"
Nov 01 06:09:14.378220 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:14.378186    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-cdt6d_c3884e26-630f-4983-abc1-6dc5e9cb922a/proxy/0.log"
Nov 01 06:09:15.014175 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:15.014150    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-74b5bc6fff-vqr9s_90275ed4-c333-4c1f-8583-853bf05c6165/manager/0.log"
Nov 01 06:09:19.061414 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:19.061382    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"]
Nov 01 06:09:19.061907 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:19.061576    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7" podUID="dc37f8bd-cf86-47e5-823d-9e9d357ddf13" containerName="istio-proxy" containerID="cri-o://0336bcdf655f593c34d79fee5b5ef12ff3b45aaeef308567f464c552866ae28b" gracePeriod=30
Nov 01 06:09:22.528953 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:09:22.528910    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:09:24.321085 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.321062    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 06:09:24.378253 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.378198    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-kube-api-access-bthms\" (UniqueName: \"kubernetes.io/projected/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-kube-api-access-bthms\") pod \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") "
Nov 01 06:09:24.378360 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.378253    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-credential-socket\") pod \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") "
Nov 01 06:09:24.378360 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.378301    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-data\") pod \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") "
Nov 01 06:09:24.378360 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.378334    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-podinfo\") pod \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") "
Nov 01 06:09:24.378502 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.378380    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istiod-ca-cert\") pod \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") "
Nov 01 06:09:24.378502 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.378416    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-envoy\") pod \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") "
Nov 01 06:09:24.378502 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.378469    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-workload-certs\") pod \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") "
Nov 01 06:09:24.378661 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.378509    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-workload-socket\") pod \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") "
Nov 01 06:09:24.378661 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.378545    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-token\" (UniqueName: \"kubernetes.io/projected/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-token\") pod \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\" (UID: \"dc37f8bd-cf86-47e5-823d-9e9d357ddf13\") "
Nov 01 06:09:24.378661 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.378593    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-credential-socket" pod "dc37f8bd-cf86-47e5-823d-9e9d357ddf13" (UID: "dc37f8bd-cf86-47e5-823d-9e9d357ddf13"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:09:24.378661 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.378616    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-data" pod "dc37f8bd-cf86-47e5-823d-9e9d357ddf13" (UID: "dc37f8bd-cf86-47e5-823d-9e9d357ddf13"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:09:24.378927 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.378863    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-workload-socket" pod "dc37f8bd-cf86-47e5-823d-9e9d357ddf13" (UID: "dc37f8bd-cf86-47e5-823d-9e9d357ddf13"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:09:24.378927 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.378877    4864 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-credential-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:09:24.378927 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.378895    4864 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-data\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:09:24.379127 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.379099    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-workload-certs" pod "dc37f8bd-cf86-47e5-823d-9e9d357ddf13" (UID: "dc37f8bd-cf86-47e5-823d-9e9d357ddf13"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:09:24.379207 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.379142    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istiod-ca-cert" pod "dc37f8bd-cf86-47e5-823d-9e9d357ddf13" (UID: "dc37f8bd-cf86-47e5-823d-9e9d357ddf13"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 06:09:24.380608 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.380581    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-kube-api-access-bthms" pod "dc37f8bd-cf86-47e5-823d-9e9d357ddf13" (UID: "dc37f8bd-cf86-47e5-823d-9e9d357ddf13"). InnerVolumeSpecName "kube-api-access-bthms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:09:24.381068 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.381047    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-token" pod "dc37f8bd-cf86-47e5-823d-9e9d357ddf13" (UID: "dc37f8bd-cf86-47e5-823d-9e9d357ddf13"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:09:24.381280 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.381259    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-podinfo" pod "dc37f8bd-cf86-47e5-823d-9e9d357ddf13" (UID: "dc37f8bd-cf86-47e5-823d-9e9d357ddf13"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 06:09:24.381331 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.381257    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-envoy" pod "dc37f8bd-cf86-47e5-823d-9e9d357ddf13" (UID: "dc37f8bd-cf86-47e5-823d-9e9d357ddf13"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:09:24.479313 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.479290    4864 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-workload-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:09:24.479313 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.479314    4864 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-workload-socket\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:09:24.479489 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.479323    4864 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-token\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:09:24.479489 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.479332    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bthms\" (UniqueName: \"kubernetes.io/projected/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-kube-api-access-bthms\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:09:24.479489 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.479341    4864 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-podinfo\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:09:24.479489 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.479349    4864 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istiod-ca-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:09:24.479489 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.479358    4864 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc37f8bd-cf86-47e5-823d-9e9d357ddf13-istio-envoy\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:09:24.770774 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.770690    4864 generic.go:356] "Generic (PLEG): container finished" podID="dc37f8bd-cf86-47e5-823d-9e9d357ddf13" containerID="0336bcdf655f593c34d79fee5b5ef12ff3b45aaeef308567f464c552866ae28b" exitCode=0
Nov 01 06:09:24.770774 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.770750    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"
Nov 01 06:09:24.770996 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.770779    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7" event={"ID":"dc37f8bd-cf86-47e5-823d-9e9d357ddf13","Type":"ContainerDied","Data":"0336bcdf655f593c34d79fee5b5ef12ff3b45aaeef308567f464c552866ae28b"}
Nov 01 06:09:24.770996 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.770849    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7" event={"ID":"dc37f8bd-cf86-47e5-823d-9e9d357ddf13","Type":"ContainerDied","Data":"89adc441efa829bf9e4744bb8e95ce1e55be6a6d34776ba058438e6ef418759b"}
Nov 01 06:09:24.770996 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.770872    4864 scope.go:122] "RemoveContainer" containerID="0336bcdf655f593c34d79fee5b5ef12ff3b45aaeef308567f464c552866ae28b"
Nov 01 06:09:24.783053 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.783015    4864 scope.go:122] "RemoveContainer" containerID="0336bcdf655f593c34d79fee5b5ef12ff3b45aaeef308567f464c552866ae28b"
Nov 01 06:09:24.783410 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:09:24.783386    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0336bcdf655f593c34d79fee5b5ef12ff3b45aaeef308567f464c552866ae28b\": container with ID starting with 0336bcdf655f593c34d79fee5b5ef12ff3b45aaeef308567f464c552866ae28b not found: ID does not exist" containerID="0336bcdf655f593c34d79fee5b5ef12ff3b45aaeef308567f464c552866ae28b"
Nov 01 06:09:24.783504 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.783418    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0336bcdf655f593c34d79fee5b5ef12ff3b45aaeef308567f464c552866ae28b"} err="failed to get container status \"0336bcdf655f593c34d79fee5b5ef12ff3b45aaeef308567f464c552866ae28b\": rpc error: code = NotFound desc = could not find container \"0336bcdf655f593c34d79fee5b5ef12ff3b45aaeef308567f464c552866ae28b\": container with ID starting with 0336bcdf655f593c34d79fee5b5ef12ff3b45aaeef308567f464c552866ae28b not found: ID does not exist"
Nov 01 06:09:24.934912 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.934877    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"]
Nov 01 06:09:24.985419 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:24.985387    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7"]
Nov 01 06:09:25.246631 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:09:25.246599    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc37f8bd-cf86-47e5-823d-9e9d357ddf13" path="/var/lib/kubelet/pods/dc37f8bd-cf86-47e5-823d-9e9d357ddf13/volumes"
Nov 01 06:09:52.523624 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:09:52.523539    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:10:22.524531 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:10:22.524493    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:10:24.992206 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:24.992179    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74b5bc6fff-vqr9s"]
Nov 01 06:10:24.992728 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:24.992395    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-74b5bc6fff-vqr9s" podUID="90275ed4-c333-4c1f-8583-853bf05c6165" containerName="manager" containerID="cri-o://cf2c69cc0bd7c3fe9591c90168ad0e21375c81582428a5f33056eb4efe58f217" gracePeriod=30
Nov 01 06:10:25.245794 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:25.245722    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74b5bc6fff-vqr9s"
Nov 01 06:10:25.429350 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:25.429321    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/90275ed4-c333-4c1f-8583-853bf05c6165-metrics-certs\" (UniqueName: \"kubernetes.io/secret/90275ed4-c333-4c1f-8583-853bf05c6165-metrics-certs\") pod \"90275ed4-c333-4c1f-8583-853bf05c6165\" (UID: \"90275ed4-c333-4c1f-8583-853bf05c6165\") "
Nov 01 06:10:25.429560 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:25.429384    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/90275ed4-c333-4c1f-8583-853bf05c6165-webhook-cert\" (UniqueName: \"kubernetes.io/secret/90275ed4-c333-4c1f-8583-853bf05c6165-webhook-cert\") pod \"90275ed4-c333-4c1f-8583-853bf05c6165\" (UID: \"90275ed4-c333-4c1f-8583-853bf05c6165\") "
Nov 01 06:10:25.429560 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:25.429463    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90275ed4-c333-4c1f-8583-853bf05c6165-kube-api-access-lr9jz\" (UniqueName: \"kubernetes.io/projected/90275ed4-c333-4c1f-8583-853bf05c6165-kube-api-access-lr9jz\") pod \"90275ed4-c333-4c1f-8583-853bf05c6165\" (UID: \"90275ed4-c333-4c1f-8583-853bf05c6165\") "
Nov 01 06:10:25.431484 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:25.431458    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90275ed4-c333-4c1f-8583-853bf05c6165-metrics-certs" pod "90275ed4-c333-4c1f-8583-853bf05c6165" (UID: "90275ed4-c333-4c1f-8583-853bf05c6165"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 06:10:25.431618 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:25.431595    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90275ed4-c333-4c1f-8583-853bf05c6165-webhook-cert" pod "90275ed4-c333-4c1f-8583-853bf05c6165" (UID: "90275ed4-c333-4c1f-8583-853bf05c6165"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 06:10:25.431730 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:25.431712    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90275ed4-c333-4c1f-8583-853bf05c6165-kube-api-access-lr9jz" pod "90275ed4-c333-4c1f-8583-853bf05c6165" (UID: "90275ed4-c333-4c1f-8583-853bf05c6165"). InnerVolumeSpecName "kube-api-access-lr9jz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:10:25.531098 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:25.531037    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90275ed4-c333-4c1f-8583-853bf05c6165-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:10:25.531098 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:25.531063    4864 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/90275ed4-c333-4c1f-8583-853bf05c6165-webhook-cert\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:10:25.531098 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:25.531078    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lr9jz\" (UniqueName: \"kubernetes.io/projected/90275ed4-c333-4c1f-8583-853bf05c6165-kube-api-access-lr9jz\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:10:26.107861 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:26.107817    4864 generic.go:356] "Generic (PLEG): container finished" podID="90275ed4-c333-4c1f-8583-853bf05c6165" containerID="cf2c69cc0bd7c3fe9591c90168ad0e21375c81582428a5f33056eb4efe58f217" exitCode=0
Nov 01 06:10:26.108376 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:26.107874    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74b5bc6fff-vqr9s"
Nov 01 06:10:26.108376 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:26.107884    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74b5bc6fff-vqr9s" event={"ID":"90275ed4-c333-4c1f-8583-853bf05c6165","Type":"ContainerDied","Data":"cf2c69cc0bd7c3fe9591c90168ad0e21375c81582428a5f33056eb4efe58f217"}
Nov 01 06:10:26.108376 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:26.107956    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74b5bc6fff-vqr9s" event={"ID":"90275ed4-c333-4c1f-8583-853bf05c6165","Type":"ContainerDied","Data":"60b482619fc3fa7ef12f9bf9a767864e860906f949c4d65d841e7e61bb765570"}
Nov 01 06:10:26.108376 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:26.107989    4864 scope.go:122] "RemoveContainer" containerID="cf2c69cc0bd7c3fe9591c90168ad0e21375c81582428a5f33056eb4efe58f217"
Nov 01 06:10:26.119902 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:26.119878    4864 scope.go:122] "RemoveContainer" containerID="672b80968be43108634f54480741ede4c8d22996ddfe8ab778294a22f1928ce1"
Nov 01 06:10:26.128998 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:26.128977    4864 scope.go:122] "RemoveContainer" containerID="cf2c69cc0bd7c3fe9591c90168ad0e21375c81582428a5f33056eb4efe58f217"
Nov 01 06:10:26.129278 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:10:26.129257    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf2c69cc0bd7c3fe9591c90168ad0e21375c81582428a5f33056eb4efe58f217\": container with ID starting with cf2c69cc0bd7c3fe9591c90168ad0e21375c81582428a5f33056eb4efe58f217 not found: ID does not exist" containerID="cf2c69cc0bd7c3fe9591c90168ad0e21375c81582428a5f33056eb4efe58f217"
Nov 01 06:10:26.129337 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:26.129287    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf2c69cc0bd7c3fe9591c90168ad0e21375c81582428a5f33056eb4efe58f217"} err="failed to get container status \"cf2c69cc0bd7c3fe9591c90168ad0e21375c81582428a5f33056eb4efe58f217\": rpc error: code = NotFound desc = could not find container \"cf2c69cc0bd7c3fe9591c90168ad0e21375c81582428a5f33056eb4efe58f217\": container with ID starting with cf2c69cc0bd7c3fe9591c90168ad0e21375c81582428a5f33056eb4efe58f217 not found: ID does not exist"
Nov 01 06:10:26.129337 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:26.129305    4864 scope.go:122] "RemoveContainer" containerID="672b80968be43108634f54480741ede4c8d22996ddfe8ab778294a22f1928ce1"
Nov 01 06:10:26.129501 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:10:26.129487    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"672b80968be43108634f54480741ede4c8d22996ddfe8ab778294a22f1928ce1\": container with ID starting with 672b80968be43108634f54480741ede4c8d22996ddfe8ab778294a22f1928ce1 not found: ID does not exist" containerID="672b80968be43108634f54480741ede4c8d22996ddfe8ab778294a22f1928ce1"
Nov 01 06:10:26.129544 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:26.129506    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"672b80968be43108634f54480741ede4c8d22996ddfe8ab778294a22f1928ce1"} err="failed to get container status \"672b80968be43108634f54480741ede4c8d22996ddfe8ab778294a22f1928ce1\": rpc error: code = NotFound desc = could not find container \"672b80968be43108634f54480741ede4c8d22996ddfe8ab778294a22f1928ce1\": container with ID starting with 672b80968be43108634f54480741ede4c8d22996ddfe8ab778294a22f1928ce1 not found: ID does not exist"
Nov 01 06:10:26.290601 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:26.290568    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74b5bc6fff-vqr9s"]
Nov 01 06:10:26.300183 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:26.296028    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-74b5bc6fff-vqr9s"]
Nov 01 06:10:27.247068 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:10:27.247038    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="90275ed4-c333-4c1f-8583-853bf05c6165" path="/var/lib/kubelet/pods/90275ed4-c333-4c1f-8583-853bf05c6165/volumes"
Nov 01 06:10:52.523175 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:10:52.523137    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:11:22.523693 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:11:22.523606    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:11:50.263634 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.263598    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"]
Nov 01 06:11:50.264276 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264177    4864 cpu_manager.go:414] "containerMap: removing container" podUID="ed66905d-dc1c-4969-a140-f04142163049" containerName="istio-proxy"
Nov 01 06:11:50.264276 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264197    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed66905d-dc1c-4969-a140-f04142163049" containerName="istio-proxy"
Nov 01 06:11:50.264276 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264220    4864 cpu_manager.go:414] "containerMap: removing container" podUID="ed66905d-dc1c-4969-a140-f04142163049" containerName="istio-proxy"
Nov 01 06:11:50.264276 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264228    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed66905d-dc1c-4969-a140-f04142163049" containerName="istio-proxy"
Nov 01 06:11:50.264276 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264240    4864 cpu_manager.go:414] "containerMap: removing container" podUID="a179e91d-6c63-4820-8462-cf2ce37aacc9" containerName="istio-proxy"
Nov 01 06:11:50.264276 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264249    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a179e91d-6c63-4820-8462-cf2ce37aacc9" containerName="istio-proxy"
Nov 01 06:11:50.264276 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264259    4864 cpu_manager.go:414] "containerMap: removing container" podUID="a179e91d-6c63-4820-8462-cf2ce37aacc9" containerName="istio-proxy"
Nov 01 06:11:50.264276 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264265    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a179e91d-6c63-4820-8462-cf2ce37aacc9" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264283    4864 cpu_manager.go:414] "containerMap: removing container" podUID="0e6d5eb0-20cd-4c97-a68c-558982570e66" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264291    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e6d5eb0-20cd-4c97-a68c-558982570e66" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264306    4864 cpu_manager.go:414] "containerMap: removing container" podUID="51619ded-3924-4581-a8ae-8d2cb8dd9609" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264313    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51619ded-3924-4581-a8ae-8d2cb8dd9609" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264331    4864 cpu_manager.go:414] "containerMap: removing container" podUID="22f93dd6-caf5-4352-a81f-98878a3c485f" containerName="maas-api"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264339    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22f93dd6-caf5-4352-a81f-98878a3c485f" containerName="maas-api"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264349    4864 cpu_manager.go:414] "containerMap: removing container" podUID="61ef19b8-157e-4f82-b624-49971d51b8c7" containerName="main"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264356    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61ef19b8-157e-4f82-b624-49971d51b8c7" containerName="main"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264379    4864 cpu_manager.go:414] "containerMap: removing container" podUID="d5d265aa-c0b7-4df3-a8ca-238c1e002561" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264387    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5d265aa-c0b7-4df3-a8ca-238c1e002561" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264398    4864 cpu_manager.go:414] "containerMap: removing container" podUID="ca90edd7-9e6f-497b-8c41-71d14efe2a09" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264404    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca90edd7-9e6f-497b-8c41-71d14efe2a09" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264417    4864 cpu_manager.go:414] "containerMap: removing container" podUID="90275ed4-c333-4c1f-8583-853bf05c6165" containerName="manager"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264425    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90275ed4-c333-4c1f-8583-853bf05c6165" containerName="manager"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264436    4864 cpu_manager.go:414] "containerMap: removing container" podUID="90275ed4-c333-4c1f-8583-853bf05c6165" containerName="manager"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264442    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90275ed4-c333-4c1f-8583-853bf05c6165" containerName="manager"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264452    4864 cpu_manager.go:414] "containerMap: removing container" podUID="dc37f8bd-cf86-47e5-823d-9e9d357ddf13" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264459    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc37f8bd-cf86-47e5-823d-9e9d357ddf13" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264466    4864 cpu_manager.go:414] "containerMap: removing container" podUID="a0add254-e457-48d7-85f0-9c1cebb0d7de" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264473    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0add254-e457-48d7-85f0-9c1cebb0d7de" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264565    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e6d5eb0-20cd-4c97-a68c-558982570e66" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264577    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="51619ded-3924-4581-a8ae-8d2cb8dd9609" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264588    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="22f93dd6-caf5-4352-a81f-98878a3c485f" containerName="maas-api"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264595    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="61ef19b8-157e-4f82-b624-49971d51b8c7" containerName="main"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264607    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5d265aa-c0b7-4df3-a8ca-238c1e002561" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264615    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca90edd7-9e6f-497b-8c41-71d14efe2a09" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264624    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="90275ed4-c333-4c1f-8583-853bf05c6165" containerName="manager"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264632    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="90275ed4-c333-4c1f-8583-853bf05c6165" containerName="manager"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264639    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc37f8bd-cf86-47e5-823d-9e9d357ddf13" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264647    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0add254-e457-48d7-85f0-9c1cebb0d7de" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264656    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed66905d-dc1c-4969-a140-f04142163049" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264666    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed66905d-dc1c-4969-a140-f04142163049" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264674    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="a179e91d-6c63-4820-8462-cf2ce37aacc9" containerName="istio-proxy"
Nov 01 06:11:50.264705 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.264683    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="a179e91d-6c63-4820-8462-cf2ce37aacc9" containerName="istio-proxy"
Nov 01 06:11:50.268221 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.268203    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.371800 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.371748    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de1abde-348b-480d-8172-dc9a677eb592-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.371967 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.371849    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5de1abde-348b-480d-8172-dc9a677eb592-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.371967 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.371888    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5de1abde-348b-480d-8172-dc9a677eb592-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.372105 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.371965    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vtj6\" (UniqueName: \"kubernetes.io/projected/5de1abde-348b-480d-8172-dc9a677eb592-kube-api-access-8vtj6\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.372105 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.372011    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5de1abde-348b-480d-8172-dc9a677eb592-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.372105 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.372035    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5de1abde-348b-480d-8172-dc9a677eb592-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.372105 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.372058    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de1abde-348b-480d-8172-dc9a677eb592-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.372105 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.372079    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5de1abde-348b-480d-8172-dc9a677eb592-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.372389 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.372123    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5de1abde-348b-480d-8172-dc9a677eb592-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.372389 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.372238    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"]
Nov 01 06:11:50.473284 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.473260    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5de1abde-348b-480d-8172-dc9a677eb592-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.473425 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.473311    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5de1abde-348b-480d-8172-dc9a677eb592-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.473425 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.473377    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vtj6\" (UniqueName: \"kubernetes.io/projected/5de1abde-348b-480d-8172-dc9a677eb592-kube-api-access-8vtj6\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.473425 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.473418    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5de1abde-348b-480d-8172-dc9a677eb592-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.473576 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.473436    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5de1abde-348b-480d-8172-dc9a677eb592-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.473576 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.473452    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de1abde-348b-480d-8172-dc9a677eb592-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.473576 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.473468    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5de1abde-348b-480d-8172-dc9a677eb592-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.473731 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.473595    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5de1abde-348b-480d-8172-dc9a677eb592-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.473731 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.473642    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de1abde-348b-480d-8172-dc9a677eb592-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.473873 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.473840    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5de1abde-348b-480d-8172-dc9a677eb592-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.473933 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.473881    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de1abde-348b-480d-8172-dc9a677eb592-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.473989 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.473934    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5de1abde-348b-480d-8172-dc9a677eb592-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.474035 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.474011    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de1abde-348b-480d-8172-dc9a677eb592-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.474073 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.474029    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5de1abde-348b-480d-8172-dc9a677eb592-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.476019 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.475996    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5de1abde-348b-480d-8172-dc9a677eb592-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.476259 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.476237    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5de1abde-348b-480d-8172-dc9a677eb592-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.533885 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.533830    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5de1abde-348b-480d-8172-dc9a677eb592-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.533976 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.533908    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vtj6\" (UniqueName: \"kubernetes.io/projected/5de1abde-348b-480d-8172-dc9a677eb592-kube-api-access-8vtj6\") pod \"maas-default-gateway-openshift-default-58d898bd55-xnxx8\" (UID: \"5de1abde-348b-480d-8172-dc9a677eb592\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:50.580953 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:50.580934    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:51.004888 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:51.004865    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"]
Nov 01 06:11:51.010185 ip-10-0-134-253 kubenswrapper[4864]: W1101 06:11:51.010141    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5de1abde_348b_480d_8172_dc9a677eb592.slice/crio-e0f25fac67129741395c4a78d382fb00b1180be0415a650cf719d47d20aa0879 WatchSource:0}: Error finding container e0f25fac67129741395c4a78d382fb00b1180be0415a650cf719d47d20aa0879: Status 404 returned error can't find the container with id e0f25fac67129741395c4a78d382fb00b1180be0415a650cf719d47d20aa0879
Nov 01 06:11:51.011846 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:51.011769    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 06:11:51.011956 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:51.011880    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 06:11:51.011956 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:51.011939    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 06:11:51.570509 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:51.570468    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8" event={"ID":"5de1abde-348b-480d-8172-dc9a677eb592","Type":"ContainerStarted","Data":"dbfd608c1bd659a5168c273bd58dd75fd264859058bdee8e3cb7157a99f26972"}
Nov 01 06:11:51.571153 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:51.571128    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8" event={"ID":"5de1abde-348b-480d-8172-dc9a677eb592","Type":"ContainerStarted","Data":"e0f25fac67129741395c4a78d382fb00b1180be0415a650cf719d47d20aa0879"}
Nov 01 06:11:51.582946 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:51.582903    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:11:51.692161 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:51.692097    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8" podStartSLOduration=1.6920793170000001 podStartE2EDuration="1.692079317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 06:11:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 06:11:51.690090328 +0000 UTC m=+4128.580466304" watchObservedRunningTime="2026-11-01 06:11:51.692079317 +0000 UTC m=+4128.582455294"
Nov 01 06:11:52.523261 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:11:52.523217    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:11:52.583415 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:52.583382    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-xnxx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Nov 01 06:11:52.583988 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:52.583442    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8" podUID="5de1abde-348b-480d-8172-dc9a677eb592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Nov 01 06:11:53.582190 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:53.582158    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-xnxx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Nov 01 06:11:53.582406 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:53.582215    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8" podUID="5de1abde-348b-480d-8172-dc9a677eb592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Nov 01 06:11:54.582236 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:54.582192    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-xnxx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Nov 01 06:11:54.582748 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:54.582257    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8" podUID="5de1abde-348b-480d-8172-dc9a677eb592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Nov 01 06:11:55.582255 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:55.582214    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-xnxx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Nov 01 06:11:55.582807 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:55.582272    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8" podUID="5de1abde-348b-480d-8172-dc9a677eb592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Nov 01 06:11:56.582296 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:56.582260    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-xnxx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Nov 01 06:11:56.582735 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:56.582329    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8" podUID="5de1abde-348b-480d-8172-dc9a677eb592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Nov 01 06:11:57.582175 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:57.582135    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-xnxx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Nov 01 06:11:57.582368 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:57.582217    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8" podUID="5de1abde-348b-480d-8172-dc9a677eb592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Nov 01 06:11:59.581987 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:59.581942    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-xnxx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 06:11:59.582414 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:11:59.581997    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8" podUID="5de1abde-348b-480d-8172-dc9a677eb592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 06:12:00.622982 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:00.622935    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-xnxx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 06:12:00.623518 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:00.623017    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8" podUID="5de1abde-348b-480d-8172-dc9a677eb592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 06:12:01.314146 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:01.314111    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:12:01.314526 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:01.314506    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:12:01.315221 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:01.315201    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8"
Nov 01 06:12:05.366369 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.366339    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"]
Nov 01 06:12:05.370166 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.370137    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.476141 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.476101    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"]
Nov 01 06:12:05.525628 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.525600    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575b4414-cbde-452b-bbe4-46be0265d77f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.525773 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.525634    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575b4414-cbde-452b-bbe4-46be0265d77f-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.525773 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.525659    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5557h\" (UniqueName: \"kubernetes.io/projected/575b4414-cbde-452b-bbe4-46be0265d77f-kube-api-access-5557h\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.525936 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.525774    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575b4414-cbde-452b-bbe4-46be0265d77f-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.525936 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.525816    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575b4414-cbde-452b-bbe4-46be0265d77f-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.526045 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.525925    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575b4414-cbde-452b-bbe4-46be0265d77f-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.526045 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.525965    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575b4414-cbde-452b-bbe4-46be0265d77f-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.526153 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.526045    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575b4414-cbde-452b-bbe4-46be0265d77f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.526153 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.526135    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575b4414-cbde-452b-bbe4-46be0265d77f-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.627085 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.627017    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575b4414-cbde-452b-bbe4-46be0265d77f-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.627085 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.627054    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575b4414-cbde-452b-bbe4-46be0265d77f-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.627282 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.627084    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575b4414-cbde-452b-bbe4-46be0265d77f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.627282 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.627143    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575b4414-cbde-452b-bbe4-46be0265d77f-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.627282 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.627184    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575b4414-cbde-452b-bbe4-46be0265d77f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.627282 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.627214    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575b4414-cbde-452b-bbe4-46be0265d77f-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.627282 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.627253    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5557h\" (UniqueName: \"kubernetes.io/projected/575b4414-cbde-452b-bbe4-46be0265d77f-kube-api-access-5557h\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.627549 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.627386    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575b4414-cbde-452b-bbe4-46be0265d77f-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.627549 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.627426    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575b4414-cbde-452b-bbe4-46be0265d77f-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.627549 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.627521    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575b4414-cbde-452b-bbe4-46be0265d77f-credential-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.627708 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.627552    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575b4414-cbde-452b-bbe4-46be0265d77f-istio-data\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.627856 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.627837    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575b4414-cbde-452b-bbe4-46be0265d77f-workload-certs\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.627971 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.627871    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575b4414-cbde-452b-bbe4-46be0265d77f-workload-socket\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.627971 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.627938    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575b4414-cbde-452b-bbe4-46be0265d77f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.629657 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.629634    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575b4414-cbde-452b-bbe4-46be0265d77f-istio-envoy\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.629877 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.629857    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575b4414-cbde-452b-bbe4-46be0265d77f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.687577 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.687541    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575b4414-cbde-452b-bbe4-46be0265d77f-istio-token\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.688022 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.688001    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5557h\" (UniqueName: \"kubernetes.io/projected/575b4414-cbde-452b-bbe4-46be0265d77f-kube-api-access-5557h\") pod \"maas-default-gateway-openshift-default-58d898bd55-p22bd\" (UID: \"575b4414-cbde-452b-bbe4-46be0265d77f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:05.983257 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:05.983183    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:06.410930 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:06.410906    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"]
Nov 01 06:12:06.412621 ip-10-0-134-253 kubenswrapper[4864]: W1101 06:12:06.412591    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod575b4414_cbde_452b_bbe4_46be0265d77f.slice/crio-42c9c98e19c27ead3ca39130da037863ed548ab746c1c72ec9260912a27ca06e WatchSource:0}: Error finding container 42c9c98e19c27ead3ca39130da037863ed548ab746c1c72ec9260912a27ca06e: Status 404 returned error can't find the container with id 42c9c98e19c27ead3ca39130da037863ed548ab746c1c72ec9260912a27ca06e
Nov 01 06:12:06.414649 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:06.414619    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 06:12:06.414728 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:06.414682    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 06:12:06.414728 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:06.414713    4864 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Nov 01 06:12:06.663066 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:06.662973    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd" event={"ID":"575b4414-cbde-452b-bbe4-46be0265d77f","Type":"ContainerStarted","Data":"68a1f1a24f3f65841d2c68c9789b07bbd0918f479957899eed0851850eb91017"}
Nov 01 06:12:06.663066 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:06.663035    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd" event={"ID":"575b4414-cbde-452b-bbe4-46be0265d77f","Type":"ContainerStarted","Data":"42c9c98e19c27ead3ca39130da037863ed548ab746c1c72ec9260912a27ca06e"}
Nov 01 06:12:06.782924 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:06.782868    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd" podStartSLOduration=1.782849216 podStartE2EDuration="1.782849216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 06:12:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 06:12:06.781523044 +0000 UTC m=+4143.671899027" watchObservedRunningTime="2026-11-01 06:12:06.782849216 +0000 UTC m=+4143.673225192"
Nov 01 06:12:06.987870 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:06.985881    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:07.986810 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:07.985942    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-p22bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Nov 01 06:12:07.986810 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:07.986015    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd" podUID="575b4414-cbde-452b-bbe4-46be0265d77f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Nov 01 06:12:08.988181 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:08.988140    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-p22bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Nov 01 06:12:08.988588 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:08.988202    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd" podUID="575b4414-cbde-452b-bbe4-46be0265d77f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Nov 01 06:12:09.984557 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:09.984517    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-p22bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Nov 01 06:12:09.984729 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:09.984584    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd" podUID="575b4414-cbde-452b-bbe4-46be0265d77f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Nov 01 06:12:10.983930 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:10.983893    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-p22bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Nov 01 06:12:10.984325 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:10.983962    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd" podUID="575b4414-cbde-452b-bbe4-46be0265d77f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Nov 01 06:12:11.983957 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:11.983907    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-p22bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Nov 01 06:12:11.984370 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:11.983978    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd" podUID="575b4414-cbde-452b-bbe4-46be0265d77f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Nov 01 06:12:12.983940 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:12.983901    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-p22bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Nov 01 06:12:12.984133 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:12.983967    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd" podUID="575b4414-cbde-452b-bbe4-46be0265d77f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Nov 01 06:12:13.984809 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:13.984145    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-p22bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Nov 01 06:12:13.984809 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:13.984207    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd" podUID="575b4414-cbde-452b-bbe4-46be0265d77f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Nov 01 06:12:15.988610 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:15.988569    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-p22bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 06:12:15.989068 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:15.988623    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd" podUID="575b4414-cbde-452b-bbe4-46be0265d77f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 06:12:17.028938 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:17.028897    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-p22bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 06:12:17.029415 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:17.028950    4864 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd" podUID="575b4414-cbde-452b-bbe4-46be0265d77f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 06:12:17.413955 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:17.413926    4864 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:17.414357 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:17.414333    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:17.415155 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:12:17.415138    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd"
Nov 01 06:12:22.523475 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:12:22.523439    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:12:52.524008 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:12:52.523921    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:13:04.242689 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:13:04.242660    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 06:13:04.243328 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:13:04.243289    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 06:13:04.252628 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:13:04.252603    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 06:13:04.253120 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:13:04.253100    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 06:13:22.524084 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:13:22.524047    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:13:52.525677 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:13:52.525633    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:14:22.524708 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:14:22.524621    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:14:47.360938 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:14:47.360893    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-xnxx8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 06:14:47.361696 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:14:47.361661    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8" podUID="5de1abde-348b-480d-8172-dc9a677eb592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 06:14:52.525037 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:14:52.525000    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:15:22.530687 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:15:22.530647    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:15:52.536536 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:15:52.536344    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:16:02.360971 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:16:02.360930    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-xnxx8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 06:16:02.361678 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:16:02.361642    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8" podUID="5de1abde-348b-480d-8172-dc9a677eb592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 06:16:17.356988 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:16:17.356948    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-xnxx8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 06:16:17.359511 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:16:17.357499    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8" podUID="5de1abde-348b-480d-8172-dc9a677eb592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 06:16:22.577116 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:16:22.577049    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:16:52.523778 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:16:52.523741    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:17:22.523009 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:17:22.522922    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:17:52.523757 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:17:52.523720    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:18:04.305672 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:18:04.305640    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 06:18:04.313992 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:18:04.313969    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 06:18:04.318321 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:18:04.318290    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 06:18:04.325754 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:18:04.325738    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 06:18:22.523631 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:18:22.523589    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:18:52.524234 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:18:52.524143    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:19:22.524658 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:19:22.524621    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:19:52.523004 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:19:52.522966    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:20:22.523806 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:20:22.523750    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:20:52.523456 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:20:52.523415    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:21:22.523583 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:21:22.523543    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:21:52.523908 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:21:52.523824    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:22:22.523761 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:22:22.523719    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:22:52.523889 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:22:52.523848    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:23:04.343866 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:23:04.343836    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 06:23:04.352199 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:23:04.352171    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 06:23:04.356949 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:23:04.356923    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 06:23:04.364732 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:23:04.364709    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 06:23:22.524464 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:23:22.524425    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:23:52.524236 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:23:52.524196    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:24:22.523389 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:24:22.523350    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:24:52.523970 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:24:52.523886    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:25:22.522651 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:25:22.522610    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:25:52.523473 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:25:52.523422    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:26:22.524526 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:26:22.524470    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:26:52.524083 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:26:52.524044    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:27:20.204490 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:27:20.204448    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9"]
Nov 01 06:27:20.205407 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:27:20.204664    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9" podUID="a763a5c5-de38-4a17-9cff-69cf629c1270" containerName="manager" containerID="cri-o://6295c8daf8f032ea42f58a62d9a342e7bd1297bf9d819a8499691ed189234e4f" gracePeriod=10
Nov 01 06:27:21.322000 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:27:21.321968    4864 generic.go:356] "Generic (PLEG): container finished" podID="a763a5c5-de38-4a17-9cff-69cf629c1270" containerID="6295c8daf8f032ea42f58a62d9a342e7bd1297bf9d819a8499691ed189234e4f" exitCode=0
Nov 01 06:27:21.322366 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:27:21.322037    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9" event={"ID":"a763a5c5-de38-4a17-9cff-69cf629c1270","Type":"ContainerDied","Data":"6295c8daf8f032ea42f58a62d9a342e7bd1297bf9d819a8499691ed189234e4f"}
Nov 01 06:27:21.467164 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:27:21.467139    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9"
Nov 01 06:27:21.576637 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:27:21.576601    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a763a5c5-de38-4a17-9cff-69cf629c1270-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a763a5c5-de38-4a17-9cff-69cf629c1270-extensions-socket-volume\") pod \"a763a5c5-de38-4a17-9cff-69cf629c1270\" (UID: \"a763a5c5-de38-4a17-9cff-69cf629c1270\") "
Nov 01 06:27:21.576858 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:27:21.576661    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a763a5c5-de38-4a17-9cff-69cf629c1270-kube-api-access-nm55f\" (UniqueName: \"kubernetes.io/projected/a763a5c5-de38-4a17-9cff-69cf629c1270-kube-api-access-nm55f\") pod \"a763a5c5-de38-4a17-9cff-69cf629c1270\" (UID: \"a763a5c5-de38-4a17-9cff-69cf629c1270\") "
Nov 01 06:27:21.577052 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:27:21.577030    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a763a5c5-de38-4a17-9cff-69cf629c1270-extensions-socket-volume" pod "a763a5c5-de38-4a17-9cff-69cf629c1270" (UID: "a763a5c5-de38-4a17-9cff-69cf629c1270"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 06:27:21.578948 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:27:21.578924    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a763a5c5-de38-4a17-9cff-69cf629c1270-kube-api-access-nm55f" pod "a763a5c5-de38-4a17-9cff-69cf629c1270" (UID: "a763a5c5-de38-4a17-9cff-69cf629c1270"). InnerVolumeSpecName "kube-api-access-nm55f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:27:21.677425 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:27:21.677382    4864 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a763a5c5-de38-4a17-9cff-69cf629c1270-extensions-socket-volume\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:27:21.677425 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:27:21.677416    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nm55f\" (UniqueName: \"kubernetes.io/projected/a763a5c5-de38-4a17-9cff-69cf629c1270-kube-api-access-nm55f\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:27:22.336911 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:27:22.336876    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9"
Nov 01 06:27:22.337408 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:27:22.336881    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9" event={"ID":"a763a5c5-de38-4a17-9cff-69cf629c1270","Type":"ContainerDied","Data":"8b4c11a896cea8547049c97b21eb67d465b88205a9722c1db86b049afc8fd7f0"}
Nov 01 06:27:22.337408 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:27:22.337051    4864 scope.go:122] "RemoveContainer" containerID="6295c8daf8f032ea42f58a62d9a342e7bd1297bf9d819a8499691ed189234e4f"
Nov 01 06:27:22.523925 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:27:22.523890    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:27:22.550233 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:27:22.550195    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9"]
Nov 01 06:27:22.600153 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:27:22.600074    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9"]
Nov 01 06:27:23.246141 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:27:23.246107    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a763a5c5-de38-4a17-9cff-69cf629c1270" path="/var/lib/kubelet/pods/a763a5c5-de38-4a17-9cff-69cf629c1270/volumes"
Nov 01 06:27:52.523361 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:27:52.523279    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:28:04.382763 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:04.382730    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 06:28:04.391532 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:04.391510    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 06:28:04.395989 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:04.395965    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 06:28:04.404342 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:04.404325    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 06:28:22.524743 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:28:22.524704    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:28:23.770034 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:23.770000    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-rcf8t"]
Nov 01 06:28:23.770436 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:23.770383    4864 cpu_manager.go:414] "containerMap: removing container" podUID="a763a5c5-de38-4a17-9cff-69cf629c1270" containerName="manager"
Nov 01 06:28:23.770436 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:23.770395    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a763a5c5-de38-4a17-9cff-69cf629c1270" containerName="manager"
Nov 01 06:28:23.770511 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:23.770482    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="a763a5c5-de38-4a17-9cff-69cf629c1270" containerName="manager"
Nov 01 06:28:23.773559 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:23.773544    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-rcf8t"
Nov 01 06:28:23.826670 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:23.826638    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nrw9v\""
Nov 01 06:28:23.863686 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:23.863648    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4hm8\" (UniqueName: \"kubernetes.io/projected/a72bb36c-cebc-40e2-a995-3732b13f209f-kube-api-access-m4hm8\") pod \"kuadrant-operator-controller-manager-678d87766f-rcf8t\" (UID: \"a72bb36c-cebc-40e2-a995-3732b13f209f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-rcf8t"
Nov 01 06:28:23.863914 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:23.863702    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a72bb36c-cebc-40e2-a995-3732b13f209f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-rcf8t\" (UID: \"a72bb36c-cebc-40e2-a995-3732b13f209f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-rcf8t"
Nov 01 06:28:23.878410 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:23.878381    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-rcf8t"]
Nov 01 06:28:23.964948 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:23.964911    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4hm8\" (UniqueName: \"kubernetes.io/projected/a72bb36c-cebc-40e2-a995-3732b13f209f-kube-api-access-m4hm8\") pod \"kuadrant-operator-controller-manager-678d87766f-rcf8t\" (UID: \"a72bb36c-cebc-40e2-a995-3732b13f209f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-rcf8t"
Nov 01 06:28:23.965148 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:23.964975    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a72bb36c-cebc-40e2-a995-3732b13f209f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-rcf8t\" (UID: \"a72bb36c-cebc-40e2-a995-3732b13f209f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-rcf8t"
Nov 01 06:28:23.965347 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:23.965329    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a72bb36c-cebc-40e2-a995-3732b13f209f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-rcf8t\" (UID: \"a72bb36c-cebc-40e2-a995-3732b13f209f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-rcf8t"
Nov 01 06:28:24.025833 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:24.025725    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4hm8\" (UniqueName: \"kubernetes.io/projected/a72bb36c-cebc-40e2-a995-3732b13f209f-kube-api-access-m4hm8\") pod \"kuadrant-operator-controller-manager-678d87766f-rcf8t\" (UID: \"a72bb36c-cebc-40e2-a995-3732b13f209f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-rcf8t"
Nov 01 06:28:24.086848 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:24.086816    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-rcf8t"
Nov 01 06:28:24.511193 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:24.511171    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-rcf8t"]
Nov 01 06:28:24.513598 ip-10-0-134-253 kubenswrapper[4864]: W1101 06:28:24.513569    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda72bb36c_cebc_40e2_a995_3732b13f209f.slice/crio-3bb304eafb1e2e03bf091c52dd7d01efe3430e29a7219fc0c3cbe307300d5ac3 WatchSource:0}: Error finding container 3bb304eafb1e2e03bf091c52dd7d01efe3430e29a7219fc0c3cbe307300d5ac3: Status 404 returned error can't find the container with id 3bb304eafb1e2e03bf091c52dd7d01efe3430e29a7219fc0c3cbe307300d5ac3
Nov 01 06:28:24.671074 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:24.671039    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-rcf8t" event={"ID":"a72bb36c-cebc-40e2-a995-3732b13f209f","Type":"ContainerStarted","Data":"2736d86ecb3fd7fc68edcaf795c5c9cda89280cb34a820364b9b7aa53f60b3d2"}
Nov 01 06:28:24.671285 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:24.671092    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-rcf8t" event={"ID":"a72bb36c-cebc-40e2-a995-3732b13f209f","Type":"ContainerStarted","Data":"3bb304eafb1e2e03bf091c52dd7d01efe3430e29a7219fc0c3cbe307300d5ac3"}
Nov 01 06:28:24.671285 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:24.671119    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-rcf8t"
Nov 01 06:28:24.791134 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:24.791008    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-rcf8t" podStartSLOduration=1.7909917210000001 podStartE2EDuration="1.790991721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 06:28:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 06:28:24.789145069 +0000 UTC m=+5121.679521056" watchObservedRunningTime="2026-11-01 06:28:24.790991721 +0000 UTC m=+5121.681367696"
Nov 01 06:28:35.678759 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:35.678723    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-rcf8t"
Nov 01 06:28:47.361099 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:47.361059    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-xnxx8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 06:28:47.361878 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:28:47.361839    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8" podUID="5de1abde-348b-480d-8172-dc9a677eb592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 06:28:52.524507 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:28:52.524469    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:29:22.524592 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:29:22.524554    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:29:33.945149 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:33.945106    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn"]
Nov 01 06:29:33.948906 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:33.948884    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn"
Nov 01 06:29:34.048261 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:34.048222    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1393126-3933-4956-9cb7-60e0a7aca5d2-openshift-service-ca\") pod \"maas-api-5cdb759fdc-drcvn\" (UID: \"f1393126-3933-4956-9cb7-60e0a7aca5d2\") " pod="odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn"
Nov 01 06:29:34.048261 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:34.048262    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmpf6\" (UniqueName: \"kubernetes.io/projected/f1393126-3933-4956-9cb7-60e0a7aca5d2-kube-api-access-vmpf6\") pod \"maas-api-5cdb759fdc-drcvn\" (UID: \"f1393126-3933-4956-9cb7-60e0a7aca5d2\") " pod="odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn"
Nov 01 06:29:34.048473 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:34.048381    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1393126-3933-4956-9cb7-60e0a7aca5d2-metrics-certs\") pod \"maas-api-5cdb759fdc-drcvn\" (UID: \"f1393126-3933-4956-9cb7-60e0a7aca5d2\") " pod="odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn"
Nov 01 06:29:34.048473 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:34.048425    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1393126-3933-4956-9cb7-60e0a7aca5d2-maas-api-tls\") pod \"maas-api-5cdb759fdc-drcvn\" (UID: \"f1393126-3933-4956-9cb7-60e0a7aca5d2\") " pod="odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn"
Nov 01 06:29:34.052967 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:34.052942    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn"]
Nov 01 06:29:34.149689 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:34.149661    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1393126-3933-4956-9cb7-60e0a7aca5d2-maas-api-tls\") pod \"maas-api-5cdb759fdc-drcvn\" (UID: \"f1393126-3933-4956-9cb7-60e0a7aca5d2\") " pod="odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn"
Nov 01 06:29:34.149890 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:34.149740    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1393126-3933-4956-9cb7-60e0a7aca5d2-openshift-service-ca\") pod \"maas-api-5cdb759fdc-drcvn\" (UID: \"f1393126-3933-4956-9cb7-60e0a7aca5d2\") " pod="odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn"
Nov 01 06:29:34.149890 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:34.149756    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmpf6\" (UniqueName: \"kubernetes.io/projected/f1393126-3933-4956-9cb7-60e0a7aca5d2-kube-api-access-vmpf6\") pod \"maas-api-5cdb759fdc-drcvn\" (UID: \"f1393126-3933-4956-9cb7-60e0a7aca5d2\") " pod="odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn"
Nov 01 06:29:34.149890 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:34.149820    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1393126-3933-4956-9cb7-60e0a7aca5d2-metrics-certs\") pod \"maas-api-5cdb759fdc-drcvn\" (UID: \"f1393126-3933-4956-9cb7-60e0a7aca5d2\") " pod="odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn"
Nov 01 06:29:34.150490 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:34.150467    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1393126-3933-4956-9cb7-60e0a7aca5d2-openshift-service-ca\") pod \"maas-api-5cdb759fdc-drcvn\" (UID: \"f1393126-3933-4956-9cb7-60e0a7aca5d2\") " pod="odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn"
Nov 01 06:29:34.152309 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:34.152283    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1393126-3933-4956-9cb7-60e0a7aca5d2-metrics-certs\") pod \"maas-api-5cdb759fdc-drcvn\" (UID: \"f1393126-3933-4956-9cb7-60e0a7aca5d2\") " pod="odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn"
Nov 01 06:29:34.152453 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:34.152432    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1393126-3933-4956-9cb7-60e0a7aca5d2-maas-api-tls\") pod \"maas-api-5cdb759fdc-drcvn\" (UID: \"f1393126-3933-4956-9cb7-60e0a7aca5d2\") " pod="odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn"
Nov 01 06:29:34.210866 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:34.210778    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmpf6\" (UniqueName: \"kubernetes.io/projected/f1393126-3933-4956-9cb7-60e0a7aca5d2-kube-api-access-vmpf6\") pod \"maas-api-5cdb759fdc-drcvn\" (UID: \"f1393126-3933-4956-9cb7-60e0a7aca5d2\") " pod="odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn"
Nov 01 06:29:34.261905 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:34.261867    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn"
Nov 01 06:29:34.638151 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:34.638123    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn"]
Nov 01 06:29:34.640291 ip-10-0-134-253 kubenswrapper[4864]: W1101 06:29:34.640261    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1393126_3933_4956_9cb7_60e0a7aca5d2.slice/crio-c5bb2fad59a9d10ec5cec724a6633d057bfdb99bf8b877169b2ef0db837a09bb WatchSource:0}: Error finding container c5bb2fad59a9d10ec5cec724a6633d057bfdb99bf8b877169b2ef0db837a09bb: Status 404 returned error can't find the container with id c5bb2fad59a9d10ec5cec724a6633d057bfdb99bf8b877169b2ef0db837a09bb
Nov 01 06:29:34.641637 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:34.641615    4864 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 06:29:35.109507 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:35.109468    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn" event={"ID":"f1393126-3933-4956-9cb7-60e0a7aca5d2","Type":"ContainerStarted","Data":"c5bb2fad59a9d10ec5cec724a6633d057bfdb99bf8b877169b2ef0db837a09bb"}
Nov 01 06:29:36.118034 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:36.117992    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn" event={"ID":"f1393126-3933-4956-9cb7-60e0a7aca5d2","Type":"ContainerStarted","Data":"29d53e9cfd39603b996b69037750c16c40c5fbdd149600d64b4b3f9e6f335694"}
Nov 01 06:29:36.118514 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:36.118070    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn"
Nov 01 06:29:36.236825 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:36.236742    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn" podStartSLOduration=2.410248896 podStartE2EDuration="3.236726705s" totalImagesPullingTime="826.477809ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 06:29:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 06:29:36.234085985 +0000 UTC m=+5193.124461958" watchObservedRunningTime="2026-11-01 06:29:36.236726705 +0000 UTC m=+5193.127102722"
Nov 01 06:29:40.243962 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:40.243918    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-ck46t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 06:29:40.244490 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:40.243993    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t" podUID="92982041-1d56-4b3d-8067-3d393da5cad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 06:29:42.129594 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:42.129553    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5cdb759fdc-drcvn"
Nov 01 06:29:42.299284 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:42.299247    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"]
Nov 01 06:29:43.162600 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:43.162534    4864 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj" podUID="4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6" containerName="maas-api" containerID="cri-o://eeca14d8025ad0179799015d1d4d4f6c900b0e6b34c1cb9ff11b2d7ef4282bed" gracePeriod=30
Nov 01 06:29:44.730531 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:44.728983    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"
Nov 01 06:29:44.730531 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:44.729730    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-maas-api-tls\") pod \"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6\" (UID: \"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6\") "
Nov 01 06:29:44.730531 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:44.729778    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-kube-api-access-97cl5\" (UniqueName: \"kubernetes.io/projected/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-kube-api-access-97cl5\") pod \"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6\" (UID: \"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6\") "
Nov 01 06:29:44.730531 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:44.729958    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-metrics-certs\") pod \"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6\" (UID: \"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6\") "
Nov 01 06:29:44.730531 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:44.729998    4864 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-openshift-service-ca\") pod \"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6\" (UID: \"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6\") "
Nov 01 06:29:44.732026 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:44.731724    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-openshift-service-ca" pod "4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6" (UID: "4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 06:29:44.732701 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:44.732662    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-maas-api-tls" pod "4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6" (UID: "4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 06:29:44.734599 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:44.734552    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-kube-api-access-97cl5" pod "4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6" (UID: "4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6"). InnerVolumeSpecName "kube-api-access-97cl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 06:29:44.735423 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:44.735390    4864 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-metrics-certs" pod "4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6" (UID: "4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 06:29:44.835848 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:44.832386    4864 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-metrics-certs\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:29:44.835848 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:44.832429    4864 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-openshift-service-ca\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:29:44.835848 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:44.832447    4864 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-maas-api-tls\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:29:44.835848 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:44.832463    4864 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97cl5\" (UniqueName: \"kubernetes.io/projected/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6-kube-api-access-97cl5\") on node \"ip-10-0-134-253.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 06:29:45.221878 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:45.221259    4864 generic.go:356] "Generic (PLEG): container finished" podID="4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6" containerID="eeca14d8025ad0179799015d1d4d4f6c900b0e6b34c1cb9ff11b2d7ef4282bed" exitCode=0
Nov 01 06:29:45.221878 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:45.221361    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj" event={"ID":"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6","Type":"ContainerDied","Data":"eeca14d8025ad0179799015d1d4d4f6c900b0e6b34c1cb9ff11b2d7ef4282bed"}
Nov 01 06:29:45.221878 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:45.221415    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj" event={"ID":"4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6","Type":"ContainerDied","Data":"00e3507a0c91ecfa48e0ee15421cebe7a79307c3104be2a1dfc115411fa1202f"}
Nov 01 06:29:45.221878 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:45.221446    4864 scope.go:122] "RemoveContainer" containerID="eeca14d8025ad0179799015d1d4d4f6c900b0e6b34c1cb9ff11b2d7ef4282bed"
Nov 01 06:29:45.221878 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:45.221621    4864 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"
Nov 01 06:29:45.242919 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:45.242882    4864 scope.go:122] "RemoveContainer" containerID="eeca14d8025ad0179799015d1d4d4f6c900b0e6b34c1cb9ff11b2d7ef4282bed"
Nov 01 06:29:45.243238 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:29:45.243217    4864 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeca14d8025ad0179799015d1d4d4f6c900b0e6b34c1cb9ff11b2d7ef4282bed\": container with ID starting with eeca14d8025ad0179799015d1d4d4f6c900b0e6b34c1cb9ff11b2d7ef4282bed not found: ID does not exist" containerID="eeca14d8025ad0179799015d1d4d4f6c900b0e6b34c1cb9ff11b2d7ef4282bed"
Nov 01 06:29:45.243327 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:45.243251    4864 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeca14d8025ad0179799015d1d4d4f6c900b0e6b34c1cb9ff11b2d7ef4282bed"} err="failed to get container status \"eeca14d8025ad0179799015d1d4d4f6c900b0e6b34c1cb9ff11b2d7ef4282bed\": rpc error: code = NotFound desc = could not find container \"eeca14d8025ad0179799015d1d4d4f6c900b0e6b34c1cb9ff11b2d7ef4282bed\": container with ID starting with eeca14d8025ad0179799015d1d4d4f6c900b0e6b34c1cb9ff11b2d7ef4282bed not found: ID does not exist"
Nov 01 06:29:45.434811 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:45.433206    4864 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"]
Nov 01 06:29:45.444818 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:45.444345    4864 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b479cdcf5-rlphj"]
Nov 01 06:29:47.248108 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:29:47.248076    4864 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6" path="/var/lib/kubelet/pods/4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6/volumes"
Nov 01 06:29:52.527087 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:29:52.526963    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:30:22.523980 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:30:22.523945    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:30:52.524240 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:30:52.524157    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:31:03.455949 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:31:03.455905    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-p22bd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 06:31:03.456472 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:31:03.455985    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd" podUID="575b4414-cbde-452b-bbe4-46be0265d77f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 06:31:22.524890 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:31:22.524767    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:31:52.550489 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:31:52.550452    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:32:22.525169 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:32:22.525129    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:32:52.523766 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:32:52.523722    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:33:04.421386 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:33:04.421354    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 06:33:04.429999 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:33:04.429974    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 06:33:04.436908 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:33:04.436881    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cksgs8_a52e528e-c723-42b6-8d1a-03c3675518b9/extract/0.log"
Nov 01 06:33:04.444841 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:33:04.444823    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-6twtw_878896d2-fa82-46ef-af5e-0bf491216b03/console-operator/2.log"
Nov 01 06:33:22.527668 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:33:22.527626    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:33:52.524066 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:33:52.523985    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:34:22.524461 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:34:22.524422    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:34:52.556519 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:34:52.556484    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:35:03.466005 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:35:03.465872    4864 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58d898bd55-p22bd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 06:35:03.466005 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:35:03.465955    4864 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd" podUID="575b4414-cbde-452b-bbe4-46be0265d77f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 06:35:22.538498 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:35:22.538453    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:35:52.523486 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:35:52.523449    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:36:04.056234 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:04.056203    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-8qncv_39569001-5722-4b9e-ac79-84952518f6ca/manager/0.log"
Nov 01 06:36:04.364074 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:04.363980    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-8krz2_f4aae4c0-bc4d-4889-92e6-b5e4d22d7a06/manager/0.log"
Nov 01 06:36:05.250850 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:05.250817    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-h2pkm_3340fa76-a621-451b-8c96-82b31e6b40d1/server/0.log"
Nov 01 06:36:05.730748 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:05.730699    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-cdt6d_c3884e26-630f-4983-abc1-6dc5e9cb922a/proxy/0.log"
Nov 01 06:36:07.147414 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:07.147383    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5cdb759fdc-drcvn_f1393126-3933-4956-9cb7-60e0a7aca5d2/maas-api/0.log"
Nov 01 06:36:13.613625 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:13.613588    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-k2nkj_cb788d11-cc64-4f1c-b1e0-0bed74754f03/manager/0.log"
Nov 01 06:36:13.840304 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:13.840270    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-9t2hp_194e1c98-db83-4d11-b279-b43783313267/manager/0.log"
Nov 01 06:36:14.065412 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:14.065385    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-crc2n_60ec3670-705a-45e2-9dbe-e71edd5fb2fc/kuadrant-console-plugin/0.log"
Nov 01 06:36:14.491350 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:14.491258    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-rcf8t_a72bb36c-cebc-40e2-a995-3732b13f209f/manager/0.log"
Nov 01 06:36:14.951293 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:14.951263    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-wttv8_af89e1f3-fb0d-44e2-b35f-748885de9474/manager/0.log"
Nov 01 06:36:15.960516 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:15.960483    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9_6dd8055d-05e5-4204-85ac-3ed22dc9b493/istio-proxy/0.log"
Nov 01 06:36:18.150572 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:18.150530    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58d898bd55-ck46t_92982041-1d56-4b3d-8067-3d393da5cad5/istio-proxy/1.log"
Nov 01 06:36:18.714905 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:18.714875    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58d898bd55-p22bd_575b4414-cbde-452b-bbe4-46be0265d77f/istio-proxy/0.log"
Nov 01 06:36:19.558010 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:19.557978    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58d898bd55-xnxx8_5de1abde-348b-480d-8172-dc9a677eb592/istio-proxy/0.log"
Nov 01 06:36:19.836126 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:19.836050    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v_8860f9dd-4607-488f-9f91-045a50877f36/istio-proxy/0.log"
Nov 01 06:36:20.490016 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:20.489987    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74dd76c985-n79rt_82335b9e-5584-4afc-b1fc-cc003dbcad4b/router/0.log"
Nov 01 06:36:21.211009 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:21.210979    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58d898bd55-ck46t_92982041-1d56-4b3d-8067-3d393da5cad5/istio-proxy/0.log"
Nov 01 06:36:22.524191 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:36:22.524154    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:36:33.980646 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:33.980612    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b9nzz/must-gather-4572q"]
Nov 01 06:36:33.983162 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:33.981095    4864 cpu_manager.go:414] "containerMap: removing container" podUID="4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6" containerName="maas-api"
Nov 01 06:36:33.983162 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:33.981108    4864 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6" containerName="maas-api"
Nov 01 06:36:33.983162 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:33.981185    4864 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ea7d2c8-500a-4d5c-8c47-6d12ccdceec6" containerName="maas-api"
Nov 01 06:36:33.984391 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:33.984372    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b9nzz/must-gather-4572q"
Nov 01 06:36:34.042049 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:34.042021    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b9nzz/must-gather-4572q"]
Nov 01 06:36:34.088533 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:34.088502    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b9nzz\"/\"kube-root-ca.crt\""
Nov 01 06:36:34.088533 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:34.088512    4864 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b9nzz\"/\"openshift-service-ca.crt\""
Nov 01 06:36:34.088728 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:34.088540    4864 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-b9nzz\"/\"default-dockercfg-87f45\""
Nov 01 06:36:34.130510 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:34.130485    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8z7v\" (UniqueName: \"kubernetes.io/projected/e3185335-e7dd-4650-9030-63e3b6a4af7e-kube-api-access-s8z7v\") pod \"must-gather-4572q\" (UID: \"e3185335-e7dd-4650-9030-63e3b6a4af7e\") " pod="openshift-must-gather-b9nzz/must-gather-4572q"
Nov 01 06:36:34.130674 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:34.130546    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e3185335-e7dd-4650-9030-63e3b6a4af7e-must-gather-output\") pod \"must-gather-4572q\" (UID: \"e3185335-e7dd-4650-9030-63e3b6a4af7e\") " pod="openshift-must-gather-b9nzz/must-gather-4572q"
Nov 01 06:36:34.231856 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:34.231758    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8z7v\" (UniqueName: \"kubernetes.io/projected/e3185335-e7dd-4650-9030-63e3b6a4af7e-kube-api-access-s8z7v\") pod \"must-gather-4572q\" (UID: \"e3185335-e7dd-4650-9030-63e3b6a4af7e\") " pod="openshift-must-gather-b9nzz/must-gather-4572q"
Nov 01 06:36:34.231856 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:34.231848    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e3185335-e7dd-4650-9030-63e3b6a4af7e-must-gather-output\") pod \"must-gather-4572q\" (UID: \"e3185335-e7dd-4650-9030-63e3b6a4af7e\") " pod="openshift-must-gather-b9nzz/must-gather-4572q"
Nov 01 06:36:34.232232 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:34.232208    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e3185335-e7dd-4650-9030-63e3b6a4af7e-must-gather-output\") pod \"must-gather-4572q\" (UID: \"e3185335-e7dd-4650-9030-63e3b6a4af7e\") " pod="openshift-must-gather-b9nzz/must-gather-4572q"
Nov 01 06:36:34.292574 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:34.292536    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8z7v\" (UniqueName: \"kubernetes.io/projected/e3185335-e7dd-4650-9030-63e3b6a4af7e-kube-api-access-s8z7v\") pod \"must-gather-4572q\" (UID: \"e3185335-e7dd-4650-9030-63e3b6a4af7e\") " pod="openshift-must-gather-b9nzz/must-gather-4572q"
Nov 01 06:36:34.298353 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:34.298328    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b9nzz/must-gather-4572q"
Nov 01 06:36:34.682830 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:34.673930    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b9nzz/must-gather-4572q"]
Nov 01 06:36:34.682830 ip-10-0-134-253 kubenswrapper[4864]: W1101 06:36:34.682230    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3185335_e7dd_4650_9030_63e3b6a4af7e.slice/crio-1025824bd195ab34a626332fd608e56d1baf43e59ee3db1dd8dfaaab341b39fb WatchSource:0}: Error finding container 1025824bd195ab34a626332fd608e56d1baf43e59ee3db1dd8dfaaab341b39fb: Status 404 returned error can't find the container with id 1025824bd195ab34a626332fd608e56d1baf43e59ee3db1dd8dfaaab341b39fb
Nov 01 06:36:34.684172 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:34.684131    4864 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 06:36:34.949315 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:34.949225    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b9nzz/must-gather-4572q" event={"ID":"e3185335-e7dd-4650-9030-63e3b6a4af7e","Type":"ContainerStarted","Data":"1025824bd195ab34a626332fd608e56d1baf43e59ee3db1dd8dfaaab341b39fb"}
Nov 01 06:36:36.962578 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:36.962534    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b9nzz/must-gather-4572q" event={"ID":"e3185335-e7dd-4650-9030-63e3b6a4af7e","Type":"ContainerStarted","Data":"6393ef77b99b39ea91391c348fdddfda3b5fd0b83112c48be0ffd19715e653e5"}
Nov 01 06:36:36.963009 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:36.962589    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b9nzz/must-gather-4572q" event={"ID":"e3185335-e7dd-4650-9030-63e3b6a4af7e","Type":"ContainerStarted","Data":"7f5d28a9045fe81f2d4c53f1a3c5f01046aaaa62e2e24cdfb61e62b5997d9034"}
Nov 01 06:36:37.077470 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:37.077410    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-b9nzz/must-gather-4572q" podStartSLOduration=2.798661178 podStartE2EDuration="4.077394869s" totalImagesPullingTime="1.278733691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 06:36:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 06:36:37.077080111 +0000 UTC m=+5613.967456078" watchObservedRunningTime="2026-11-01 06:36:37.077394869 +0000 UTC m=+5613.967770846"
Nov 01 06:36:44.063860 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:44.059458    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-q92dh_36a4e628-5f12-4849-9d77-deaa3e93b190/global-pull-secret-syncer/0.log"
Nov 01 06:36:44.678819 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:44.675443    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-d2cm4_fb78a5ab-3a2d-45fa-abaa-7b26c33ec929/connection-checker/0.log"
Nov 01 06:36:45.101463 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:45.101427    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-qb994_df7d0108-685b-4c27-8d0f-bccf4ca5dbbe/connection-checker/0.log"
Nov 01 06:36:45.523486 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:45.523412    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-qszhw_b91ced44-8be6-47c8-9229-549400076eb7/connection-checker/0.log"
Nov 01 06:36:46.581647 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:46.581620    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jswq7_45a40b3e-ab72-4b18-933f-f52272a6742c/konnectivity-agent/0.log"
Nov 01 06:36:46.761806 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:46.761397    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-8qncv_39569001-5722-4b9e-ac79-84952518f6ca/manager/0.log"
Nov 01 06:36:47.310213 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:47.310177    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-8krz2_f4aae4c0-bc4d-4889-92e6-b5e4d22d7a06/manager/0.log"
Nov 01 06:36:47.744593 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:47.744516    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-h2pkm_3340fa76-a621-451b-8c96-82b31e6b40d1/server/0.log"
Nov 01 06:36:47.856036 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:47.855997    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-253.us-west-1.compute.internal_1c1427a4a27caa7c2e9df3986def0a2a/haproxy/0.log"
Nov 01 06:36:48.939632 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:48.939598    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-cdt6d_c3884e26-630f-4983-abc1-6dc5e9cb922a/proxy/0.log"
Nov 01 06:36:52.428838 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:52.428774    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-8qncv_39569001-5722-4b9e-ac79-84952518f6ca/manager/0.log"
Nov 01 06:36:52.531988 ip-10-0-134-253 kubenswrapper[4864]: E1101 06:36:52.531852    4864 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-b6sl7" containerName="extractor"
Nov 01 06:36:52.910206 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:52.910174    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-8krz2_f4aae4c0-bc4d-4889-92e6-b5e4d22d7a06/manager/0.log"
Nov 01 06:36:53.184765 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:53.184696    4864 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"]
Nov 01 06:36:53.188585 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:53.188566    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"
Nov 01 06:36:53.281524 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:53.281482    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"]
Nov 01 06:36:53.326964 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:53.326827    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b235cf18-7304-43d6-9fae-8cd38b3a1c98-sys\") pod \"perf-node-gather-daemonset-fznhg\" (UID: \"b235cf18-7304-43d6-9fae-8cd38b3a1c98\") " pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"
Nov 01 06:36:53.327287 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:53.327241    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bmbp\" (UniqueName: \"kubernetes.io/projected/b235cf18-7304-43d6-9fae-8cd38b3a1c98-kube-api-access-5bmbp\") pod \"perf-node-gather-daemonset-fznhg\" (UID: \"b235cf18-7304-43d6-9fae-8cd38b3a1c98\") " pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"
Nov 01 06:36:53.327537 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:53.327513    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b235cf18-7304-43d6-9fae-8cd38b3a1c98-lib-modules\") pod \"perf-node-gather-daemonset-fznhg\" (UID: \"b235cf18-7304-43d6-9fae-8cd38b3a1c98\") " pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"
Nov 01 06:36:53.327750 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:53.327731    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b235cf18-7304-43d6-9fae-8cd38b3a1c98-proc\") pod \"perf-node-gather-daemonset-fznhg\" (UID: \"b235cf18-7304-43d6-9fae-8cd38b3a1c98\") " pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"
Nov 01 06:36:53.327987 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:53.327968    4864 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b235cf18-7304-43d6-9fae-8cd38b3a1c98-podres\") pod \"perf-node-gather-daemonset-fznhg\" (UID: \"b235cf18-7304-43d6-9fae-8cd38b3a1c98\") " pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"
Nov 01 06:36:53.428738 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:53.428704    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b235cf18-7304-43d6-9fae-8cd38b3a1c98-sys\") pod \"perf-node-gather-daemonset-fznhg\" (UID: \"b235cf18-7304-43d6-9fae-8cd38b3a1c98\") " pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"
Nov 01 06:36:53.428738 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:53.428739    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bmbp\" (UniqueName: \"kubernetes.io/projected/b235cf18-7304-43d6-9fae-8cd38b3a1c98-kube-api-access-5bmbp\") pod \"perf-node-gather-daemonset-fznhg\" (UID: \"b235cf18-7304-43d6-9fae-8cd38b3a1c98\") " pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"
Nov 01 06:36:53.429188 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:53.428762    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b235cf18-7304-43d6-9fae-8cd38b3a1c98-lib-modules\") pod \"perf-node-gather-daemonset-fznhg\" (UID: \"b235cf18-7304-43d6-9fae-8cd38b3a1c98\") " pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"
Nov 01 06:36:53.429188 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:53.428813    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b235cf18-7304-43d6-9fae-8cd38b3a1c98-proc\") pod \"perf-node-gather-daemonset-fznhg\" (UID: \"b235cf18-7304-43d6-9fae-8cd38b3a1c98\") " pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"
Nov 01 06:36:53.429188 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:53.428836    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b235cf18-7304-43d6-9fae-8cd38b3a1c98-sys\") pod \"perf-node-gather-daemonset-fznhg\" (UID: \"b235cf18-7304-43d6-9fae-8cd38b3a1c98\") " pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"
Nov 01 06:36:53.429188 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:53.428845    4864 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b235cf18-7304-43d6-9fae-8cd38b3a1c98-podres\") pod \"perf-node-gather-daemonset-fznhg\" (UID: \"b235cf18-7304-43d6-9fae-8cd38b3a1c98\") " pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"
Nov 01 06:36:53.429188 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:53.428919    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b235cf18-7304-43d6-9fae-8cd38b3a1c98-proc\") pod \"perf-node-gather-daemonset-fznhg\" (UID: \"b235cf18-7304-43d6-9fae-8cd38b3a1c98\") " pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"
Nov 01 06:36:53.429188 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:53.428929    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b235cf18-7304-43d6-9fae-8cd38b3a1c98-podres\") pod \"perf-node-gather-daemonset-fznhg\" (UID: \"b235cf18-7304-43d6-9fae-8cd38b3a1c98\") " pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"
Nov 01 06:36:53.429188 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:53.428946    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b235cf18-7304-43d6-9fae-8cd38b3a1c98-lib-modules\") pod \"perf-node-gather-daemonset-fznhg\" (UID: \"b235cf18-7304-43d6-9fae-8cd38b3a1c98\") " pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"
Nov 01 06:36:53.490358 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:53.490285    4864 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bmbp\" (UniqueName: \"kubernetes.io/projected/b235cf18-7304-43d6-9fae-8cd38b3a1c98-kube-api-access-5bmbp\") pod \"perf-node-gather-daemonset-fznhg\" (UID: \"b235cf18-7304-43d6-9fae-8cd38b3a1c98\") " pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"
Nov 01 06:36:53.504481 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:53.504456    4864 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"
Nov 01 06:36:53.676492 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:53.676465    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-h2pkm_3340fa76-a621-451b-8c96-82b31e6b40d1/server/0.log"
Nov 01 06:36:53.888572 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:53.888492    4864 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"]
Nov 01 06:36:53.891054 ip-10-0-134-253 kubenswrapper[4864]: W1101 06:36:53.891018    4864 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb235cf18_7304_43d6_9fae_8cd38b3a1c98.slice/crio-87d60aaf2387f2bbb414d22d19e35d771a677446104deb1b6fb54825e907fa3b WatchSource:0}: Error finding container 87d60aaf2387f2bbb414d22d19e35d771a677446104deb1b6fb54825e907fa3b: Status 404 returned error can't find the container with id 87d60aaf2387f2bbb414d22d19e35d771a677446104deb1b6fb54825e907fa3b
Nov 01 06:36:54.135332 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:54.135290    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg" event={"ID":"b235cf18-7304-43d6-9fae-8cd38b3a1c98","Type":"ContainerStarted","Data":"266e0015da5899cdf1e8cc0f7d723f4188de20e79fc07e1843feb40fe70962a7"}
Nov 01 06:36:54.135518 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:54.135358    4864 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg" event={"ID":"b235cf18-7304-43d6-9fae-8cd38b3a1c98","Type":"ContainerStarted","Data":"87d60aaf2387f2bbb414d22d19e35d771a677446104deb1b6fb54825e907fa3b"}
Nov 01 06:36:54.135518 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:54.135383    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"
Nov 01 06:36:54.252852 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:54.252738    4864 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg" podStartSLOduration=1.252717862 podStartE2EDuration="1.252717862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 06:36:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 06:36:54.250371491 +0000 UTC m=+5631.140747466" watchObservedRunningTime="2026-11-01 06:36:54.252717862 +0000 UTC m=+5631.143093837"
Nov 01 06:36:54.295359 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:54.295305    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-k2nkj_cb788d11-cc64-4f1c-b1e0-0bed74754f03/manager/0.log"
Nov 01 06:36:54.710232 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:54.710203    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-9t2hp_194e1c98-db83-4d11-b279-b43783313267/manager/0.log"
Nov 01 06:36:55.125912 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:55.125885    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-crc2n_60ec3670-705a-45e2-9dbe-e71edd5fb2fc/kuadrant-console-plugin/0.log"
Nov 01 06:36:55.330852 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:55.330813    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-cdt6d_c3884e26-630f-4983-abc1-6dc5e9cb922a/proxy/0.log"
Nov 01 06:36:55.616442 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:55.616412    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-rcf8t_a72bb36c-cebc-40e2-a995-3732b13f209f/manager/0.log"
Nov 01 06:36:56.446232 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:56.446201    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-wttv8_af89e1f3-fb0d-44e2-b35f-748885de9474/manager/0.log"
Nov 01 06:36:58.711414 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:58.711360    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-8qncv_39569001-5722-4b9e-ac79-84952518f6ca/manager/0.log"
Nov 01 06:36:59.193189 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:59.193154    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-8krz2_f4aae4c0-bc4d-4889-92e6-b5e4d22d7a06/manager/0.log"
Nov 01 06:36:59.779508 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:36:59.779484    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-h2pkm_3340fa76-a621-451b-8c96-82b31e6b40d1/server/0.log"
Nov 01 06:37:00.155297 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:37:00.155263    4864 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-b9nzz/perf-node-gather-daemonset-fznhg"
Nov 01 06:37:01.453092 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:37:01.453060    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-cdt6d_c3884e26-630f-4983-abc1-6dc5e9cb922a/proxy/0.log"
Nov 01 06:37:01.935601 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:37:01.935568    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-mtms9_c0b1d57c-ca0d-4dfe-8ba4-98668bd12b47/cluster-monitoring-operator/0.log"
Nov 01 06:37:04.423360 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:37:04.423329    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tpp66_2db64172-f3dd-498b-a505-47477423806c/node-exporter/0.log"
Nov 01 06:37:04.836045 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:37:04.836018    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tpp66_2db64172-f3dd-498b-a505-47477423806c/kube-rbac-proxy/0.log"
Nov 01 06:37:04.897154 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:37:04.897127    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9_6dd8055d-05e5-4204-85ac-3ed22dc9b493/istio-proxy/0.log"
Nov 01 06:37:05.249821 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:37:05.249722    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tpp66_2db64172-f3dd-498b-a505-47477423806c/init-textfile/0.log"
Nov 01 06:37:06.971906 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:37:06.971858    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58d898bd55-ck46t_92982041-1d56-4b3d-8067-3d393da5cad5/istio-proxy/0.log"
Nov 01 06:37:06.975600 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:37:06.975575    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58d898bd55-ck46t_92982041-1d56-4b3d-8067-3d393da5cad5/istio-proxy/1.log"
Nov 01 06:37:07.308835 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:37:07.308807    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58d898bd55-p22bd_575b4414-cbde-452b-bbe4-46be0265d77f/istio-proxy/0.log"
Nov 01 06:37:07.952038 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:37:07.952000    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58d898bd55-xnxx8_5de1abde-348b-480d-8172-dc9a677eb592/istio-proxy/0.log"
Nov 01 06:37:08.164530 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:37:08.164502    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v_8860f9dd-4607-488f-9f91-045a50877f36/istio-proxy/0.log"
Nov 01 06:37:08.805653 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:37:08.805623    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74dd76c985-n79rt_82335b9e-5584-4afc-b1fc-cc003dbcad4b/router/0.log"
Nov 01 06:37:14.933136 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:37:14.933104    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-k2nkj_cb788d11-cc64-4f1c-b1e0-0bed74754f03/manager/0.log"
Nov 01 06:37:15.143516 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:37:15.143481    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-9t2hp_194e1c98-db83-4d11-b279-b43783313267/manager/0.log"
Nov 01 06:37:15.354658 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:37:15.354629    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-crc2n_60ec3670-705a-45e2-9dbe-e71edd5fb2fc/kuadrant-console-plugin/0.log"
Nov 01 06:37:15.650898 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:37:15.650812    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-rcf8t_a72bb36c-cebc-40e2-a995-3732b13f209f/manager/0.log"
Nov 01 06:37:16.049550 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:37:16.049467    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-wttv8_af89e1f3-fb0d-44e2-b35f-748885de9474/manager/0.log"
Nov 01 06:37:16.501646 ip-10-0-134-253 kubenswrapper[4864]: I1101 06:37:16.501614    4864 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9_6dd8055d-05e5-4204-85ac-3ed22dc9b493/istio-proxy/0.log"